Class LocalClient

Client for local OpenAI-compatible inference servers.

Supports chat, streaming, tool calling, and structured output via the same interface as OpenAIClient. The server is expected to implement the OpenAI Chat Completions API at the given baseUrl.

Example

// From env vars (LOCAL_BASE_URL + LOCAL_API_KEY)
const client = createLocalClient({
  baseUrl: process.env.LOCAL_BASE_URL!,
  apiKey: process.env.LOCAL_API_KEY,
  defaultModel: 'Qwen3.5-9B-MLX-4bit',
});

// Chat
const reply = await client.chatSimple([
  { role: 'user', content: 'Hello!' }
]);

// Stream
for await (const delta of client.streamContent([
  { role: 'user', content: 'Tell me a story' }
])) {
  process.stdout.write(delta.content);
}

// List models served by the local server
const ids = await client.listModelIds();

Hierarchy (View Summary)

OpenAIClient
- LocalClient

Constructors

constructor

new LocalClient(config: LocalClientConfig): LocalClient
Parameters
- config: LocalClientConfig
Returns LocalClient
Overrides OpenAIClient.constructor
- Defined in providers/local/client.ts:66

Properties

cosineSimilarity

cosineSimilarity: (a: number[], b: number[]) => number = cosineSimilarity

Calculate cosine similarity between two vectors.

Type Declaration

- (a: number[], b: number[]): number
- Calculate cosine similarity between two vectors.
  Parameters
  - a: number[]
    First vector
  - b: number[]
    Second vector
  Returns number
  Cosine similarity (-1 to 1)

findSimilar

findSimilar: <T>(
    query: number[],
    candidates: { embedding: number[]; data: T }[],
    topK?: number,
) => { data: T; score: number }[] = findSimilar

Find similar items from a corpus.

Type Declaration

- <T>(
      query: number[],
      candidates: { embedding: number[]; data: T }[],
      topK?: number,
  ): { data: T; score: number }[]
- Find most similar items from a list of embeddings.
  Type Parameters
  - T
  Parameters
  - query: number[]
    Query embedding
  - candidates: { embedding: number[]; data: T }[]
    Array of candidate embeddings with associated data
  - topK: number = 5
    Number of results to return
  Returns { data: T; score: number }[]
  Top K most similar items with scores

Methods

chat

chat(
messages: Message[],
options?: Partial<ChatOptions>,
): Promise<ChatCompletion>
Create a chat completion.
Parameters
- messages: Message[]
- Optionaloptions: Partial<ChatOptions>
Returns Promise<ChatCompletion>
Inherited from OpenAIClient.chat
- Defined in providers/openai/client.ts:158

chatSimple

chatSimple(messages: Message[], options?: Partial<ChatOptions>): Promise<string>
Create a chat completion and return just the content.
Parameters
- messages: Message[]
- Optionaloptions: Partial<ChatOptions>
Returns Promise<string>
Inherited from OpenAIClient.chatSimple
- Defined in providers/openai/client.ts:165

chatWithTools

chatWithTools(
    messages: Message[],
    tools: Tool[],
    options?: Partial<ChatOptions>,
): Promise<
    | { type: "content"; content: string; usage: Usage }
    | { type: "tool_calls"; toolCalls: ToolCall[]; usage: Usage },
>
Chat with tool handling.
Parameters
- messages: Message[]
- tools: Tool[]
- Optionaloptions: Partial<ChatOptions>
Returns Promise<
| { type: "content"; content: string; usage: Usage }
| { type: "tool_calls"; toolCalls: ToolCall[]; usage: Usage },
>
Inherited from OpenAIClient.chatWithTools
- Defined in providers/openai/client.ts:172

chatStructured

chatStructured<T>(
    messages: Message[],
    schema: {
        name: string;
        description?: string;
        schema: Record<string, unknown>;
        strict?: boolean;
    },
    options?: Partial<ChatOptions>,
): Promise<T>
Chat with structured output.
Type Parameters
- T
Parameters
- messages: Message[]
- schema: {
      name: string;
      description?: string;
      schema: Record<string, unknown>;
      strict?: boolean;
  }
- Optionaloptions: Partial<ChatOptions>
Returns Promise<T>
Inherited from OpenAIClient.chatStructured
- Defined in providers/openai/client.ts:182

stream

stream(
messages: Message[],
options?: Partial<ChatStreamOptions>,
): AsyncGenerator<ChatCompletionChunk, void, undefined>
Stream chat completion chunks.
Parameters
- messages: Message[]
- Optionaloptions: Partial<ChatStreamOptions>
Returns AsyncGenerator<ChatCompletionChunk, void, undefined>
Inherited from OpenAIClient.stream
- Defined in providers/openai/client.ts:202

streamContent

streamContent(
messages: Message[],
options?: Partial<ChatStreamOptions>,
): AsyncGenerator<StreamDelta, void, undefined>
Stream content deltas.
Parameters
- messages: Message[]
- Optionaloptions: Partial<ChatStreamOptions>
Returns AsyncGenerator<StreamDelta, void, undefined>
Inherited from OpenAIClient.streamContent
- Defined in providers/openai/client.ts:216

streamAccumulate

streamAccumulate(
    messages: Message[],
    options?: Partial<ChatStreamOptions>,
    onChunk?: (delta: { content: string; chunk: ChatCompletionChunk }) => void,
): Promise<
    {
        content: string;
        toolCalls: {
            id: string;
            type: "function";
            function: { name: string; arguments: string };
        }[];
        finishReason: string
        | null;
        usage: Usage | null;
        model: string;
        id: string;
    },
>
Stream and accumulate full response.
Parameters
- messages: Message[]
- Optionaloptions: Partial<ChatStreamOptions>
- OptionalonChunk: (delta: { content: string; chunk: ChatCompletionChunk }) => void
Returns Promise<
    {
        content: string;
        toolCalls: {
            id: string;
            type: "function";
            function: { name: string; arguments: string };
        }[];
        finishReason: string
        | null;
        usage: Usage | null;
        model: string;
        id: string;
    },
>
Inherited from OpenAIClient.streamAccumulate
- Defined in providers/openai/client.ts:227

streamReadable

streamReadable(
messages: Message[],
options?: Partial<ChatStreamOptions>,
): ReadableStream<string>
Create a readable stream of content.
Parameters
- messages: Message[]
- Optionaloptions: Partial<ChatStreamOptions>
Returns ReadableStream<string>
Inherited from OpenAIClient.streamReadable
- Defined in providers/openai/client.ts:246

embed

embed(
input: string | string[],
options?: Partial<EmbeddingOptions>,
): Promise<EmbeddingResponse>
Create embeddings.
Parameters
- input: string | string[]
- Optionaloptions: Partial<EmbeddingOptions>
Returns Promise<EmbeddingResponse>
Inherited from OpenAIClient.embed
- Defined in providers/openai/client.ts:264

embedOne

embedOne(input: string, options?: Partial<EmbeddingOptions>): Promise<number[]>
Create embedding for single text.
Parameters
- input: string
- Optionaloptions: Partial<EmbeddingOptions>
Returns Promise<number[]>
Inherited from OpenAIClient.embedOne
- Defined in providers/openai/client.ts:274

embedMany

embedMany(
inputs: string[],
options?: Partial<EmbeddingOptions>,
): Promise<number[][]>
Create embeddings for multiple texts.
Parameters
- inputs: string[]
- Optionaloptions: Partial<EmbeddingOptions>
Returns Promise<number[][]>
Inherited from OpenAIClient.embedMany
- Defined in providers/openai/client.ts:281

generateImage

generateImage(
prompt: string,
options?: Partial<ImageGenerationOptions>,
): Promise<ImageGenerationResponse>
Generate images.
Parameters
- prompt: string
- Optionaloptions: Partial<ImageGenerationOptions>
Returns Promise<ImageGenerationResponse>
Inherited from OpenAIClient.generateImage
- Defined in providers/openai/client.ts:292

generateImageUrl

generateImageUrl(
prompt: string,
options?: Partial<ImageGenerationOptions>,
): Promise<string>
Generate a single image URL.
Parameters
- prompt: string
- Optionaloptions: Partial<ImageGenerationOptions>
Returns Promise<string>
Inherited from OpenAIClient.generateImageUrl
- Defined in providers/openai/client.ts:302

generateImageBase64

generateImageBase64(
prompt: string,
options?: Partial<ImageGenerationOptions>,
): Promise<string>
Generate a single image as base64.
Parameters
- prompt: string
- Optionaloptions: Partial<ImageGenerationOptions>
Returns Promise<string>
Inherited from OpenAIClient.generateImageBase64
- Defined in providers/openai/client.ts:312

generateImages

generateImages(
    prompt: string,
    count: number,
    options?: Partial<ImageGenerationOptions>,
): Promise<ImageData[]>
Generate multiple images.
Parameters
- prompt: string
- count: number
- Optionaloptions: Partial<ImageGenerationOptions>
Returns Promise<ImageData[]>
Inherited from OpenAIClient.generateImages
- Defined in providers/openai/client.ts:322

listModels

listModels(): Promise<ModelsResponse>
List all available models.

Returns Promise<ModelsResponse>
Inherited from OpenAIClient.listModels
- Defined in providers/openai/client.ts:333

getModel

getModel(modelId: string): Promise<Model>
Get model details.
Parameters
- modelId: string
Returns Promise<Model>
Inherited from OpenAIClient.getModel
- Defined in providers/openai/client.ts:340

listModelIds

listModelIds(): Promise<string[]>
List model IDs.

Returns Promise<string[]>
Inherited from OpenAIClient.listModelIds
- Defined in providers/openai/client.ts:347

modelExists

modelExists(modelId: string): Promise<boolean>
Check if a model exists.
Parameters
- modelId: string
Returns Promise<boolean>
Inherited from OpenAIClient.modelExists
- Defined in providers/openai/client.ts:354

Class LocalClient

Example

Hierarchy (View Summary)

Index

Constructors

Properties

Methods

Constructors

constructor

Parameters

Returns LocalClient

Properties

cosineSimilarity

Type Declaration

Parameters

Returns number

findSimilar

Type Declaration

Type Parameters

Parameters

Returns { data: T; score: number }[]

Methods

chat

Parameters

Returns Promise<ChatCompletion>

chatSimple

Parameters

Returns Promise<string>

chatWithTools

Parameters

Returns Promise< | { type: "content"; content: string; usage: Usage } | { type: "tool_calls"; toolCalls: ToolCall[]; usage: Usage },>

chatStructured

Type Parameters

Parameters

Returns Promise<T>

stream

Parameters

Returns AsyncGenerator<ChatCompletionChunk, void, undefined>

streamContent

Parameters

Returns AsyncGenerator<StreamDelta, void, undefined>

streamAccumulate

Parameters

Returns Promise< { content: string; toolCalls: { id: string; type: "function"; function: { name: string; arguments: string }; }[]; finishReason: string | null; usage: Usage | null; model: string; id: string; },>

streamReadable

Parameters

Returns ReadableStream<string>

embed

Parameters

Returns Promise<EmbeddingResponse>

embedOne

Parameters

Returns Promise<number[]>

embedMany

Parameters

Returns Promise<number[][]>

generateImage

Parameters

Returns Promise<ImageGenerationResponse>

generateImageUrl

Parameters

Returns Promise<string>

generateImageBase64

Parameters

Returns Promise<string>

generateImages

Parameters

Returns Promise<ImageData[]>

listModels

Returns Promise<ModelsResponse>

getModel

Parameters

Returns Promise<Model>

listModelIds

Returns Promise<string[]>

modelExists

Parameters

Returns Promise<boolean>

Settings

On This Page

Returns Promise<
| { type: "content"; content: string; usage: Usage }
| { type: "tool_calls"; toolCalls: ToolCall[]; usage: Usage },
>

Returns Promise<
{
content: string;
toolCalls: {
id: string;
type: "function";
function: { name: string; arguments: string };
}[];
finishReason: string
| null;
usage: Usage | null;
model: string;
id: string;
},
>