Class LLM`Abstract`

Hierarchy (view full)

TypedEventEmitter<LLMCallbacks, this>
- LLM
  - inference.LLM

Index

Constructors

Accessors

Methods

aclose chat label prewarm

Constructors

constructor

new LLM(): llm.LLM
Returns llm.LLM
Overrides (EventEmitter as new () => TypedEmitter<LLMCallbacks>).constructor
- Defined in llm/llm.ts:47

Accessors

model

get model(): string
Get the model name/identifier for this LLM instance.

Returns string
The model name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their model information.
- Defined in llm/llm.ts:61

Methods

aclose

aclose(): Promise<void>
Returns Promise<void>
- Defined in llm/llm.ts:91

`Abstract` chat

chat(__namedParameters): llm.LLMStream
Returns a LLMStream that can be used to push text and receive LLM responses.
Parameters
- __namedParameters: {
      chatCtx: ChatContext;
      connOptions?: APIConnectOptions;
      extraKwargs?: Record<string, unknown>;
      parallelToolCalls?: boolean;
      toolChoice?: llm.ToolChoice;
      toolCtx?: ToolContext;
  }
  - chatCtx: ChatContext
  - Optional connOptions?: APIConnectOptions
  - Optional extraKwargs?: Record<string, unknown>
  - Optional parallelToolCalls?: boolean
  - Optional toolChoice?: llm.ToolChoice
  - Optional toolCtx?: ToolContext
Returns llm.LLMStream
- Defined in llm/llm.ts:68

`Abstract` label

label(): string
Returns string
- Defined in llm/llm.ts:51

prewarm

prewarm(): void
Pre-warm connection to the LLM service

Returns void
- Defined in llm/llm.ts:87