Class LLM

Livekit Cloud Inference LLM

Hierarchy (view full)

llm.LLM
- LLM

Index

Constructors

Accessors

Methods

aclose chat label prewarm fromModelString

Constructors

constructor

new LLM(opts): inference.LLM
Parameters
- opts: {
      apiKey?: string;
      apiSecret?: string;
      baseURL?: string;
      model: LLMModels;
      modelOptions?: ChatCompletionOptions;
      provider?: string;
  }
  - Optional apiKey?: string
  - Optional apiSecret?: string
  - Optional baseURL?: string
  - model: LLMModels
  - Optional modelOptions?: ChatCompletionOptions
  - Optional provider?: string
Returns inference.LLM
Overrides LLM.constructor
- Defined in inference/llm.ts:103

Accessors

model

get model(): string
Get the model name/identifier for this LLM instance.

Returns string
The model name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their model information.
Overrides llm.LLM.model
- Defined in inference/llm.ts:147

Methods

aclose

aclose(): Promise<void>
Returns Promise<void>
Inherited from LLM.aclose
- Defined in llm/llm.ts:91

chat

chat(__namedParameters): inference.LLMStream
Returns a LLMStream that can be used to push text and receive LLM responses.
Parameters
- __namedParameters: {
      chatCtx: ChatContext;
      connOptions?: APIConnectOptions;
      extraKwargs?: Record<string, unknown>;
      parallelToolCalls?: boolean;
      toolChoice?: llm.ToolChoice;
      toolCtx?: ToolContext;
  }
  - chatCtx: ChatContext
  - Optional connOptions?: APIConnectOptions
  - Optional extraKwargs?: Record<string, unknown>
  - Optional parallelToolCalls?: boolean
  - Optional toolChoice?: llm.ToolChoice
  - Optional toolCtx?: ToolContext
Returns inference.LLMStream
Overrides LLM.chat
- Defined in inference/llm.ts:155

label

label(): string
Returns string
Overrides LLM.label
- Defined in inference/llm.ts:143

prewarm

prewarm(): void
Pre-warm connection to the LLM service

Returns void
Inherited from LLM.prewarm
- Defined in llm/llm.ts:87

`Static` fromModelString

fromModelString(modelString): inference.LLM
Parameters
- modelString: string
Returns inference.LLM
- Defined in inference/llm.ts:151