Class LLM

Livekit Cloud Inference LLM

Hierarchy (view full)

llm.LLM
- LLM

Index

Constructors

constructor

new LLM(opts): inference.LLM
Parameters
- opts: {
      apiKey?: string;
      apiSecret?: string;
      baseURL?: string;
      inferenceClass?: InferenceClass;
      model: LLMModels;
      modelOptions?: ChatCompletionOptions;
      provider?: string;
      strictToolSchema?: boolean;
  }
  - Optional apiKey?: string
  - Optional apiSecret?: string
  - Optional baseURL?: string
  - Optional inferenceClass?: InferenceClass
  - model: LLMModels
  - Optional modelOptions?: ChatCompletionOptions
  - Optional provider?: string
  - Optional strictToolSchema?: boolean
Returns inference.LLM
Overrides LLM.constructor
- Defined in agents/src/inference/llm.ts:180

Accessors

model

get model(): string
Get the model name/identifier for this LLM instance.

Returns string
The model name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their model information.
Overrides llm.LLM.model
- Defined in agents/src/inference/llm.ts:238

provider

get provider(): string
Get the provider name for this LLM instance.

Returns string
The provider name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their provider information.
Overrides llm.LLM.provider
- Defined in agents/src/inference/llm.ts:242

Methods

aclose

aclose(): Promise<void>
Returns Promise<void>
Inherited from LLM.aclose
- Defined in agents/src/llm/llm.ts:108

chat

chat(__namedParameters): inference.LLMStream
Returns a LLMStream that can be used to push text and receive LLM responses.
Parameters
- __namedParameters: {
      chatCtx: ChatContext;
      connOptions?: APIConnectOptions;
      extraKwargs?: Record<string, unknown>;
      inferenceClass?: InferenceClass;
      parallelToolCalls?: boolean;
      toolChoice?: llm.ToolChoice;
      toolCtx?: ToolContext;
  }
  - chatCtx: ChatContext
  - Optional connOptions?: APIConnectOptions
  - Optional extraKwargs?: Record<string, unknown>
  - Optional inferenceClass?: InferenceClass
  - Optional parallelToolCalls?: boolean
  - Optional toolChoice?: llm.ToolChoice
  - Optional toolCtx?: ToolContext
Returns inference.LLMStream
Overrides LLM.chat
- Defined in agents/src/inference/llm.ts:271

label

label(): string
Returns string
Overrides LLM.label
- Defined in agents/src/inference/llm.ts:234

prewarm

prewarm(): void
Pre-warm connection to the LLM service

Returns void
Inherited from LLM.prewarm
- Defined in agents/src/llm/llm.ts:104

updateOptions

updateOptions(__namedParameters): void
Update LLM configuration options used by the next chat call.

modelOptions replaces the persistent options object instead of merging; pass {} to clear it.
Parameters
- __namedParameters: {
  model?: LLMModels;
  modelOptions?: ChatCompletionOptions;
  }
  - Optional model?: LLMModels
  - Optional modelOptions?: ChatCompletionOptions
Returns void
- Defined in agents/src/inference/llm.ts:256

`Static` fromModelString

fromModelString(modelString): inference.LLM
Parameters
- modelString: string
Returns inference.LLM
- Defined in agents/src/inference/llm.ts:246

Class LLM

Hierarchy (view full)

Index

Constructors

Accessors

Methods

Constructors

constructor

Parameters

Optional apiKey?: string

Optional apiSecret?: string

Optional baseURL?: string

Optional inferenceClass?: InferenceClass

model: LLMModels

Optional modelOptions?: ChatCompletionOptions

Optional provider?: string

Optional strictToolSchema?: boolean

Returns inference.LLM

Accessors

model

Returns string

Remarks

provider

Returns string

Remarks

Methods

aclose

Returns Promise<void>

chat

Parameters

chatCtx: ChatContext

Optional connOptions?: APIConnectOptions

Optional extraKwargs?: Record<string, unknown>

Optional inferenceClass?: InferenceClass

Optional parallelToolCalls?: boolean

Optional toolChoice?: llm.ToolChoice

Optional toolCtx?: ToolContext

Returns inference.LLMStream

label

Returns string

prewarm

Returns void

updateOptions

Parameters

Optional model?: LLMModels

Optional modelOptions?: ChatCompletionOptions

Returns void

Static fromModelString

Parameters

Returns inference.LLM

Settings

Member Visibility

Theme

On This Page

`Optional` apiKey?: string

`Optional` apiSecret?: string

`Optional` baseURL?: string

`Optional` inferenceClass?: InferenceClass

`Optional` modelOptions?: ChatCompletionOptions

`Optional` provider?: string

`Optional` strictToolSchema?: boolean

`Optional` connOptions?: APIConnectOptions

`Optional` extraKwargs?: Record<string, unknown>

`Optional` inferenceClass?: InferenceClass

`Optional` parallelToolCalls?: boolean

`Optional` toolChoice?: llm.ToolChoice

`Optional` toolCtx?: ToolContext

`Optional` model?: LLMModels

`Optional` modelOptions?: ChatCompletionOptions

`Static` fromModelString