Class LLM

Livekit Cloud Inference LLM

Hierarchy (view full)

llm.LLM
- LLM

Index

Constructors

constructor

new LLM(opts): inference.LLM
Parameters
- opts: {
      apiKey?: string;
      apiSecret?: string;
      baseURL?: string;
      model: LLMModels;
      modelOptions?: ChatCompletionOptions;
      provider?: string;
      strictToolSchema?: boolean;
  }
  - Optional apiKey?: string
  - Optional apiSecret?: string
  - Optional baseURL?: string
  - model: LLMModels
  - Optional modelOptions?: ChatCompletionOptions
  - Optional provider?: string
  - Optional strictToolSchema?: boolean
Returns inference.LLM
Overrides LLM.constructor
- Defined in agents/src/inference/llm.ts:104

Accessors

model

get model(): string
Get the model name/identifier for this LLM instance.

Returns string
The model name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their model information.
Overrides llm.LLM.model
- Defined in agents/src/inference/llm.ts:157

provider

get provider(): string
Get the provider name for this LLM instance.

Returns string
The provider name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their provider information.
Overrides llm.LLM.provider
- Defined in agents/src/inference/llm.ts:161

Methods

aclose

aclose(): Promise<void>
Returns Promise<void>
Inherited from LLM.aclose
- Defined in agents/src/llm/llm.ts:106

chat

chat(__namedParameters): inference.LLMStream
Returns a LLMStream that can be used to push text and receive LLM responses.
Parameters
- __namedParameters: {
      chatCtx: ChatContext;
      connOptions?: APIConnectOptions;
      extraKwargs?: Record<string, unknown>;
      parallelToolCalls?: boolean;
      toolChoice?: llm.ToolChoice;
      toolCtx?: ToolContext;
  }
  - chatCtx: ChatContext
  - Optional connOptions?: APIConnectOptions
  - Optional extraKwargs?: Record<string, unknown>
  - Optional parallelToolCalls?: boolean
  - Optional toolChoice?: llm.ToolChoice
  - Optional toolCtx?: ToolContext
Returns inference.LLMStream
Overrides LLM.chat
- Defined in agents/src/inference/llm.ts:169

label

label(): string
Returns string
Overrides LLM.label
- Defined in agents/src/inference/llm.ts:153

prewarm

prewarm(): void
Pre-warm connection to the LLM service

Returns void
Inherited from LLM.prewarm
- Defined in agents/src/llm/llm.ts:102

`Static` fromModelString

fromModelString(modelString): inference.LLM
Parameters
- modelString: string
Returns inference.LLM
- Defined in agents/src/inference/llm.ts:165

Class LLM

Hierarchy (view full)

Index

Constructors

Accessors

Methods

Constructors

constructor

Parameters

Optional apiKey?: string

Optional apiSecret?: string

Optional baseURL?: string

model: LLMModels

Optional modelOptions?: ChatCompletionOptions

Optional provider?: string

Optional strictToolSchema?: boolean

Returns inference.LLM

Accessors

model

Returns string

Remarks

provider

Returns string

Remarks

Methods

aclose

Returns Promise<void>

chat

Parameters

chatCtx: ChatContext

Optional connOptions?: APIConnectOptions

Optional extraKwargs?: Record<string, unknown>

Optional parallelToolCalls?: boolean

Optional toolChoice?: llm.ToolChoice

Optional toolCtx?: ToolContext

Returns inference.LLMStream

label

Returns string

prewarm

Returns void

Static fromModelString

Parameters

Returns inference.LLM

Settings

Member Visibility

Theme

On This Page

`Optional` apiKey?: string

`Optional` apiSecret?: string

`Optional` baseURL?: string

`Optional` modelOptions?: ChatCompletionOptions

`Optional` provider?: string

`Optional` strictToolSchema?: boolean

`Optional` connOptions?: APIConnectOptions

`Optional` extraKwargs?: Record<string, unknown>

`Optional` parallelToolCalls?: boolean

`Optional` toolChoice?: llm.ToolChoice

`Optional` toolCtx?: ToolContext

`Static` fromModelString