Class FallbackAdapter

Agent Fallback Adapter for LLM. Manages multiple LLM instances with automatic fallback when the current LLM fails.

Example

const fallbackLLM = new FallbackAdapter({
  llms: [primaryLLM, secondaryLLM, tertiaryLLM],
  attemptTimeout: 5.0,
  maxRetryPerLLM: 1,
});

Hierarchy (view full)

llm.LLM
- FallbackAdapter

Index

Constructors

constructor

new FallbackAdapter(options): llm.FallbackAdapter
Parameters
- options: llm.FallbackAdapterOptions
Returns llm.FallbackAdapter
Overrides LLM.constructor
- Defined in agents/src/llm/fallback_adapter.ts:81

Properties

`Readonly` attemptTimeout

attemptTimeout: number

`Readonly` llms

llms: llm.LLM[]

`Readonly` maxRetryPerLLM

maxRetryPerLLM: number

`Readonly` retryInterval

retryInterval: number

`Readonly` retryOnChunkSent

retryOnChunkSent: boolean

Accessors

model

get model(): string
The model of the instance that serves next (see nextInstance). Spans and metrics read this, so a failover shows the model expected to answer rather than the adapter; the instance that actually served is stamped per request by the stream.

Returns string
Overrides LLM.model
- Defined in agents/src/llm/fallback_adapter.ts:124

provider

get provider(): string
The provider of the instance that serves next (see model).

Returns string
Overrides LLM.provider
- Defined in agents/src/llm/fallback_adapter.ts:129

Methods

aclose

aclose(): Promise<void>
Returns Promise<void>
Inherited from LLM.aclose
- Defined in agents/src/llm/llm.ts:180

chat

chat(opts): llm.LLMStream
Returns a LLMStream that can be used to push text and receive LLM responses.
Parameters
- opts: {
      chatCtx: ChatContext;
      connOptions?: APIConnectOptions;
      extraKwargs?: Record<string, unknown>;
      parallelToolCalls?: boolean;
      toolChoice?: llm.ToolChoice;
      toolCtx?: ToolContextLike;
  }
  - chatCtx: ChatContext
  - Optional connOptions?: APIConnectOptions
  - Optional extraKwargs?: Record<string, unknown>
  - Optional parallelToolCalls?: boolean
  - Optional toolChoice?: llm.ToolChoice
  - Optional toolCtx?: ToolContextLike
Returns llm.LLMStream
Overrides LLM.chat
- Defined in agents/src/llm/fallback_adapter.ts:137

label

label(): string
Returns string
Overrides LLM.label
- Defined in agents/src/llm/fallback_adapter.ts:133

prewarm

prewarm(): void
Pre-warm connection to the LLM service.

Establishes DNS resolution and the TLS connection to the provider before the first inference request, reducing time-to-first-token on the initial reply. Non-blocking (fire-and-forget) and idempotent; calls made after aclose are ignored. Providers enable it by overriding _prewarmImpl.

Returns void
Inherited from LLM.prewarm
- Defined in agents/src/llm/llm.ts:153

Class FallbackAdapter

Example

Hierarchy (view full)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns llm.FallbackAdapter

Properties

Readonly attemptTimeout

Readonly llms

Readonly maxRetryPerLLM

Readonly retryInterval

Readonly retryOnChunkSent

Accessors

model

Returns string

provider

Returns string

Methods

aclose

Returns Promise<void>

chat

Parameters

chatCtx: ChatContext

Optional connOptions?: APIConnectOptions

Optional extraKwargs?: Record<string, unknown>

Optional parallelToolCalls?: boolean

Optional toolChoice?: llm.ToolChoice

Optional toolCtx?: ToolContextLike

Returns llm.LLMStream

label

Returns string

prewarm

Returns void

Settings

Member Visibility

Theme

On This Page

`Readonly` attemptTimeout

`Readonly` llms

`Readonly` maxRetryPerLLM

`Readonly` retryInterval

`Readonly` retryOnChunkSent

`Optional` connOptions?: APIConnectOptions

`Optional` extraKwargs?: Record<string, unknown>

`Optional` parallelToolCalls?: boolean

`Optional` toolChoice?: llm.ToolChoice

`Optional` toolCtx?: ToolContextLike