Class FallbackAdapter

FallbackAdapter is an LLM that can fallback to a different LLM if the current LLM fails.

Example

const fallbackLLM = new FallbackAdapter({
  llms: [primaryLLM, secondaryLLM, tertiaryLLM],
  attemptTimeout: 5.0,
  maxRetryPerLLM: 1,
});

Hierarchy (view full)

llm.LLM
- FallbackAdapter

Index

Constructors

constructor

new FallbackAdapter(options): FallbackAdapter
Parameters
- options: FallbackAdapterOptions
Returns FallbackAdapter
Overrides LLM.constructor
- Defined in llm/fallback_adapter.ts:78

Properties

`Readonly` attemptTimeout

attemptTimeout: number

`Readonly` llms

llms: llm.LLM[]

`Readonly` maxRetryPerLLM

maxRetryPerLLM: number

`Readonly` retryInterval

retryInterval: number

`Readonly` retryOnChunkSent

retryOnChunkSent: boolean

Accessors

model

get model(): string
Get the model name/identifier for this LLM instance.

Returns string
The model name if available, "unknown" otherwise.

Remarks
Plugins should override this property to provide their model information.
Overrides LLM.model
- Defined in llm/fallback_adapter.ts:105

Methods

aclose

aclose(): Promise<void>
Returns Promise<void>
Inherited from LLM.aclose
- Defined in llm/llm.ts:94

chat

chat(opts): llm.LLMStream
Returns a LLMStream that can be used to push text and receive LLM responses.
Parameters
- opts: {
      chatCtx: ChatContext;
      connOptions?: APIConnectOptions;
      extraKwargs?: Record<string, unknown>;
      parallelToolCalls?: boolean;
      toolChoice?: llm.ToolChoice;
      toolCtx?: ToolContext;
  }
  - chatCtx: ChatContext
  - Optional connOptions?: APIConnectOptions
  - Optional extraKwargs?: Record<string, unknown>
  - Optional parallelToolCalls?: boolean
  - Optional toolChoice?: llm.ToolChoice
  - Optional toolCtx?: ToolContext
Returns llm.LLMStream
Overrides LLM.chat
- Defined in llm/fallback_adapter.ts:113

label

label(): string
Returns string
Overrides LLM.label
- Defined in llm/fallback_adapter.ts:109

prewarm

prewarm(): void
Pre-warm connection to the LLM service

Returns void
Inherited from LLM.prewarm
- Defined in llm/llm.ts:90

Class FallbackAdapter

Example

Hierarchy (view full)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns FallbackAdapter

Properties

Readonly attemptTimeout

Readonly llms

Readonly maxRetryPerLLM

Readonly retryInterval

Readonly retryOnChunkSent

Accessors

model

Returns string

Remarks

Methods

aclose

Returns Promise<void>

chat

Parameters

chatCtx: ChatContext

Optional connOptions?: APIConnectOptions

Optional extraKwargs?: Record<string, unknown>

Optional parallelToolCalls?: boolean

Optional toolChoice?: llm.ToolChoice

Optional toolCtx?: ToolContext

Returns llm.LLMStream

label

Returns string

prewarm

Returns void

Settings

Member Visibility

Theme

On This Page

`Readonly` attemptTimeout

`Readonly` llms

`Readonly` maxRetryPerLLM

`Readonly` retryInterval

`Readonly` retryOnChunkSent

`Optional` connOptions?: APIConnectOptions

`Optional` extraKwargs?: Record<string, unknown>

`Optional` parallelToolCalls?: boolean

`Optional` toolChoice?: llm.ToolChoice

`Optional` toolCtx?: ToolContext