Class LLM

Hierarchy (view full)

Constructors

constructor

Accessors

model provider

Methods

aclose chat label prewarm updateOptions

Constructors

constructor

Accessors

model

  • get model(): string

  • Get the model name/identifier for this LLM instance.

    Returns string

    The model name if available, "unknown" otherwise.

    Remarks

    Plugins should override this property to provide their model information.

provider

  • get provider(): string

  • Get the provider name for this LLM instance.

    Returns string

    The provider name if available, "unknown" otherwise.

    Remarks

    Plugins should override this property to provide their provider information.

Methods

aclose

  • aclose(): Promise<void>

  • Returns Promise<void>

chat

label

prewarm

  • prewarm(): void

  • Pre-warm connection to the LLM service.

    Establishes DNS resolution and the TLS connection to the provider before the first inference request, reducing time-to-first-token on the initial reply. Non-blocking (fire-and-forget) and idempotent; calls made after aclose are ignored. Providers enable it by overriding _prewarmImpl.

    Returns void

updateOptions

  • updateOptions(opts): void

  • Parameters

    • opts: {
          frequencyPenalty?: number;
          maxCompletionTokens?: number;
          model?: string;
          presencePenalty?: number;
          randomSeed?: number;
          temperature?: number;
          toolChoice?: llm.ToolChoice;
          topP?: number;
      }
      • Optional frequencyPenalty?: number
      • Optional maxCompletionTokens?: number
      • Optional model?: string
      • Optional presencePenalty?: number
      • Optional randomSeed?: number
      • Optional temperature?: number
      • Optional toolChoice?: llm.ToolChoice
      • Optional topP?: number

    Returns void

Settings

Member Visibility

Theme

On This Page

constructormodelprovideraclosechatlabelprewarmupdateOptions