Class LLM

Hierarchy (view full)

Constructors

constructor

Accessors

model provider

Methods

aclose chat label prewarm withAzure withCerebras withDeepSeek withFireworks withGroq withMeta withOVHcloud withOcto withOllama withPerplexity withTelnyx withTogether withXAI

Constructors

constructor

  • new LLM(opts?, providerFmt?): LLM

  • Create a new instance of OpenAI LLM.

    Parameters

    Returns LLM

    Remarks

    apiKey must be set to your OpenAI API key, either using the argument or by setting the OPENAI_API_KEY environment variable.

Accessors

model

  • get model(): string

  • Get the model name/identifier for this LLM instance.

    Returns string

    The model name if available, "unknown" otherwise.

    Remarks

    Plugins should override this property to provide their model information.

provider

  • get provider(): string

  • Get the provider name for this LLM instance.

    Returns string

    The provider name if available, "unknown" otherwise.

    Remarks

    Plugins should override this property to provide their provider information.

Methods

aclose

  • aclose(): Promise<void>

  • Returns Promise<void>

chat

label

prewarm

  • prewarm(): void

  • Pre-warm connection to the LLM service

    Returns void

Static withAzure

  • withAzure(opts?): LLM

  • Create a new instance of OpenAI LLM with Azure.

    Parameters

    • opts: {
          apiKey?: string;
          apiVersion?: string;
          azureAdToken?: string;
          azureAdTokenProvider?: (() => Promise<string>);
          azureDeployment?: string;
          azureEndpoint?: string;
          baseURL?: string;
          model: string;
          organization?: string;
          project?: string;
          temperature?: number;
          user?: string;
      } = defaultAzureLLMOptions
      • Optional apiKey?: string
      • Optional apiVersion?: string
      • Optional azureAdToken?: string
      • Optional azureAdTokenProvider?: (() => Promise<string>)
          • (): Promise<string>

          • Returns Promise<string>

      • Optional azureDeployment?: string
      • Optional azureEndpoint?: string
      • Optional baseURL?: string
      • model: string
      • Optional organization?: string
      • Optional project?: string
      • Optional temperature?: number
      • Optional user?: string

    Returns LLM

    Remarks

    This automatically infers the following arguments from their corresponding environment variables if they are not provided:

    • apiKey from AZURE_OPENAI_API_KEY
    • organization from OPENAI_ORG_ID
    • project from OPENAI_PROJECT_ID
    • azureAdToken from AZURE_OPENAI_AD_TOKEN
    • apiVersion from OPENAI_API_VERSION
    • azureEndpoint from AZURE_OPENAI_ENDPOINT

Static withCerebras

  • withCerebras(opts?): LLM

  • Create a new instance of Cerebras LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your Cerebras API key, either using the argument or by setting the CEREBRAS_API_KEY environment variable.

Static withDeepSeek

  • withDeepSeek(opts?): LLM

  • Create a new instance of DeepSeek LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your DeepSeek API key, either using the argument or by setting the DEEPSEEK_API_KEY environment variable.

Static withFireworks

  • withFireworks(opts?): LLM

  • Create a new instance of Fireworks LLM.

    Parameters

    Returns LLM

    Remarks

    apiKey must be set to your Fireworks API key, either using the argument or by setting the FIREWORKS_API_KEY environment variable.

Static withGroq

  • withGroq(opts?): LLM

  • Create a new instance of Groq LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your Groq API key, either using the argument or by setting the GROQ_API_KEY environment variable.

Static withMeta

  • withMeta(opts?): LLM

  • Create a new instance of Meta Llama LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client?: OpenAI;
          model?: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your Meta Llama API key, either using the argument or by setting the LLAMA_API_KEY environment variable.

Static withOVHcloud

  • withOVHcloud(opts?): LLM

  • Create a new instance of OVHcloud AI Endpoints LLM.

    Parameters

    Returns LLM

    Remarks

    apiKey must be set to your OVHcloud AI Endpoints API key, either using the argument or by setting the OVHCLOUD_API_KEY environment variable.

Static withOcto

  • withOcto(opts?): LLM

  • Create a new instance of OctoAI LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your OctoAI API key, either using the argument or by setting the OCTOAI_TOKEN environment variable.

Static withOllama

  • withOllama(opts?): LLM

  • Create a new instance of Ollama LLM.

    Parameters

    • opts: Partial<{
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
      }> = {}

    Returns LLM

Static withPerplexity

  • withPerplexity(opts?): LLM

  • Create a new instance of PerplexityAI LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your PerplexityAI API key, either using the argument or by setting the PERPLEXITY_API_KEY environment variable.

Static withTelnyx

  • withTelnyx(opts?): LLM

  • Create a new instance of Telnyx LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your Telnyx API key, either using the argument or by setting the TELNYX_API_KEY environment variable.

Static withTogether

  • withTogether(opts?): LLM

  • Create a new instance of TogetherAI LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your TogetherAI API key, either using the argument or by setting the TOGETHER_API_KEY environment variable.

Static withXAI

  • withXAI(opts?): LLM

  • Create a new instance of xAI LLM.

    Parameters

    • opts: Partial<{
          apiKey?: string;
          baseURL?: string;
          client: OpenAI;
          model: string;
          temperature?: number;
          user?: string;
      }> = {}

    Returns LLM

    Remarks

    apiKey must be set to your xAI API key, either using the argument or by setting the XAI_API_KEY environment variable.

Settings

Member Visibility

Theme

On This Page

constructormodelprovideraclosechatlabelprewarmwithAzurewithCerebraswithDeepSeekwithFireworkswithGroqwithMetawithOVHcloudwithOctowithOllamawithPerplexitywithTelnyxwithTogetherwithXAI