Class GPTLiveSession

GPT-Live WebSocket session, accessible through Agent.duplexSession. Also emits openai_server_event_received, openai_client_event_queued and delegation_created. Append methods queue context without waiting for acknowledgment. The appended events arrive at the estimated context-injection end and do not indicate speech completion.

Hierarchy (view full)

  • llm.DuplexSession<{
        delegation_created: ((event) => void);
        openai_client_event_queued: ((event) => void);
        openai_server_event_received: ((event) => void);
    }>
    • GPTLiveSession

    Constructors

    constructor

    Properties

    duplexModel

    Accessors

    audioStream capabilities sessionId tools

    Methods

    _appendItems _generateReply _updateInstructions _updateOptions _updateSession _updateTools appendCommentary appendInstructions appendThinking close muteInput pushAudio pushVideo sendEvent unmuteInput

    Constructors

    constructor

    Properties

    Readonly duplexModel

    duplexModel: llm.DuplexModel

    Model that created this session.

    Accessors

    audioStream

    • get audioStream(): ReadableStream<DuplexAudioFrame>

    • Mono 24 kHz PCM output, retained across reconnects and closed on shutdown.

      Returns ReadableStream<DuplexAudioFrame>

    capabilities

    • get capabilities(): DuplexCapabilities

    • Provider features declared by the owning model.

      Returns DuplexCapabilities

    sessionId

    • get sessionId(): undefined | string

    • Provider session identifier. Changes when a replacement connection starts.

      Returns undefined | string

    tools

    • get tools(): ToolContext<unknown>

    • Copy of the tools available to the backend Responses model.

      Returns ToolContext<unknown>

    Methods

    _appendItems

    • _appendItems(items): Promise<void>

    • Retain framework chat items for reconnects and send their context or tool results.

      Parameters

      Returns Promise<void>

    _generateReply

    • _generateReply(instructions?): void

    • Prompt an immediate spoken reply to instructions or the newest typed user message.

      Parameters

      • Optional instructions: string

      Returns void

    _updateInstructions

    • _updateInstructions(instructions): Promise<void>

    • Set the initial voice instructions. Changes after session.start are rejected.

      Parameters

      • instructions: string

      Returns Promise<void>

    _updateOptions

    • _updateOptions(options): void

    • Update backend tool selection for this session and future reconnects.

      Parameters

      Returns void

    _updateSession

    • _updateSession(instructions?, chatCtx?, tools?): Promise<void>

    • Apply the complete startup configuration before releasing a waiting provider connection.

      Parameters

      Returns Promise<void>

    _updateTools

    • _updateTools(tools): Promise<void>

    • Update backend tools. Client delegation requires an empty tool context.

      Parameters

      Returns Promise<void>

    appendCommentary

    • appendCommentary(text, options?): void

    • Give the model text to paraphrase aloud, or answer a client delegation. Capped at 500 tokens by the service.

      Parameters

      • text: string
      • Optional options: {
            delegationId?: null | string;
        }
        • Optional delegationId?: null | string

      Returns void

    appendInstructions

    • appendInstructions(text, options?): void

    • Add a standing rule retained in reconnect history. The service caps each append at 500 tokens.

      Parameters

      • text: string
      • Optional options: {
            delegationId?: null | string;
        }
        • Optional delegationId?: null | string

      Returns void

    appendThinking

    • appendThinking(text, options?): void

    • Add silent context retained in reconnect history. The service caps each append at 500 tokens.

      Parameters

      • text: string
      • Optional options: {
            delegationId?: null | string;
        }
        • Optional delegationId?: null | string

      Returns void

    close

    • close(): Promise<void>

    • Mark the session closing, release configuration waits, and close provider resources.

      Returns Promise<void>

    muteInput

    • muteInput(): void

    • Replace microphone input with silence while the model continues speaking. The state is remembered and re-applied to any session a reconnect opens.

      Returns void

    pushAudio

    • pushAudio(frame): void

    • Buffer microphone audio, mixing to mono and resampling to 24 kHz as needed.

      Parameters

      • frame: AudioFrame

      Returns void

    pushVideo

    • pushVideo(_frame): void

    • Feed an input video frame. Providers without video input can ignore it.

      Parameters

      • _frame: VideoFrame

      Returns void

    sendEvent

    • sendEvent(event): void

    • Queue a wire event. Commands wait for session.started before being sent.

      Parameters

      Returns void

    unmuteInput

    • unmuteInput(): void

    • Restore microphone input.

      Returns void

    Settings

    Member Visibility

    Theme

    On This Page

    constructorduplexModelaudioStreamcapabilitiessessionIdtools_appendItems_generateReply_updateInstructions_updateOptions_updateSession_updateToolsappendCommentaryappendInstructionsappendThinkingclosemuteInputpushAudiopushVideosendEventunmuteInput