Class RealtimeSession

A session for the OpenAI Realtime API.

This class is used to interact with the OpenAI Realtime API. It is responsible for sending events to the OpenAI Realtime API and receiving events from it.

It exposes two more events:

  • openai_server_event_received: expose the raw server events from the OpenAI Realtime API
  • openai_client_event_queued: expose the raw client events sent to the OpenAI Realtime API

Hierarchy (view full)

Constructors

constructor

Accessors

chatCtx realtimeModel tools

Methods

_updateSession clearAudio close commitAudio generateReply interrupt pushAudio sendEvent setInputAudioStream startUserActivity truncate updateChatCtx updateInstructions updateOptions updateTools

Constructors

constructor

Accessors

chatCtx

  • get chatCtx(): ChatContext

  • Returns ChatContext

realtimeModel

tools

  • get tools(): ToolContext<unknown>

  • Returns ToolContext<unknown>

Methods

_updateSession

  • _updateSession(instructions?, chatCtx?, tools?): Promise<void>

  • Parameters

    Returns Promise<void>

clearAudio

  • clearAudio(): Promise<void>

  • Clear the input audio buffer to the server

    Returns Promise<void>

close

  • close(): Promise<void>

  • Returns Promise<void>

commitAudio

  • commitAudio(): Promise<void>

  • Commit the input audio buffer to the server

    Returns Promise<void>

generateReply

  • generateReply(instructions?, options?): Promise<GenerationCreatedEvent>

  • Parameters

    • Optional instructions: string
    • Optional options: {
          signal?: AbortSignal;
      }
      • Optional signal?: AbortSignal

    Returns Promise<GenerationCreatedEvent>

    Throws

    RealtimeError on Timeout

interrupt

  • interrupt(): Promise<void>

  • Cancel the current generation (do nothing if no generation is in progress)

    Returns Promise<void>

pushAudio

  • pushAudio(frame): void

  • Parameters

    • frame: AudioFrame

    Returns void

sendEvent

  • sendEvent(command): void

  • Parameters

    Returns void

setInputAudioStream

  • setInputAudioStream(audioStream): void

  • Parameters

    • audioStream: ReadableStream<AudioFrame>

    Returns void

startUserActivity

  • startUserActivity(): void

  • Notifies the model that user activity has started

    Returns void

truncate

  • truncate(_options): Promise<void>

  • Truncate the message at the given audio end time

    Parameters

    • _options: {
          audioEndMs: number;
          audioTranscript?: string;
          messageId: string;
          modalities?: Modality[];
      }
      • audioEndMs: number
      • Optional audioTranscript?: string
      • messageId: string
      • Optional modalities?: Modality[]

    Returns Promise<void>

updateChatCtx

  • updateChatCtx(_chatCtx): Promise<void>

  • Parameters

    Returns Promise<void>

    Throws

    RealtimeError on Timeout

updateInstructions

  • updateInstructions(_instructions): Promise<void>

  • Parameters

    • _instructions: string

    Returns Promise<void>

updateOptions

updateTools

  • updateTools(_tools): Promise<void>

  • Parameters

    Returns Promise<void>

Settings

Member Visibility

Theme

On This Page

constructorchatCtxrealtimeModeltools_updateSessionclearAudioclosecommitAudiogenerateReplyinterruptpushAudiosendEventsetInputAudioStreamstartUserActivitytruncateupdateChatCtxupdateInstructionsupdateOptionsupdateTools