Class RealtimeSession

A session for the OpenAI Realtime API.

This class is used to interact with the OpenAI Realtime API. It is responsible for sending events to the OpenAI Realtime API and receiving events from it.

It exposes two more events:

openai_server_event_received: expose the raw server events from the OpenAI Realtime API
openai_client_event_queued: expose the raw client events sent to the OpenAI Realtime API

Hierarchy (view full)

realtime.RealtimeSession
- RealtimeSession

Index

Constructors

constructor

new RealtimeSession(realtimeModel, options?): inference.RealtimeSession
Parameters
- realtimeModel: inference.RealtimeModel
- Optional options: {
  turnDetectionDisabled?: boolean;
  }
  - Optional turnDetectionDisabled?: boolean
Returns inference.RealtimeSession
Overrides RealtimeSession.constructor
- Defined in agents/src/inference/realtime.ts:194

Accessors

chatCtx

get chatCtx(): ChatContext
Returns ChatContext
Inherited from OpenAIRealtimeSession.chatCtx
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:588

realtimeModel

get realtimeModel(): llm.RealtimeModel
Returns llm.RealtimeModel
Inherited from OpenAIRealtimeSession.realtimeModel
- Defined in agents/src/llm/realtime.ts:148

tools

get tools(): ToolContext<unknown>
Returns ToolContext<unknown>
Inherited from OpenAIRealtimeSession.tools
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:592

Methods

_updateSession

_updateSession(instructions?, chatCtx?, tools?): Promise<void>
Parameters
- Optional instructions: string
- Optional chatCtx: ChatContext
- Optional tools: ToolContext<unknown>
Returns Promise<void>
Inherited from RealtimeSession._updateSession
- Defined in agents/src/llm/realtime.ts:202

clearAudio

clearAudio(): Promise<void>
Clear the input audio buffer to the server

Returns Promise<void>
Inherited from RealtimeSession.clearAudio
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:927

close

close(): Promise<void>
Returns Promise<void>
Overrides RealtimeSession.close
- Defined in agents/src/inference/realtime.ts:207

commitAudio

commitAudio(): Promise<void>
Commit the input audio buffer to the server

Returns Promise<void>
Inherited from RealtimeSession.commitAudio
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:915

generateReply

generateReply(instructions?, options?): Promise<GenerationCreatedEvent>
Parameters
- Optional instructions: string
- options: {
  signal?: AbortSignal;
  } = {}
  - Optional signal?: AbortSignal
Returns Promise<GenerationCreatedEvent>
Throws
RealtimeError on Timeout
Inherited from RealtimeSession.generateReply
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:935

interrupt

interrupt(): Promise<void>
Cancel the current generation (do nothing if no generation is in progress)

Returns Promise<void>
Overrides RealtimeSession.interrupt
- Defined in agents/src/inference/realtime.ts:264

pushAudio

pushAudio(frame): void
Parameters
- frame: AudioFrame
Returns void
Inherited from RealtimeSession.pushAudio
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:896

sendEvent

sendEvent(command): void
Parameters
- command: realtime.ClientEvent
Returns void
Inherited from RealtimeSession.sendEvent
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:524

setInputAudioStream

setInputAudioStream(audioStream): void
Parameters
- audioStream: ReadableStream<AudioFrame>
Returns void
Inherited from RealtimeSession.setInputAudioStream
- Defined in agents/src/llm/realtime.ts:253

startUserActivity

startUserActivity(): void
Notifies the model that user activity has started

Returns void
Inherited from RealtimeSession.startUserActivity
- Defined in agents/src/llm/realtime.ts:238

truncate

truncate(_options): Promise<void>
Truncate the message at the given audio end time
Parameters
- _options: {
      audioEndMs: number;
      audioTranscript?: string;
      messageId: string;
      modalities?: Modality[];
  }
  - audioEndMs: number
  - Optional audioTranscript?: string
  - messageId: string
  - Optional modalities?: Modality[]
Returns Promise<void>
Inherited from RealtimeSession.truncate
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:993

updateChatCtx

updateChatCtx(_chatCtx): Promise<void>
Parameters
- _chatCtx: ChatContext
Returns Promise<void>
Throws
RealtimeError on Timeout
Inherited from RealtimeSession.updateChatCtx
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:596

updateInstructions

updateInstructions(_instructions): Promise<void>
Parameters
- _instructions: string
Returns Promise<void>
Inherited from RealtimeSession.updateInstructions
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:857

updateOptions

updateOptions(__namedParameters): void
Parameters
- __namedParameters: {
  toolChoice?: llm.ToolChoice;
  }
  - Optional toolChoice?: llm.ToolChoice
Returns void
Inherited from RealtimeSession.updateOptions
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:871

updateTools

updateTools(_tools): Promise<void>
Parameters
- _tools: ToolContext<unknown>
Returns Promise<void>
Inherited from RealtimeSession.updateTools
- Defined in agents/src/llm/openai_realtime/realtime_model.ts:797

Class RealtimeSession

Hierarchy (view full)

Index

Constructors

Accessors

Methods

Constructors

constructor

Parameters

Optional turnDetectionDisabled?: boolean

Returns inference.RealtimeSession

Accessors

chatCtx

Returns ChatContext

realtimeModel

Returns llm.RealtimeModel

tools

Returns ToolContext<unknown>

Methods

_updateSession

Parameters

Returns Promise<void>

clearAudio

Returns Promise<void>

close

Returns Promise<void>

commitAudio

Returns Promise<void>

generateReply

Parameters

Optional signal?: AbortSignal

Returns Promise<GenerationCreatedEvent>

Throws

interrupt

Returns Promise<void>

pushAudio

Parameters

Returns void

sendEvent

Parameters

Returns void

setInputAudioStream

Parameters

Returns void

startUserActivity

Returns void

truncate

Parameters

audioEndMs: number

Optional audioTranscript?: string

messageId: string

Optional modalities?: Modality[]

Returns Promise<void>

updateChatCtx

Parameters

Returns Promise<void>

Throws

updateInstructions

Parameters

Returns Promise<void>

updateOptions

Parameters

Optional toolChoice?: llm.ToolChoice

Returns void

updateTools

Parameters

Returns Promise<void>

Settings

Member Visibility

Theme

On This Page

`Optional` turnDetectionDisabled?: boolean

`Optional` signal?: AbortSignal

`Optional` audioTranscript?: string

`Optional` modalities?: Modality[]

`Optional` toolChoice?: llm.ToolChoice