Class RealtimeSession

A session for the OpenAI Realtime API.

This class is used to interact with the OpenAI Realtime API. It is responsible for sending events to the OpenAI Realtime API and receiving events from it.

It exposes two more events:

openai_server_event_received: expose the raw server events from the OpenAI Realtime API
openai_client_event_queued: expose the raw client events sent to the OpenAI Realtime API

Hierarchy (view full)

llm.RealtimeSession
- RealtimeSession

Index

Constructors

Accessors

chatCtx realtimeModel tools

Methods

clearAudio close commitAudio generateReply interrupt pushAudio sendEvent setInputAudioStream startUserActivity truncate updateChatCtx updateInstructions updateOptions updateTools

Constructors

constructor

new RealtimeSession(realtimeModel): realtime.RealtimeSession
Parameters
- realtimeModel: realtime.RealtimeModel
Returns realtime.RealtimeSession
Overrides RealtimeSession.constructor
- Defined in plugins/openai/src/realtime/realtime_model.ts:376

Accessors

chatCtx

get chatCtx(): ChatContext
Returns ChatContext
Overrides llm.RealtimeSession.chatCtx
- Defined in plugins/openai/src/realtime/realtime_model.ts:415

realtimeModel

get realtimeModel(): llm.RealtimeModel
Returns llm.RealtimeModel
Inherited from llm.RealtimeSession.realtimeModel
- Defined in agents/dist/llm/realtime.d.ts:54

tools

get tools(): ToolContext
Returns ToolContext
Overrides llm.RealtimeSession.tools
- Defined in plugins/openai/src/realtime/realtime_model.ts:419

Methods

clearAudio

clearAudio(): Promise<void>
Clear the input audio buffer to the server

Returns Promise<void>
Overrides RealtimeSession.clearAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:615

close

close(): Promise<void>
Returns Promise<void>
Overrides RealtimeSession.close
- Defined in plugins/openai/src/realtime/realtime_model.ts:974

commitAudio

commitAudio(): Promise<void>
Commit the input audio buffer to the server

Returns Promise<void>
Overrides RealtimeSession.commitAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:605

generateReply

generateReply(instructions?): Promise<GenerationCreatedEvent>
Parameters
- Optional instructions: string
Returns Promise<GenerationCreatedEvent>
Throws
RealtimeError on Timeout
Overrides RealtimeSession.generateReply
- Defined in plugins/openai/src/realtime/realtime_model.ts:622

interrupt

interrupt(): Promise<void>
Cancel the current generation (do nothing if no generation is in progress)

Returns Promise<void>
Overrides RealtimeSession.interrupt
- Defined in plugins/openai/src/realtime/realtime_model.ts:628

pushAudio

pushAudio(frame): void
Parameters
- frame: AudioFrame
Returns void
Overrides RealtimeSession.pushAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:592

sendEvent

sendEvent(command): void
Parameters
- command: ClientEvent
Returns void
- Defined in plugins/openai/src/realtime/realtime_model.ts:386

setInputAudioStream

setInputAudioStream(audioStream): void
Parameters
- audioStream: ReadableStream<AudioFrame>
Returns void
Inherited from RealtimeSession.setInputAudioStream
- Defined in agents/dist/llm/realtime.d.ts:96

startUserActivity

startUserActivity(): void
Notifies the model that user activity has started

Returns void
Inherited from RealtimeSession.startUserActivity
- Defined in agents/dist/llm/realtime.d.ts:94

truncate

truncate(_options): Promise<void>
Truncate the message at the given audio end time
Parameters
- _options: {
  audioEndMs: number;
  messageId: string;
  }
  - audioEndMs: number
  - messageId: string
Returns Promise<void>
Overrides RealtimeSession.truncate
- Defined in plugins/openai/src/realtime/realtime_model.ts:634

updateChatCtx

updateChatCtx(_chatCtx): Promise<void>
Parameters
- _chatCtx: ChatContext
Returns Promise<void>
Throws
RealtimeError on Timeout
Overrides RealtimeSession.updateChatCtx
- Defined in plugins/openai/src/realtime/realtime_model.ts:423

updateInstructions

updateInstructions(_instructions): Promise<void>
Parameters
- _instructions: string
Returns Promise<void>
Overrides RealtimeSession.updateInstructions
- Defined in plugins/openai/src/realtime/realtime_model.ts:565

updateOptions

updateOptions(__namedParameters): void
Parameters
- __namedParameters: {
  toolChoice?: llm.ToolChoice;
  }
  - Optional toolChoice?: llm.ToolChoice
Returns void
Overrides RealtimeSession.updateOptions
- Defined in plugins/openai/src/realtime/realtime_model.ts:577

updateTools

updateTools(_tools): Promise<void>
Parameters
- _tools: ToolContext
Returns Promise<void>
Overrides RealtimeSession.updateTools
- Defined in plugins/openai/src/realtime/realtime_model.ts:505