Class RealtimeSession

A session for the OpenAI Realtime API.

This class is used to interact with the OpenAI Realtime API. It is responsible for sending events to the OpenAI Realtime API and receiving events from it.

It exposes two more events:

openai_server_event_received: expose the raw server events from the OpenAI Realtime API
openai_client_event_queued: expose the raw client events sent to the OpenAI Realtime API

Hierarchy (view full)

llm.RealtimeSession
- RealtimeSession

Index

Constructors

Accessors

chatCtx realtimeModel tools

Methods

clearAudio close commitAudio generateReply interrupt pushAudio sendEvent setInputAudioStream startUserActivity truncate updateChatCtx updateInstructions updateOptions updateTools

Constructors

constructor

new RealtimeSession(realtimeModel): realtime.RealtimeSession
Parameters
- realtimeModel: realtime.RealtimeModel
Returns realtime.RealtimeSession
Overrides RealtimeSession.constructor
- Defined in plugins/openai/src/realtime/realtime_model.ts:377

Accessors

chatCtx

get chatCtx(): ChatContext
Returns ChatContext
Overrides llm.RealtimeSession.chatCtx
- Defined in plugins/openai/src/realtime/realtime_model.ts:416

realtimeModel

get realtimeModel(): llm.RealtimeModel
Returns llm.RealtimeModel
Inherited from llm.RealtimeSession.realtimeModel
- Defined in agents/dist/llm/realtime.d.ts:54

tools

get tools(): ToolContext
Returns ToolContext
Overrides llm.RealtimeSession.tools
- Defined in plugins/openai/src/realtime/realtime_model.ts:420

Methods

clearAudio

clearAudio(): Promise<void>
Clear the input audio buffer to the server

Returns Promise<void>
Overrides RealtimeSession.clearAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:616

close

close(): Promise<void>
Returns Promise<void>
Overrides RealtimeSession.close
- Defined in plugins/openai/src/realtime/realtime_model.ts:975

commitAudio

commitAudio(): Promise<void>
Commit the input audio buffer to the server

Returns Promise<void>
Overrides RealtimeSession.commitAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:606

generateReply

generateReply(instructions?): Promise<GenerationCreatedEvent>
Parameters
- Optional instructions: string
Returns Promise<GenerationCreatedEvent>
Throws
RealtimeError on Timeout
Overrides RealtimeSession.generateReply
- Defined in plugins/openai/src/realtime/realtime_model.ts:623

interrupt

interrupt(): Promise<void>
Cancel the current generation (do nothing if no generation is in progress)

Returns Promise<void>
Overrides RealtimeSession.interrupt
- Defined in plugins/openai/src/realtime/realtime_model.ts:629

pushAudio

pushAudio(frame): void
Parameters
- frame: AudioFrame
Returns void
Overrides RealtimeSession.pushAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:593

sendEvent

sendEvent(command): void
Parameters
- command: ClientEvent
Returns void
- Defined in plugins/openai/src/realtime/realtime_model.ts:387

setInputAudioStream

setInputAudioStream(audioStream): void
Parameters
- audioStream: ReadableStream<AudioFrame>
Returns void
Inherited from RealtimeSession.setInputAudioStream
- Defined in agents/dist/llm/realtime.d.ts:96

startUserActivity

startUserActivity(): void
Notifies the model that user activity has started

Returns void
Inherited from RealtimeSession.startUserActivity
- Defined in agents/dist/llm/realtime.d.ts:94

truncate

truncate(_options): Promise<void>
Truncate the message at the given audio end time
Parameters
- _options: {
  audioEndMs: number;
  messageId: string;
  }
  - audioEndMs: number
  - messageId: string
Returns Promise<void>
Overrides RealtimeSession.truncate
- Defined in plugins/openai/src/realtime/realtime_model.ts:635

updateChatCtx

updateChatCtx(_chatCtx): Promise<void>
Parameters
- _chatCtx: ChatContext
Returns Promise<void>
Throws
RealtimeError on Timeout
Overrides RealtimeSession.updateChatCtx
- Defined in plugins/openai/src/realtime/realtime_model.ts:424

updateInstructions

updateInstructions(_instructions): Promise<void>
Parameters
- _instructions: string
Returns Promise<void>
Overrides RealtimeSession.updateInstructions
- Defined in plugins/openai/src/realtime/realtime_model.ts:566

updateOptions

updateOptions(__namedParameters): void
Parameters
- __namedParameters: {
  toolChoice?: llm.ToolChoice;
  }
  - Optional toolChoice?: llm.ToolChoice
Returns void
Overrides RealtimeSession.updateOptions
- Defined in plugins/openai/src/realtime/realtime_model.ts:578

updateTools

updateTools(_tools): Promise<void>
Parameters
- _tools: ToolContext
Returns Promise<void>
Overrides RealtimeSession.updateTools
- Defined in plugins/openai/src/realtime/realtime_model.ts:506