Class RealtimeSession

A session for the OpenAI Realtime API.

This class is used to interact with the OpenAI Realtime API. It is responsible for sending events to the OpenAI Realtime API and receiving events from it.

It exposes two more events:

openai_server_event_received: expose the raw server events from the OpenAI Realtime API
openai_client_event_queued: expose the raw client events sent to the OpenAI Realtime API

Hierarchy (view full)

llm.RealtimeSession
- RealtimeSession

Index

Constructors

constructor

new RealtimeSession(realtimeModel): realtime.RealtimeSession
Parameters
- realtimeModel: realtime.RealtimeModel
Returns realtime.RealtimeSession
Overrides RealtimeSession.constructor
- Defined in plugins/openai/src/realtime/realtime_model.ts:442

Accessors

chatCtx

get chatCtx(): ChatContext
Returns ChatContext
Overrides llm.RealtimeSession.chatCtx
- Defined in plugins/openai/src/realtime/realtime_model.ts:520

realtimeModel

get realtimeModel(): llm.RealtimeModel
Returns llm.RealtimeModel
Inherited from llm.RealtimeSession.realtimeModel
- Defined in agents/dist/llm/realtime.d.ts:72

tools

get tools(): ToolContext
Returns ToolContext
Overrides llm.RealtimeSession.tools
- Defined in plugins/openai/src/realtime/realtime_model.ts:524

Methods

_updateSession

_updateSession(instructions?, chatCtx?, tools?): Promise<void>
Parameters
- Optional instructions: string
- Optional chatCtx: ChatContext
- Optional tools: ToolContext
Returns Promise<void>
Inherited from RealtimeSession._updateSession
- Defined in agents/dist/llm/realtime.d.ts:110

clearAudio

clearAudio(): Promise<void>
Clear the input audio buffer to the server

Returns Promise<void>
Overrides RealtimeSession.clearAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:802

close

close(): Promise<void>
Returns Promise<void>
Overrides RealtimeSession.close
- Defined in plugins/openai/src/realtime/realtime_model.ts:1239

commitAudio

commitAudio(): Promise<void>
Commit the input audio buffer to the server

Returns Promise<void>
Overrides RealtimeSession.commitAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:792

generateReply

generateReply(instructions?): Promise<GenerationCreatedEvent>
Parameters
- Optional instructions: string
Returns Promise<GenerationCreatedEvent>
Throws
RealtimeError on Timeout
Overrides RealtimeSession.generateReply
- Defined in plugins/openai/src/realtime/realtime_model.ts:809

interrupt

interrupt(): Promise<void>
Cancel the current generation (do nothing if no generation is in progress)

Returns Promise<void>
Overrides RealtimeSession.interrupt
- Defined in plugins/openai/src/realtime/realtime_model.ts:815

pushAudio

pushAudio(frame): void
Parameters
- frame: AudioFrame
Returns void
Overrides RealtimeSession.pushAudio
- Defined in plugins/openai/src/realtime/realtime_model.ts:779

sendEvent

sendEvent(command): void
Parameters
- command: ClientEvent
Returns void
- Defined in plugins/openai/src/realtime/realtime_model.ts:458

setInputAudioStream

setInputAudioStream(audioStream): void
Parameters
- audioStream: ReadableStream<AudioFrame>
Returns void
Inherited from RealtimeSession.setInputAudioStream
- Defined in agents/dist/llm/realtime.d.ts:117

startUserActivity

startUserActivity(): void
Notifies the model that user activity has started

Returns void
Inherited from RealtimeSession.startUserActivity
- Defined in agents/dist/llm/realtime.d.ts:115

truncate

truncate(_options): Promise<void>
Truncate the message at the given audio end time
Parameters
- _options: {
      audioEndMs: number;
      audioTranscript?: string;
      messageId: string;
      modalities?: Modality[];
  }
  - audioEndMs: number
  - Optional audioTranscript?: string
  - messageId: string
  - Optional modalities?: Modality[]
Returns Promise<void>
Overrides RealtimeSession.truncate
- Defined in plugins/openai/src/realtime/realtime_model.ts:825

updateChatCtx

updateChatCtx(_chatCtx): Promise<void>
Parameters
- _chatCtx: ChatContext
Returns Promise<void>
Throws
RealtimeError on Timeout
Overrides RealtimeSession.updateChatCtx
- Defined in plugins/openai/src/realtime/realtime_model.ts:528

updateInstructions

updateInstructions(_instructions): Promise<void>
Parameters
- _instructions: string
Returns Promise<void>
Overrides RealtimeSession.updateInstructions
- Defined in plugins/openai/src/realtime/realtime_model.ts:738

updateOptions

updateOptions(__namedParameters): void
Parameters
- __namedParameters: {
  toolChoice?: llm.ToolChoice;
  }
  - Optional toolChoice?: llm.ToolChoice
Returns void
Overrides RealtimeSession.updateOptions
- Defined in plugins/openai/src/realtime/realtime_model.ts:753

updateTools

updateTools(_tools): Promise<void>
Parameters
- _tools: ToolContext
Returns Promise<void>
Overrides RealtimeSession.updateTools
- Defined in plugins/openai/src/realtime/realtime_model.ts:675

Class RealtimeSession

Hierarchy (view full)

Index

Constructors

Accessors

Methods

Constructors

constructor

Parameters

Returns realtime.RealtimeSession

Accessors

chatCtx

Returns ChatContext

realtimeModel

Returns llm.RealtimeModel

tools

Returns ToolContext

Methods

_updateSession

Parameters

Returns Promise<void>

clearAudio

Returns Promise<void>

close

Returns Promise<void>

commitAudio

Returns Promise<void>

generateReply

Parameters

Returns Promise<GenerationCreatedEvent>

Throws

interrupt

Returns Promise<void>

pushAudio

Parameters

Returns void

sendEvent

Parameters

Returns void

setInputAudioStream

Parameters

Returns void

startUserActivity

Returns void

truncate

Parameters

audioEndMs: number

Optional audioTranscript?: string

messageId: string

Optional modalities?: Modality[]

Returns Promise<void>

updateChatCtx

Parameters

Returns Promise<void>

Throws

updateInstructions

Parameters

Returns Promise<void>

updateOptions

Parameters

Optional toolChoice?: llm.ToolChoice

Returns void

updateTools

Parameters

Returns Promise<void>

Settings

Member Visibility

Theme

On This Page

`Optional` audioTranscript?: string

`Optional` modalities?: Modality[]

`Optional` toolChoice?: llm.ToolChoice