Module agents - v1.5.1

LiveKit Agents is a framework for building realtime programmable participants that run on servers.

See

LiveKit Agents documentation

References

AMD AMDCallbacks AMDCategory AMDOptions AMDPredictionEvent Agent AgentConfigUpdate AgentContext AgentCreateOptions AgentEvent AgentFalseInterruptionEvent AgentHandoff AgentHandoffItem AgentHookNodeResult AgentHooks AgentMetrics AgentOptions AgentSession AgentSessionEventTypes AgentSessionOptions AgentSessionUsage AgentState AgentStateChangedEvent AgentTask AgentTaskContext AgentTaskCreateOptions AgentsConsole AnonFunctionTool AsyncToolOptions AsyncToolset AsyncToolsetCreateOptions AudioConfig AudioContent AudioOutput AudioOutputCapabilities AudioOutputOptions AudioSegmentEnd AudioSourceType AvailabilityChangedEvent AvatarMetrics AvatarSession AvatarSessionCallbacks BackgroundAudioPlayer BackgroundAudioPlayerOptions BackgroundAudioStartOptions BaseEndpointing BuiltinAudioClip CONFIRM_DUPLICATE_PARAM ChatChunk ChatContent ChatContext ChatContextValidationIssue ChatContextValidationResult ChatContextValidationSeverity ChatItem ChatMessage ChatRole ChoiceDelta CloseEvent CloseReason CollectedResponse CompletionUsage ConversationItemAddedEvent DEFAULT_PARTICIPANT_KINDS DEFAULT_TEXT_INPUT_CALLBACK DataStreamAudioOutput DataStreamAudioOutputOptions DuplicateMode DuplicatePromptArgs DynamicEndpointing EOTInferenceMetrics EOTModelUsage EOUMetrics EndpointingOptions EotPredictionEvent ErrorEvent FallbackAdapter FallbackAdapterOptions FormatChatHistoryOptions FunctionCall FunctionCallOutput FunctionTool FunctionToolsExecutedEvent GenerationCreatedEvent ImageContent InputSpeechStartedEvent InputSpeechStoppedEvent InputTranscriptionCompleted Instructions InterruptionMetrics InterruptionModelUsage KeytermDetectionOptions KeytermDetector KeytermsOptions LLM LLMCallbacks LLMMetrics LLMModelUsage LLMStream MessageGeneration MetricsCollectedEvent MetricsMetadata MetricsReport ModelSettings ModelUsage ModelUsageCollector OpenAIFunctionParameters ParalellTextOutput ParticipantAudioOutput ParticipantLegacyTranscriptionOutput ParticipantTranscriptionOutput ParticipantTranscriptionOutputOptions PlayHandle PlaybackFinishedEvent PlaybackStartedEvent ProviderFormat ProviderTool QueueAudioOutput QueueAudioOutputClearEvent QueueAudioOutputItem RealtimeCapabilities RealtimeModel RealtimeModelError RealtimeModelMetrics RealtimeSession RealtimeSessionReconnectedEvent RemoteChatContext RemoteSession RemoteSessionCallbacks RemoteSessionEventTypes ReplyPromptArgs RoomIO RoomInputOptions RoomOutputOptions RoomSessionTransport RunContext RunOutputOptions STTMetrics STTModelUsage SerializedImage SessionHost SessionReport SessionReportOptions SessionTransport SessionUsageUpdatedEvent ShutdownReason SpeechCreatedEvent SpeechSource StopResponse TTSMetrics TTSModelUsage TcpAudioInput TcpAudioOutput TcpSessionTransport TextInputCallback TextInputEvent TimedString Tool ToolCalledEvent ToolChoice ToolCompletedEvent ToolContext ToolContextEntry ToolContextLike ToolError ToolFlag ToolHandlingOptions ToolOptions ToolType Toolset ToolsetContext ToolsetCreateOptions UsageCollector UsageSummary UserInputTranscribedEvent UserState UserStateChangedEvent UserTurnExceededEvent UserTurnLimitOptions VADMetrics VoiceOptions computeChatCtxDiff concatInstructions createAgentFalseInterruptionEvent createAgentStateChangedEvent createAudioContent createCloseEvent createConversationItemAddedEvent createEndpointing createEotPredictionEvent createErrorEvent createFunctionToolsExecutedEvent createImageContent createMetricsCollectedEvent createSessionReport createSessionUsageUpdatedEvent createSpeechCreatedEvent createTimedString createToolOptions createUserInputTranscribedEvent createUserStateChangedEvent createUserTurnExceededEvent defaultEndpointingOptions defaultUserTurnLimitOptions executeToolCall filterZeroValues formatChatHistory getBuiltinAudioPath handoff isBuiltinAudioClip isFunctionTool isProviderTool isTimedString isTool isToolset logMetrics serializeImage sessionReportToJSON streamingEndpointingOptions testing textTransforms toJsonSchema toToolContext tool validateChatContextStructure zipFunctionCallsAndOutputs

Namespaces

beta cli inference ipc llm metrics stream stt telemetry tokenize tts voice workflows

Enumerations

AutoSubscribe PluginEventTypes TaskResult VADEventType

Classes

APIConnectionError APIError APIStatusError APITimeoutError AgentServer AssignmentTimeoutError AudioByteStream ConnectionPool FunctionExistsError IdleTimeoutError InvalidErrorType JobContext JobProcess JobRequest MissingCredentialsError Plugin ServerOptions Task TextAudioSynchronizer UnexpectedModelBehavior VAD VADStream WorkerError WorkerPermissions

Interfaces

APIConnectOptions AgentDefinition AudioDecodeOptions ConnectionPoolOptions SessionConnectOptions TextSyncOptions VADCapabilities VADEvent

Type Aliases

Aborted AudioBuffer DelayOptions Expand FlushSentinel JobAcceptArguments KnownLanguageCode LanguageCode PluginEventMap RunningJobInfo VADCallbacks

Variables

DEFAULT_API_CONNECT_OPTIONS DEFAULT_SESSION_CONNECT_OPTIONS FlushSentinel KNOWN_LANGUAGE_CODES USERDATA_TIMED_TRANSCRIPT USERDATA_TTS_STARTED_TIME Worker WorkerOptions defaultTextSyncOptions version

Functions

areLanguagesEquivalent asError asLanguageCode audioFramesFromFile calculateAudioDurationSeconds cancelAndWait combineSignals createImmutableArray dedent defineAgent delay getBaseLanguage getIsoLanguage getJobContext getLanguageRegion isAPIError isCloud isDevMode isFfmpegTeardownError isHosted isImmutableArray isPending isStreamClosedError isWritableStreamClosedError loopAudioFramesFromFile mergeFrames noop normalizeLanguage readStream rejectOnAbort resampleStream resolveEnvVar shortuuid startSoon toError toLanguageName toStream waitFor waitForAbort waitForParticipant waitForParticipantAttribute waitForTrackPublication waitUntilAborted waitUntilTimeout withResolvers

References

AMD

Re-exports AMD

AMDCallbacks

Re-exports AMDCallbacks

AMDCategory

Re-exports AMDCategory

AMDOptions

Re-exports AMDOptions

AMDPredictionEvent

Re-exports AMDPredictionEvent

Agent

Re-exports Agent

AgentConfigUpdate

Re-exports AgentConfigUpdate

AgentContext

Re-exports AgentContext

AgentCreateOptions

Re-exports AgentCreateOptions

AgentEvent

Re-exports AgentEvent

AgentFalseInterruptionEvent

Re-exports AgentFalseInterruptionEvent

AgentHandoff

Re-exports AgentHandoff

AgentHandoffItem

Re-exports AgentHandoffItem

AgentHookNodeResult

Re-exports AgentHookNodeResult

AgentHooks

Re-exports AgentHooks

AgentMetrics

Re-exports AgentMetrics

AgentOptions

Re-exports AgentOptions

AgentSession

Re-exports AgentSession

AgentSessionEventTypes

Re-exports AgentSessionEventTypes

AgentSessionOptions

Re-exports AgentSessionOptions

AgentSessionUsage

Re-exports AgentSessionUsage

AgentState

Re-exports AgentState

AgentStateChangedEvent

Re-exports AgentStateChangedEvent

AgentTask

Re-exports AgentTask

AgentTaskContext

Re-exports AgentTaskContext

AgentTaskCreateOptions

Re-exports AgentTaskCreateOptions

AgentsConsole

Re-exports AgentsConsole

AnonFunctionTool

Re-exports AnonFunctionTool

AsyncToolOptions

Re-exports AsyncToolOptions

AsyncToolset

Re-exports AsyncToolset

AsyncToolsetCreateOptions

Re-exports AsyncToolsetCreateOptions

AudioConfig

Re-exports AudioConfig

AudioContent

Re-exports AudioContent

AudioOutput

Re-exports AudioOutput

AudioOutputCapabilities

Re-exports AudioOutputCapabilities

AudioOutputOptions

Re-exports AudioOutputOptions

AudioSegmentEnd

Re-exports AudioSegmentEnd

AudioSourceType

Re-exports AudioSourceType

AvailabilityChangedEvent

Re-exports AvailabilityChangedEvent

AvatarMetrics

Re-exports AvatarMetrics

AvatarSession

Re-exports AvatarSession

AvatarSessionCallbacks

Re-exports AvatarSessionCallbacks

BackgroundAudioPlayer

Re-exports BackgroundAudioPlayer

BackgroundAudioPlayerOptions

Re-exports BackgroundAudioPlayerOptions

BackgroundAudioStartOptions

Re-exports BackgroundAudioStartOptions

BaseEndpointing

Re-exports BaseEndpointing

BuiltinAudioClip

Re-exports BuiltinAudioClip

CONFIRM_DUPLICATE_PARAM

Re-exports CONFIRM_DUPLICATE_PARAM

ChatChunk

Re-exports ChatChunk

ChatContent

Re-exports ChatContent

ChatContext

Re-exports ChatContext

ChatContextValidationIssue

Re-exports ChatContextValidationIssue

ChatContextValidationResult

Re-exports ChatContextValidationResult

ChatContextValidationSeverity

Re-exports ChatContextValidationSeverity

ChatItem

Re-exports ChatItem

ChatMessage

Re-exports ChatMessage

ChatRole

Re-exports ChatRole

ChoiceDelta

Re-exports ChoiceDelta

CloseEvent

Re-exports CloseEvent

CloseReason

Re-exports CloseReason

CollectedResponse

Re-exports CollectedResponse

CompletionUsage

Re-exports CompletionUsage

ConversationItemAddedEvent

Re-exports ConversationItemAddedEvent

DEFAULT_PARTICIPANT_KINDS

Re-exports DEFAULT_PARTICIPANT_KINDS

DEFAULT_TEXT_INPUT_CALLBACK

Re-exports DEFAULT_TEXT_INPUT_CALLBACK

DataStreamAudioOutput

Re-exports DataStreamAudioOutput

DataStreamAudioOutputOptions

Re-exports DataStreamAudioOutputOptions

DuplicateMode

Re-exports DuplicateMode

DuplicatePromptArgs

Re-exports DuplicatePromptArgs

DynamicEndpointing

Re-exports DynamicEndpointing

EOTInferenceMetrics

Re-exports EOTInferenceMetrics

EOTModelUsage

Re-exports EOTModelUsage

EOUMetrics

Re-exports EOUMetrics

EndpointingOptions

Re-exports EndpointingOptions

EotPredictionEvent

Re-exports EotPredictionEvent

ErrorEvent

Re-exports ErrorEvent

FallbackAdapter

Re-exports FallbackAdapter

FallbackAdapterOptions

Re-exports FallbackAdapterOptions

FormatChatHistoryOptions

Re-exports FormatChatHistoryOptions

FunctionCall

Re-exports FunctionCall

FunctionCallOutput

Re-exports FunctionCallOutput

FunctionTool

Re-exports FunctionTool

FunctionToolsExecutedEvent

Re-exports FunctionToolsExecutedEvent

GenerationCreatedEvent

Re-exports GenerationCreatedEvent

ImageContent

Re-exports ImageContent

InputSpeechStartedEvent

Re-exports InputSpeechStartedEvent

InputSpeechStoppedEvent

Re-exports InputSpeechStoppedEvent

InputTranscriptionCompleted

Re-exports InputTranscriptionCompleted

Instructions

Re-exports Instructions

InterruptionMetrics

Re-exports InterruptionMetrics

InterruptionModelUsage

Re-exports InterruptionModelUsage

KeytermDetectionOptions

Re-exports KeytermDetectionOptions

KeytermDetector

Re-exports KeytermDetector

KeytermsOptions

Re-exports KeytermsOptions

LLM

Re-exports LLM

LLMCallbacks

Re-exports LLMCallbacks

LLMMetrics

Re-exports LLMMetrics

LLMModelUsage

Re-exports LLMModelUsage

LLMStream

Re-exports LLMStream

MessageGeneration

Re-exports MessageGeneration

MetricsCollectedEvent

Re-exports MetricsCollectedEvent

MetricsMetadata

Re-exports MetricsMetadata

MetricsReport

Re-exports MetricsReport

ModelSettings

Re-exports ModelSettings

ModelUsage

Re-exports ModelUsage

ModelUsageCollector

Re-exports ModelUsageCollector

OpenAIFunctionParameters

Re-exports OpenAIFunctionParameters

ParalellTextOutput

Re-exports ParalellTextOutput

ParticipantAudioOutput

Re-exports ParticipantAudioOutput

ParticipantLegacyTranscriptionOutput

Re-exports ParticipantLegacyTranscriptionOutput

ParticipantTranscriptionOutput

Re-exports ParticipantTranscriptionOutput

ParticipantTranscriptionOutputOptions

Re-exports ParticipantTranscriptionOutputOptions

PlayHandle

Re-exports PlayHandle

PlaybackFinishedEvent

Re-exports PlaybackFinishedEvent

PlaybackStartedEvent

Re-exports PlaybackStartedEvent

ProviderFormat

Re-exports ProviderFormat

ProviderTool

Re-exports ProviderTool

QueueAudioOutput

Re-exports QueueAudioOutput

QueueAudioOutputClearEvent

Re-exports QueueAudioOutputClearEvent

QueueAudioOutputItem

Re-exports QueueAudioOutputItem

RealtimeCapabilities

Re-exports RealtimeCapabilities

RealtimeModel

Re-exports RealtimeModel

RealtimeModelError

Re-exports RealtimeModelError

RealtimeModelMetrics

Re-exports RealtimeModelMetrics

RealtimeSession

Re-exports RealtimeSession

RealtimeSessionReconnectedEvent

Re-exports RealtimeSessionReconnectedEvent

RemoteChatContext

Re-exports RemoteChatContext

RemoteSession

Re-exports RemoteSession

RemoteSessionCallbacks

Re-exports RemoteSessionCallbacks

RemoteSessionEventTypes

Re-exports RemoteSessionEventTypes

ReplyPromptArgs

Re-exports ReplyPromptArgs

RoomIO

Re-exports RoomIO

RoomInputOptions

Re-exports RoomInputOptions

RoomOutputOptions

Re-exports RoomOutputOptions

RoomSessionTransport

Re-exports RoomSessionTransport

RunContext

Re-exports RunContext

RunOutputOptions

Re-exports RunOutputOptions

STTMetrics

Re-exports STTMetrics

STTModelUsage

Re-exports STTModelUsage

SerializedImage

Re-exports SerializedImage

SessionHost

Re-exports SessionHost

SessionReport

Re-exports SessionReport

SessionReportOptions

Re-exports SessionReportOptions

SessionTransport

Re-exports SessionTransport

SessionUsageUpdatedEvent

Re-exports SessionUsageUpdatedEvent

ShutdownReason

Re-exports ShutdownReason

SpeechCreatedEvent

Re-exports SpeechCreatedEvent

SpeechSource

Re-exports SpeechSource

StopResponse

Re-exports StopResponse

TTSMetrics

Re-exports TTSMetrics

TTSModelUsage

Re-exports TTSModelUsage

TcpAudioInput

Re-exports TcpAudioInput

TcpAudioOutput

Re-exports TcpAudioOutput

TcpSessionTransport

Re-exports TcpSessionTransport

TextInputCallback

Re-exports TextInputCallback

TextInputEvent

Re-exports TextInputEvent

TimedString

Re-exports TimedString

Tool

Re-exports Tool

ToolCalledEvent

Re-exports ToolCalledEvent

ToolChoice

Re-exports ToolChoice

ToolCompletedEvent

Re-exports ToolCompletedEvent

ToolContext

Re-exports ToolContext

ToolContextEntry

Re-exports ToolContextEntry

ToolContextLike

Re-exports ToolContextLike

ToolError

Re-exports ToolError

ToolFlag

Re-exports ToolFlag

ToolHandlingOptions

Re-exports ToolHandlingOptions

ToolOptions

Re-exports ToolOptions

ToolType

Re-exports ToolType

Toolset

Re-exports Toolset

ToolsetContext

Re-exports ToolsetContext

ToolsetCreateOptions

Re-exports ToolsetCreateOptions

UsageCollector

Re-exports UsageCollector

UsageSummary

Re-exports UsageSummary

UserInputTranscribedEvent

Re-exports UserInputTranscribedEvent

UserState

Re-exports UserState

UserStateChangedEvent

Re-exports UserStateChangedEvent

UserTurnExceededEvent

Re-exports UserTurnExceededEvent

UserTurnLimitOptions

Re-exports UserTurnLimitOptions

VADMetrics

Re-exports VADMetrics

VoiceOptions

Re-exports VoiceOptions

computeChatCtxDiff

Re-exports computeChatCtxDiff

concatInstructions

Re-exports concatInstructions

createAgentFalseInterruptionEvent

Re-exports createAgentFalseInterruptionEvent

createAgentStateChangedEvent

Re-exports createAgentStateChangedEvent

createAudioContent

Re-exports createAudioContent

createCloseEvent

Re-exports createCloseEvent

createConversationItemAddedEvent

Re-exports createConversationItemAddedEvent

createEndpointing

Re-exports createEndpointing

createEotPredictionEvent

Re-exports createEotPredictionEvent

createErrorEvent

Re-exports createErrorEvent

createFunctionToolsExecutedEvent

Re-exports createFunctionToolsExecutedEvent

createImageContent

Re-exports createImageContent

createMetricsCollectedEvent

Re-exports createMetricsCollectedEvent

createSessionReport

Re-exports createSessionReport

createSessionUsageUpdatedEvent

Re-exports createSessionUsageUpdatedEvent

createSpeechCreatedEvent

Re-exports createSpeechCreatedEvent

createTimedString

Re-exports createTimedString

createToolOptions

Re-exports createToolOptions

createUserInputTranscribedEvent

Re-exports createUserInputTranscribedEvent

createUserStateChangedEvent

Re-exports createUserStateChangedEvent

createUserTurnExceededEvent

Re-exports createUserTurnExceededEvent

defaultEndpointingOptions

Re-exports defaultEndpointingOptions

defaultUserTurnLimitOptions

Re-exports defaultUserTurnLimitOptions

executeToolCall

Re-exports executeToolCall

filterZeroValues

Re-exports filterZeroValues

formatChatHistory

Re-exports formatChatHistory

getBuiltinAudioPath

Re-exports getBuiltinAudioPath

handoff

Re-exports handoff

isBuiltinAudioClip

Re-exports isBuiltinAudioClip

isFunctionTool

Re-exports isFunctionTool

isProviderTool

Re-exports isProviderTool

isTimedString

Re-exports isTimedString

isTool

Re-exports isTool

isToolset

Re-exports isToolset

logMetrics

Re-exports logMetrics

serializeImage

Re-exports serializeImage

sessionReportToJSON

Re-exports sessionReportToJSON

streamingEndpointingOptions

Re-exports streamingEndpointingOptions

testing

Re-exports testing

textTransforms

Re-exports textTransforms

toJsonSchema

Re-exports toJsonSchema

toToolContext

Re-exports toToolContext

tool

Re-exports tool

validateChatContextStructure

Re-exports validateChatContextStructure

zipFunctionCallsAndOutputs

Re-exports zipFunctionCallsAndOutputs

Settings

Member Visibility

Theme

On This Page

AMDAMDCallbacksAMDCategoryAMDOptionsAMDPredictionEventAgentAgentConfigUpdateAgentContextAgentCreateOptionsAgentEventAgentFalseInterruptionEventAgentHandoffAgentHandoffItemAgentHookNodeResultAgentHooksAgentMetricsAgentOptionsAgentSessionAgentSessionEventTypesAgentSessionOptionsAgentSessionUsageAgentStateAgentStateChangedEventAgentTaskAgentTaskContextAgentTaskCreateOptionsAgentsConsoleAnonFunctionToolAsyncToolOptionsAsyncToolsetAsyncToolsetCreateOptionsAudioConfigAudioContentAudioOutputAudioOutputCapabilitiesAudioOutputOptionsAudioSegmentEndAudioSourceTypeAvailabilityChangedEventAvatarMetricsAvatarSessionAvatarSessionCallbacksBackgroundAudioPlayerBackgroundAudioPlayerOptionsBackgroundAudioStartOptionsBaseEndpointingBuiltinAudioClipCONFIRM_DUPLICATE_PARAMChatChunkChatContentChatContextChatContextValidationIssueChatContextValidationResultChatContextValidationSeverityChatItemChatMessageChatRoleChoiceDeltaCloseEventCloseReasonCollectedResponseCompletionUsageConversationItemAddedEventDEFAULT_PARTICIPANT_KINDSDEFAULT_TEXT_INPUT_CALLBACKDataStreamAudioOutputDataStreamAudioOutputOptionsDuplicateModeDuplicatePromptArgsDynamicEndpointingEOTInferenceMetricsEOTModelUsageEOUMetricsEndpointingOptionsEotPredictionEventErrorEventFallbackAdapterFallbackAdapterOptionsFormatChatHistoryOptionsFunctionCallFunctionCallOutputFunctionToolFunctionToolsExecutedEventGenerationCreatedEventImageContentInputSpeechStartedEventInputSpeechStoppedEventInputTranscriptionCompletedInstructionsInterruptionMetricsInterruptionModelUsageKeytermDetectionOptionsKeytermDetectorKeytermsOptionsLLMLLMCallbacksLLMMetricsLLMModelUsageLLMStreamMessageGenerationMetricsCollectedEventMetricsMetadataMetricsReportModelSettingsModelUsageModelUsageCollectorOpenAIFunctionParametersParalellTextOutputParticipantAudioOutputParticipantLegacyTranscriptionOutputParticipantTranscriptionOutputParticipantTranscriptionOutputOptionsPlayHandlePlaybackFinishedEventPlaybackStartedEventProviderFormatProviderToolQueueAudioOutputQueueAudioOutputClearEventQueueAudioOutputItemRealtimeCapabilitiesRealtimeModelRealtimeModelErrorRealtimeModelMetricsRealtimeSessionRealtimeSessionReconnectedEventRemoteChatContextRemoteSessionRemoteSessionCallbacksRemoteSessionEventTypesReplyPromptArgsRoomIORoomInputOptionsRoomOutputOptionsRoomSessionTransportRunContextRunOutputOptionsSTTMetricsSTTModelUsageSerializedImageSessionHostSessionReportSessionReportOptionsSessionTransportSessionUsageUpdatedEventShutdownReasonSpeechCreatedEventSpeechSourceStopResponseTTSMetricsTTSModelUsageTcpAudioInputTcpAudioOutputTcpSessionTransportTextInputCallbackTextInputEventTimedStringToolToolCalledEventToolChoiceToolCompletedEventToolContextToolContextEntryToolContextLikeToolErrorToolFlagToolHandlingOptionsToolOptionsToolTypeToolsetToolsetContextToolsetCreateOptionsUsageCollectorUsageSummaryUserInputTranscribedEventUserStateUserStateChangedEventUserTurnExceededEventUserTurnLimitOptionsVADMetricsVoiceOptionscomputeChatCtxDiffconcatInstructionscreateAgentFalseInterruptionEventcreateAgentStateChangedEventcreateAudioContentcreateCloseEventcreateConversationItemAddedEventcreateEndpointingcreateEotPredictionEventcreateErrorEventcreateFunctionToolsExecutedEventcreateImageContentcreateMetricsCollectedEventcreateSessionReportcreateSessionUsageUpdatedEventcreateSpeechCreatedEventcreateTimedStringcreateToolOptionscreateUserInputTranscribedEventcreateUserStateChangedEventcreateUserTurnExceededEventdefaultEndpointingOptionsdefaultUserTurnLimitOptionsexecuteToolCallfilterZeroValuesformatChatHistorygetBuiltinAudioPathhandoffisBuiltinAudioClipisFunctionToolisProviderToolisTimedStringisToolisToolsetlogMetricsserializeImagesessionReportToJSONstreamingEndpointingOptionstestingtextTransformstoJsonSchematoToolContexttoolvalidateChatContextStructurezipFunctionCallsAndOutputs