Server options

Options

The constructor for AgentServer includes some parameters for configuring the agent server. The following includes some of the available parameters. For the complete list, see the AgentServer reference.

server = AgentServer(
    # Whether the agent can subscribe to tracks, publish data, update metadata, etc.
    permissions,
    # Amount of time to wait for existing jobs to finish when SIGTERM or SIGINT is received
    drain_timeout,
    # The maximum value of load_fnc, above which no new processes will spawn
    load_threshold,
    # A function to perform any necessary initialization before the job starts.
    setup_fnc,
    # Function to determine the current load of the worker. Should return a value between 0 and 1.
    load_fnc,
    # The log level for the agent server and its job processes. Defaults to 'info'.
    log_level,
)

# start the agent server
cli.run_app(server)

server.setup_fnc = my_prewarm_function

const opts = new ServerOptions({
  // path to the agent module, which must export a default Agent object (usually the current file)
  agent: fileURLToPath(import.meta.url),
  // the agent name, used for explicit dispatch
  agentName: 'my-agent',
  // inspect the request and decide if the current agent server should handle it.
  requestFunc,
  // whether the agent can subscribe to tracks, publish data, update metadata, etc.
  permissions,
  // milliseconds to wait for existing jobs to finish when SIGTERM or SIGINT is received
  drainTimeout,
  // the type of agent server to create, either JT_ROOM or JT_PUBLISHER. Defaults to JT_ROOM.
  serverType: JobType.JT_ROOM,
  // a function that reports the current load of the agent server. returns a value between 0-1.
  loadFunc,
  // the maximum value of loadFunc, above which agent server is marked as unavailable.
  loadThreshold,
  // the log level for the agent server and its job processes. defaults to 'info'.
  logLevel: 'info',
})

// start the agent server
cli.runApp(opts);

Entrypoint function

The entrypoint function is the main function called for each new job, and is the core of your agent app. To learn more, see the entrypoint documentation in the job lifecycle topic.

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext):
    # connect to the room

    # handle the session
    ...

export default defineAgent({
  entry: async (ctx: JobContext) => {
    // connect to the room
    await ctx.connect();
    // handle the session
  },
});

Request handler

The on_request function runs each time the server has a job for the agent. The framework expects agent servers to explicitly accept or reject each job request. If the agent server accepts the request, your entrypoint function is called. If the request is rejected, it's sent to the next available agent server. A rejection indicates that the agent server is unable to handle the job, not that the job itself is invalid. The framework simply reassigns it to another agent server.

If on_request is not defined, the default behavior is to automatically accept all requests dispatched to the agent server.

async def request_fnc(req: JobRequest):
    # accept the job request
    await req.accept(
        # the agent's name (Participant.name), defaults to ""
        name="agent",
        # the agent's identity (Participant.identity), defaults to "agent-<jobid>"
        identity="identity",
        # attributes to set on the agent participant upon join
        attributes={"myagent": "rocks"},
    )

    # or reject it
    # await req.reject()

server = AgentServer()

@server.rtc_session(agent_name="my-agent", on_request=request_fnc)
async def my_agent(ctx: JobContext):
    # set up entrypoint function
    # handle the session
    ...

const requestFunc = async (req: JobRequest) => {
  // accept the job request
  await req.accept(
    // the agent's name (Participant.name), defaults to ""
    'my-agent',
    // the agent's identity (Participant.identity), defaults to "agent-<jobid>"
    'identity',
  );
};

const opts = new ServerOptions({
  requestFunc,
});

The name parameter is the agent's display name (Participant.name), used to identify the agent in the room. It defaults to the agent's identity, and is separate from the agent's dispatch name used for explicit dispatch.

Prewarm function

For isolation and performance reasons, the framework runs each agent job in its own process. Agents often need access to model files that take time to load. To address this, you can use a prewarm function to warm up the process before assigning any jobs to it. You can control the number of processes to keep warm using the num_idle_processes parameter.

In production, the default number of idle processes is based on the available CPU count:

• Python: math.ceil(cpu_count)
• Node.js: Math.min(os.availableParallelism(), 4)

Both SDKs read cgroup CPU limits when computing these defaults, so containers with limited CPU allocations pre-warm fewer processes than the host machine has cores. The Node.js default caps at 4 to limit memory from pre-warmed child processes on large machines. In development mode, both SDKs default to 0 (no pre-warming).

server = AgentServer()

def prewarm(proc: JobProcess):
    # load silero weights and store to process userdata
    proc.userdata["vad"] = silero.VAD.load()


server.setup_fnc = prewarm

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext):
    # access the loaded silero instance
    vad: silero.VAD = ctx.proc.userdata["vad"]

export default defineAgent({
  prewarm: async (proc: JobProcess) => {
    // load silero weights and store to process userdata
    proc.userData.vad = await silero.VAD.load();
  },
  entry: async (ctx: JobContext) => {
    // access the loaded silero instance
    const vad = ctx.proc.userData.vad! as silero.VAD;
  },
});

Agent server load

In custom deployments, you can configure the conditions under which the agent server stops accepting new jobs through the load_fnc and load_threshold parameters.

• load_fnc: A function that returns the current load of the agent server as a float between 0 and 1.0.
• load_threshold: The maximum load value at which the agent server still accepts new jobs.

The default load_fnc is the agent server's average CPU utilization over a 5-second window. The default load_threshold is 0.7.

from livekit.agents import AgentServer

server = AgentServer(
    load_threshold=0.9,
)

def compute_load(agent_server: AgentServer) -> float:
    return min(len(agent_server.active_jobs) / 10, 1.0)

server.load_fnc=compute_load

import { type AgentServer, ServerOptions } from '@livekit/agents';

const computeLoad = (server: AgentServer): Promise<number> => {
  return Math.min(server.activeJobs.length / 10, 1.0);
};

const opts = new ServerOptions({
  agent: fileURLToPath(import.meta.url),
  loadFunc: computeLoad,
  loadThreshold: 0.9,
});

Health check endpoint

The agent server automatically runs a local HTTP server that serves as a health check endpoint. The health check returns a 200 status when the agent server is connected to LiveKit server and operating normally, or a 503 status if there's a problem (for example, the inference process isn't running or the server isn't connected).

The endpoint is available at the root path (/) of the HTTP server. By default, it listens on all network interfaces (0.0.0.0) on port 8081 in production mode and a random available port in development mode. No configuration is needed for most deployments.

To customize the host or port, pass the host and port parameters:

server = AgentServer(
    host="0.0.0.0",  # default: all interfaces
    port=9090,        # default: 8081 in production, random in dev
)

const opts = new ServerOptions({
  agent: fileURLToPath(import.meta.url),
  host: '0.0.0.0',  // default: all interfaces
  port: 9090,        // default: 8081 in production, random in dev
});

Drain timeout

Agent sessions are stateful and should not be terminated abruptly. The Agents framework supports graceful termination: when a SIGTERM or SIGINT signal is received, the agent server enters a draining state. In this state, it stops accepting new jobs but allows existing ones to complete, up to a configured timeout.

The drain_timeout (Python) or drainTimeout (Node.js) parameter sets the maximum time to wait for active jobs to finish. It defaults to one hour. Python takes the value in seconds and Node.js takes it in milliseconds.

Permissions

By default, agents can both publish to and subscribe from the other participants in the same room. However, you can customize these permissions by setting the permissions parameter. To see the full list of parameters, see the WorkerPermissions reference.

server = AgentServer(
    ...
    permissions=WorkerPermissions(
        can_publish=True,
        can_subscribe=True,
        can_publish_data=True,
        # when set to true, the agent won't be visible to others in the room.
        # when hidden, it will also not be able to publish tracks to the room as it won't be visible.
        hidden=False,
    ),
)

const opts = new ServerOptions({
  agent: fileURLToPath(import.meta.url),
  permissions: {
    canPublish: true,
    canPublishData: true,
    canPublishSources: [],
    canSubscribe: true,
    canUpdateMetadata: true,
    // when set to true, the agent won't be visible to others in the room.
    // when hidden, it will also not be able to publish tracks to the room as it won't be visible
    hidden: false,
  },
});

Agent server type

You can choose to start a new instance of the agent for each room or for each publisher in the room. This can be set when you register your agent server:

@server.rtc_session(agent_name="my-agent", type=ServerType.ROOM)
async def my_agent(ctx: JobContext):
    # ...

const opts = new ServerOptions({
  // path to the agent module, which must export a default Agent object
  agent: fileURLToPath(import.meta.url),
  // when omitted, the default is JobType.JT_ROOM
  serverType: JobType.JT_ROOM,
});

The ServerType enum has two options:

• ROOM: Create a new instance of the agent for each room.
• PUBLISHER: Create a new instance of the agent for each publisher in the room.

If the agent is performing resource-intensive operations in a room that could potentially include multiple publishers (for example, processing incoming video from a set of security cameras), you can set agent server_type to JT_PUBLISHER to ensure that each publisher has its own instance of the agent.

For PUBLISHER jobs, call the entrypoint function once for each publisher in the room. The JobContext.publisher object contains a RemoteParticipant representing that publisher.

Starting the agent server

To spin up an agent server with the configuration defined in the AgentServer constructor, call the CLI:

if __name__ == "__main__":
    cli.run_app(server)

cli.runApp(opts);

The Agents agent server CLI provides two subcommands: start and dev. The former outputs raw JSON data to stdout, and is recommended for production. dev is recommended to use for development, as it outputs human-friendly colored logs, and supports hot reloading on Python.

Log levels

By default, your agent server and all of its job processes output logs at the info level or higher. Configure the log level in any of the following ways:

• Set the LIVEKIT_LOG_LEVEL (Python) or LOG_LEVEL (Node.js) environment variable.
• Pass log_level to AgentServer in Python.
• Use the --log-level CLI flag when starting the agent server.

The CLI flag takes precedence over the environment variable, which takes precedence over the value set in code.

Environment variable

Set the environment variable to configure the log level without changing your code or startup command. This is useful for deployment environments where you want to adjust log verbosity without rebuilding your agent. The agent reads the variable when it starts directly, from your terminal or a Dockerfile:

LIVEKIT_LOG_LEVEL=debug uv run src/agent.py start

LOG_LEVEL=debug node dist/main.js start

You can also add it to your .env.local file alongside your other LiveKit credentials:

LIVEKIT_LOG_LEVEL=debug

LOG_LEVEL=debug

Server options parameter

Pass log_level to AgentServer to set the log level in code:

server = AgentServer(
    log_level="debug",
)

This applies when you start the agent server directly. The lk agent commands set the log level themselves, so use the --log-level flag with the CLI.

In Node.js, cli.runApp always applies its own log level, so logLevel in ServerOptions has no effect. Use the environment variable or the CLI flag instead.

CLI flag

Pass --log-level when starting the agent server to override the log level at startup:

lk agent start --log-level=debug

Available log levels

The following log levels are available:

• trace: Very detailed tracing information.
• debug: Detailed information for debugging.
• info: Default level for general information.
• warn: Warning messages.
• error: Error messages.
• critical (Python) or fatal (Node.js): Critical error messages.

Deployment environment variable

LiveKit Cloud sets the LIVEKIT_AGENT_DEPLOYMENT environment variable on every agent's containers, regardless of which deployment it runs in. Your worker registers under the right deployment automatically — no code change is required.

The value tells you which deployment the agent is running in:

• Production: LIVEKIT_AGENT_DEPLOYMENT is set to an empty string (and is unset when running locally).
• Non-production deployment: LIVEKIT_AGENT_DEPLOYMENT is set to the deployment name, for example staging.

Use this variable to branch on the current deployment at runtime. To learn more and for example code, see Branch on deployment name at runtime.