Ollama integration guide

How to run models locally using Ollama with LiveKit Agents.

Overview

Ollama is an open source LLM engine that you can use to run models locally with an OpenAI-compatible API. Ollama support is available using the OpenAI plugin for LiveKit Agents.

Usage

Install the OpenAI plugin to add Ollama support:

pip install "livekit-agents[openai]~=1.0"

Create an Ollama LLM using the with_ollama method:

from livekit.plugins import openai
session = AgentSession(
llm=openai.LLM.with_ollama(
model="llama3.1",
base_url="http://localhost:11434/v1",
),
# ... tts, stt, vad, turn_detection, etc.
)

Parameters

This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.

modelstringOptionalDefault: llama3.1

Ollama model to use. For a list of available models, see Ollama models.

base_urlstringOptionalDefault: http://localhost:11434/v1

Base URL for the Ollama API.

temperaturefloatOptional

Controls the randomness of the model's output. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make it more focused and deterministic.

The following links provide more information about the Ollama integration.

Voice AI quickstart

Get started with LiveKit Agents and Ollama.