Ollama: Run PocketPaw with Free Local Models

Run PocketPaw entirely on your machine with Ollama — no API keys, no cloud, no costs. The Claude Agent SDK and OpenAI Agents backends work with Ollama out of the box.

How It Works

Since Ollama v0.14.0, Ollama exposes an Anthropic Messages API compatible endpoint. PocketPaw points the same AsyncAnthropic client (or Claude SDK subprocess) at your local Ollama server instead of Anthropic’s cloud. Same tool format, same streaming, zero format conversion.

Prerequisites

curl -fsSL https://ollama.com/install.sh | sh

ollama pull qwen2.5:7b       # Good balance of speed and quality
# or
ollama pull llama3.2          # Default model

ollama serve

Quick Start

export POCKETPAW_LLM_PROVIDER=ollama
export POCKETPAW_OLLAMA_MODEL=qwen2.5:7b
pocketpaw

{
  "llm_provider": "ollama",
  "ollama_host": "http://localhost:11434",
  "ollama_model": "qwen2.5:7b"
}

Verify Setup

Run the built-in connectivity check:

pocketpaw --check-ollama

This performs 4 checks:

Configuration

Auto-Detection

When llm_provider is "auto" (the default):

1. If anthropic_api_key is set → uses Anthropic
2. If no API key is set → falls back to Ollama automatically

This means if you install PocketPaw and Ollama without any API keys, it just works.

Compatible Backends

Claude Agent SDK + Ollama

The default backend. The SDK subprocess receives these environment variables:

• ANTHROPIC_BASE_URL → your Ollama host
• ANTHROPIC_API_KEY → "ollama" (accepted but not validated)

All SDK built-in tools (Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch) work as usual.

OpenAI Agents + Ollama

Set the provider to ollama and the model will be wrapped in OpenAIChatCompletionsModel from the OpenAI Agents SDK, communicating with Ollama’s OpenAI-compatible endpoint:

export POCKETPAW_AGENT_BACKEND="openai_agents"
export POCKETPAW_OPENAI_AGENTS_PROVIDER="ollama"
export POCKETPAW_OLLAMA_MODEL="qwen2.5:7b"

Recommended Models

Limitations

• Smart Model Router is skipped — When using Ollama, the Model Router cannot switch between models. Smart routing is automatically disabled.
• Tool calling quality varies — Smaller models may not use tools reliably. If tools aren’t being called, try a larger model.
• Ollama v0.14.0+ required — Older versions don’t expose the Anthropic Messages API endpoint.

Error Messages

PocketPaw provides Ollama-specific error messages instead of generic API errors:

Troubleshooting

”Model not found in Ollama”

This means PocketPaw is trying to use a model you haven’t pulled. The default model is llama3.2 — if you use a different model (e.g., deepseek), make sure to update the setting:

# Check what models you have
ollama list

# Set the correct model
export POCKETPAW_OLLAMA_MODEL="deepseek-r1:8b"

Or update it in the dashboard: Settings → General → Ollama Model.

”Cannot reach Ollama server"

# Check Ollama is running
ollama serve

# Verify it's listening
curl http://localhost:11434/api/tags

"Messages API failed”

Your Ollama version may be too old. Update:

# macOS
brew upgrade ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

“Model responded but did not use the tool”

Try a more capable model:

ollama pull qwen2.5:14b

Then set ollama_model to qwen2.5:14b in Settings or config.

Implementation

Related