LLM Providers: Claude, GPT, Gemini, Ollama & More

PocketPaw supports eight LLM providers, from free local models to cloud APIs and multi-provider proxies. This guide covers everything: which provider to pick, which models to use, how they interact with agent backends, and how to configure them.

Quick Start: Which Provider Should I Use?

Provider Details

Anthropic (Default)

The recommended provider for maximum capability. Claude models excel at coding, tool use, and complex reasoning.

Configuration:

export POCKETPAW_LLM_PROVIDER="anthropic"
export POCKETPAW_ANTHROPIC_API_KEY="sk-ant-..."
export POCKETPAW_ANTHROPIC_MODEL="claude-sonnet-4-5-20250929"

Available Models:

Gemini

Google’s Gemini models via their OpenAI-compatible API. Free tier available with generous limits. Internally, PocketPaw routes Gemini through the OpenAI-compatible code path using Google’s endpoint at https://generativelanguage.googleapis.com/v1beta/openai/.

Configuration:

export POCKETPAW_LLM_PROVIDER="gemini"
export POCKETPAW_GOOGLE_API_KEY="AIza..."
export POCKETPAW_GEMINI_MODEL="gemini-2.5-flash"

Available Models:

Ollama (Local)

Run models entirely on your machine. No API keys, no cloud, no costs. Requires Ollama installed and running.

Configuration:

export POCKETPAW_LLM_PROVIDER="ollama"
export POCKETPAW_OLLAMA_HOST="http://localhost:11434"
export POCKETPAW_OLLAMA_MODEL="llama3.2"

Recommended Models:

Verify connectivity:

pocketpaw --check-ollama

This runs 4 checks: server reachable, model available, API compatibility, and tool calling support.

OpenAI

Use OpenAI’s GPT models directly.

Configuration:

export POCKETPAW_LLM_PROVIDER="openai"
export POCKETPAW_OPENAI_API_KEY="sk-..."
export POCKETPAW_OPENAI_MODEL="gpt-4o"

Available Models:

OpenAI-Compatible

Connect to any endpoint that implements the OpenAI Chat Completions API. This includes hosted services (OpenRouter, Together AI, Fireworks) and self-hosted servers (vLLM, LiteLLM, text-generation-inference).

Configuration:

export POCKETPAW_LLM_PROVIDER="openai_compatible"
export POCKETPAW_OPENAI_COMPATIBLE_BASE_URL="https://openrouter.ai/api/v1"
export POCKETPAW_OPENAI_COMPATIBLE_API_KEY="sk-or-..."
export POCKETPAW_OPENAI_COMPATIBLE_MODEL="anthropic/claude-3.5-sonnet"
export POCKETPAW_OPENAI_COMPATIBLE_MAX_TOKENS=0  # 0 = no limit

Popular services:

Verify connectivity:

pocketpaw --check-openai-compatible

This runs 2 checks: API connectivity and tool calling support.

OpenRouter

OpenRouter provides access to 100+ models from multiple providers through a single API. It has a dedicated adapter in PocketPaw with proper authentication handling.

Configuration:

export POCKETPAW_LLM_PROVIDER="openrouter"
export POCKETPAW_OPENROUTER_API_KEY="sk-or-v1-..."
export POCKETPAW_OPENROUTER_MODEL="anthropic/claude-sonnet-4-6"

How it differs from OpenAI-Compatible: While you can use OpenRouter via the generic OpenAI-Compatible provider, the dedicated openrouter provider handles authentication correctly for both the OpenAI and Anthropic API formats. OpenRouter uses ANTHROPIC_AUTH_TOKEN (not ANTHROPIC_API_KEY) for Anthropic-format requests, and the adapter manages this automatically.

Popular models:

LiteLLM (100+ Providers)

LiteLLM is a unified interface to 100+ LLM providers including HuggingFace, Ollama, vLLM, Together AI, Groq, Mistral, and more. PocketPaw supports LiteLLM across all four major backends.

Two modes of operation:

• Proxy mode (recommended): Run a LiteLLM proxy server and point PocketPaw at it. The proxy handles model routing, load balancing, and provider-specific authentication.
• Direct SDK mode: LiteLLM SDK calls providers directly. Requires LiteLLM-prefixed model names (e.g., anthropic/claude-sonnet-4-6).

Configuration (proxy mode):

export POCKETPAW_LLM_PROVIDER="litellm"
export POCKETPAW_LITELLM_API_BASE="http://localhost:4000"
export POCKETPAW_LITELLM_API_KEY="sk-..."       # your proxy master key
export POCKETPAW_LITELLM_MODEL="gpt-4o"          # model name as configured in the proxy
export POCKETPAW_LITELLM_MAX_TOKENS=0             # 0 = provider default

Configuration (direct SDK mode):

export POCKETPAW_LLM_PROVIDER="litellm"
export POCKETPAW_LITELLM_API_BASE=""              # leave empty for direct mode
export POCKETPAW_LITELLM_API_KEY="sk-ant-..."     # the target provider's API key
export POCKETPAW_LITELLM_MODEL="anthropic/claude-sonnet-4-6"  # LiteLLM-prefixed name

Backend support:

Auto (Default)

When llm_provider is set to "auto" (the default), PocketPaw auto-detects the best available provider:

1. Anthropic — if POCKETPAW_ANTHROPIC_API_KEY is set
2. OpenAI — if POCKETPAW_OPENAI_API_KEY is set
3. Ollama — fallback (no key needed)

This means PocketPaw works out of the box with Ollama if no API keys are configured.

Backend Compatibility Matrix

Each backend has different provider support. Here’s the full compatibility matrix:

Claude Agent SDK (Recommended)

The default and most capable backend. Uses the official Claude Agent SDK with built-in agentic tools.

Important: API format requirement. The Claude SDK uses the Anthropic Messages API format. It only works with endpoints that understand this format — not endpoints that only speak OpenAI’s Chat Completions format.

Built-in SDK tools: Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch, Skill

Security: Uses PreToolUse hooks to intercept and block dangerous Bash commands before execution.

MCP servers: Fully supported. Supports stdio, SSE, and HTTP transports.

Best for: Anthropic and Ollama users who want maximum capability, coding tasks, and tool-heavy workflows.

OpenAI Agents SDK

OpenAI’s native agent framework with GPT model support and Ollama compatibility.

Built-in tools: code_interpreter, file_search, computer_use

Best for: OpenAI model users, Ollama users who prefer OpenAI-format APIs, or anyone wanting access to 100+ models via OpenRouter or LiteLLM.

Google ADK

Google’s Agent Development Kit with native Gemini support.

Built-in tools: google_search, code_execution

MCP servers: Native support via McpToolset (stdio, SSE, HTTP).

Best for: Gemini users who want native Google AI integration with MCP support. LiteLLM support enables using non-Gemini models with ADK’s tooling.

Other Backends

• Codex CLI — OpenAI’s Codex CLI for code-focused tasks. Requires codex binary. MCP support.
• OpenCode — External server backend. Model managed by the OpenCode server.
• Copilot SDK — GitHub Copilot with multi-provider support (Copilot, OpenAI, Azure, Anthropic, LiteLLM).

Smart Model Routing

When using Anthropic as the provider, PocketPaw can automatically select the model size based on message complexity. This optimizes cost by using cheaper models for simple queries.

How It Works

Each incoming message is classified into a tier:

Classification Signals

• Simple patterns: “hi”, “hello”, “thanks”, “what is X?”, “remind me”, short messages under 30 characters
• Complex signals: “plan”, “architect”, “debug”, “implement”, “analyze”, “optimize”, “research”
• Threshold: 1 complex signal + message > 30 chars = COMPLEX; 2+ complex signals = COMPLEX; > 400 chars = COMPLEX

Configuration

export POCKETPAW_SMART_ROUTING_ENABLED=true
export POCKETPAW_MODEL_TIER_SIMPLE="claude-haiku-4-5-20251001"
export POCKETPAW_MODEL_TIER_MODERATE="claude-sonnet-4-5-20250929"
export POCKETPAW_MODEL_TIER_COMPLEX="claude-opus-4-6"

Configuration Reference

All LLM-Related Settings

Setting via config.json

All settings can also be set in ~/.pocketpaw/config.json:

{
  "llm_provider": "gemini",
  "gemini_model": "gemini-2.5-flash",
  "smart_routing_enabled": false
}

Setting via Dashboard

Open the web dashboard (default at http://localhost:8888) and go to Settings > General to select your provider and model from dropdown menus.

Security Notes

• API keys are encrypted at rest using Fernet + PBKDF2 in ~/.pocketpaw/secrets.enc. They are never stored in plaintext in config.json.
• Environment variables override config file values and are never written to disk.
• The Gemini provider reuses the google_api_key field, which is also used for image generation. One key covers both features.

Troubleshooting

Ollama: “Model not found”

ollama list              # See available models
ollama pull llama3.2     # Download a model
pocketpaw --check-ollama # Run connectivity check

Gemini: “API key invalid”

1. Go to AI Studio and create/verify your key
2. Make sure the key is saved in Settings > API Keys > Google API Key
3. Ensure the Gemini API is enabled for your Google Cloud project

OpenAI-Compatible: “Cannot connect”

pocketpaw --check-openai-compatible  # Run connectivity check

Verify:

• The base URL is correct and includes /v1 if needed
• The server is running and accessible
• The model name matches what the endpoint expects

Wrong provider being used

If llm_provider is "auto", PocketPaw picks the first available key (Anthropic > OpenAI > Ollama). Set an explicit provider to override:

export POCKETPAW_LLM_PROVIDER="gemini"