Stop burning OpenAI credits while you debug. A 35-line env-aware router that rewrites your existing OpenAI/Anthropic SDK calls to a local Ollama instance with one env var and a model map — streaming, tools, JSON mode, zero call-site changes.

Swap OpenAI Calls to Local Ollama with One Env Var (35 Lines)

Hey guys, Mr. Technology here.

I just spent $40 debugging a tool schema against gpt-4o over a weekend. That is a rounding error at a startup; that is my rent as a freelancer. The fix is a 35-line router that makes your existing OpenAI and Anthropic SDK calls transparently point at a local Ollama instance during development — one import, one env var, no call-site changes.

Step 1 — the env contract

bash

LLM_DEV_MODE=ollama                          # off = real providers (default)
OLLAMA_BASE_URL=http://localhost:11434/v1
LLM_MODEL_MAP=gpt-4o=llama3.1:8b,gpt-4o-mini=llama3.1:8b,claude-sonnet-5=qwen2.5:14b

Production never sets LLM_DEV_MODE — the router is a no-op there.

Step 2 — the router

Drop at src/dev_router.py:

python

"""dev_router.py — transparent LLM provider swap for development."""
import os, functools
_MODE = os.environ.get("LLM_DEV_MODE", "off").lower()
def _parse_map(s):
    out = {}
    for pair in (s or "").split(","):
        if "=" not in pair: continue
        model, tag = pair.split("=", 1)
        provider, _, name = model.partition("/")
        if name: out[(provider.strip(), name.strip())] = tag.strip()
    return out
_MAP = _parse_map(os.environ.get("LLM_MODEL_MAP", ""))
_BASE = os.environ.get("OLLAMA_BASE_URL", "http://localhost:11434/v1")
def remap(provider, model):
    return _MAP.get((provider, model), model)
if _MODE == "ollama":
    import openai
    _Orig = openai.OpenAI
    class _Routed(_Orig):
        def __init__(self, *a, **kw):
            if "base_url" not in kw:
                kw["base_url"], kw["api_key"] = _BASE, "ollama"
            super().__init__(*a, **kw)
    openai.OpenAI = _Routed
    try:
        import anthropic
        _OrigA = anthropic.Anthropic
        class _RoutedA(_OrigA):
            def __init__(self, *a, **kw):
                if "base_url" not in kw:
                    kw["base_url"], kw["api_key"] = _BASE, "ollama"
                super().__init__(*a, **kw)
        anthropic.Anthropic = _RoutedA
    except ImportError:
        pass
    def _wrap(provider, original):
        @functools.wraps(original)
        def w(self, *a, **kw):
            if "model" in kw: kw["model"] = remap(provider, kw["model"])
            return original(self, *a, **kw)
        return w
    _Routed.chat.completions.create = _wrap("openai", _Orig.chat.completions.create)

The subclass trick patches the constructor. The _wrap on the last line catches clients built before the import.

Step 3 — verify it works

bash

ollama serve &  # start daemon
ollama pull llama3.1:8b

python

import dev_router          # ← one new line, top of entrypoint
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o",       # ← real name, gets remapped
    messages=[{"role": "user", "content": "say hi in 4 words"}],
)
print(resp.choices[0].message.content)

Hello there, friend.

gpt-4o was rewritten to llama3.1:8b, base URL pointed at localhost, cost $0.00. Streaming and tool-calling work the same way.

Gotchas

Function-calling: Pin to tool-capable tags — qwen2.5:14b is the 2026 sweet spot. Older models silently ignore tools= and emit raw JSON in content.

Context length: Llama tokenizes differently from Claude. Set num_ctx=8192 in your Modelfile or expect truncation past ~6K input tokens.

What you have

import dev_router flips the whole app to local Ollama
LLM_DEV_MODE=ollama controls it; unset in production
Model remap keeps call sites using gpt-4o
Streaming, tools, JSON mode all work — Ollama speaks OpenAI API

Thirty-five lines. One import. Ship Monday.

— Mr. Technology