Liquid AI Trained LFM2.5-2.6B Inside The Harnesses It Ships In. 2.6B Beats 9.7B Qwen On Tool Use. The On-Device Agent Era Started August 4.
Liquid AI Trained LFM2.5-2.6B Inside The Harnesses It Ships In. 2.6B Beats 9.7B Qwen On Tool Use. The On-Device Agent Era Started August 4.
Hey guys, Mr. Technology here.
On August 4, 2026, Liquid AI shipped LiquidAI/LFM2.5-2.6B — 2.69B parameters, 128K context, 220 tok/s on Apple M5 Max, 113 tok/s on Ryzen AI Max+ 395, 30 tok/s on a phone, under 2.5 GB of memory. It beats Qwen3.5-9B (9.7B parameters, roughly 4x the size) on every instruction-following benchmark Liquid published and on every tool-use benchmark except BFCLv4. None of that is the story.
The story is that Liquid trained this model inside the production harnesses it ships with — OpenClaw, Hermes Agent, Pi — using a "Harness Proxy" that treats those harnesses as black boxes and transparently captures token-level trajectories during rollouts, with no harness modification. The agent surfaces your production stack already runs on are the surfaces this model was shaped against. That is a different category of post-training than anything the closed labs have published in 2026, and it is the difference between "model that benchmarks well on tool use" and "model that drops into your existing harness without a glue layer."
The Training Pipeline, Quickly
Four stages. Stage 1 is supervised fine-tuning — two rounds, broad coverage then targeted agentic shaping. The SFT mix is roughly seven times the size of Liquid's prior LFM2.5-8B-A1B run, weighted toward tool use, web search, software engineering, and agent traces. Stage 2 is teacher specialization — one expert per domain (instruction following, math, knowledge including hallucination control, code, tool use, long context), each RLVR-tuned independently. Standard routing-specialization playbook.
Stage 3 is Multi-Domain On-Policy Distillation (MOPD). Standard distillation is off-policy — the student learns from trajectories generated by a different teacher. MOPD is on-policy: the student rolls out under its own policy, each prompt routes to the appropriate domain teacher, and the teacher supervises with token-level feedback. Because the teachers branch from the same SFT checkpoint as the student, no distribution shift, no destabilized training, faster convergence. The student integrates specialist capabilities without paying the cost of distilling from a model with a different prior.
Stage 4 is Agentic Reinforcement Learning. This is the headline. Multi-turn RL inside the actual production harnesses. Liquid used a Harness Proxy — a black-box wrapper around OpenClaw, Hermes Agent, Pi, and others — to transparently capture token-level trajectories during rollouts. The reward is outcome-based: LLM-as-judge rubric + programmatic checks + a hard safety gate. Optimizer is GRPO (verl orchestrator, SGLang Rollout Engine, FSDP Training Engine). Rollouts execute in a Sandbox Service with a real agent environment.
The consequence: the model's tool-calling schema, system-prompt parsing, multi-turn interaction patterns, and message-passing conventions were shaped against the same harnesses your production stack already uses. Not a private internal harness. Not a synthetic tool-call format. The harnesses the model will be deployed in.
When you stand up LFM2.5-2.6B behind an OpenAI-compatible endpoint and point OpenClaw at it, the tool calls parse, the multi-turn flow works, the error recovery patterns match. There is no glue layer because the model was trained on the glue.
The Architecture Is A Footnote
The LFM2 architecture is worth one paragraph. 30 layers: 22 are double-gated short convolution blocks, 8 are grouped-query attention. The short-convolution stacks carry the bulk of the work — Liquid published a paper last year showing carefully designed short-range convolutions match or beat transformer quality at sub-3B parameters with substantially better decode latency. The 8 GQA layers handle long-range dependencies. Vocabulary is 128K, doubled from the prior LFM tokenizer in-place without retraining from scratch. The doubling matters for non-English scripts — a Chinese sentence in LFM2 was ~1.4x the tokens of the same sentence in English. In LFM2.5 it is ~1.1x. For multilingual agent stacks that changes the prompt construction cost calculus.
The Benchmarks, Honestly Read
Liquid published a 12-row eval against gemma-4-E2B-it (5.1B), gemma-4-E4B-it (8B), Qwen3.5-4B (4.7B), and Qwen3.5-9B (9.7B):
| Benchmark | LFM2.5-2.6B | Qwen3.5-9B | Winner |
|---|---|---|---|
| IFBench | 59.17 | 56.47 | LFM |
| Multi-IF | 80.07 | 62.55 | LFM +17.5 |
| IFStruct | 85.49 | 78.50 | LFM |
| BFCLv4 | 56.88 | 60.13 | Qwen +3.25 |
| ToolSandbox | 77.83 | 76.44 | LFM |
| τ³-Banking | 5.67 | 5.15 | LFM |
| Claw-Eval (EN) | 62.85 | 66.53 | Qwen +3.68 |
| PinchBench | 68.22 | 71.45 | Qwen +3.23 |
| BrowseComp+ (OpenClaw) | 26.89 | 27.23 | Tied |
| AIME25 | 51.87 | 56.07 | Qwen +4.20 |
| LiveCodeBenchv6 | 59.41 | 69.86 | Qwen +10.45 |
| AA-Omniscience | -29.50 | -50.43 | LFM (less negative) |
The honest summary: LFM2.5-2.6B is the best sub-3B model for tool use and instruction following shipping in August 2026. It is not the best sub-10B model for coding (LiveCodeBenchv6 loses by 10.45) or pure agentic (Claw-Eval, PinchBench close losses). If your workload is "agent that calls APIs and follows structured instructions," LFM2.5-2.6B wins. If your workload is "write 500 lines of Python and unit-test them," Qwen 3.5 9B is still your pick.
The Inference Story
Day-one support across llama.cpp, MLX, vLLM, SGLang, ONNX. CPU numbers: 220 tok/s on M5 Max, 113 tok/s on Ryzen AI Max+ 395, under 2.5 GB memory. The short-convolution dominance pays off here — convolution blocks have substantially lower KV-cache pressure than transformer blocks at decode time. Phone: 30 tok/s, real agent speeds on real phone hardware. GPU: ~15K output tokens/sec at high concurrency on a single H100 SXM5 — roughly 1.3 billion tokens per day on a single GPU. The throughput that changes the math on whether on-device deployment is a marketing line or a budget line.
What This Means
The agent stack in 2027 looks like this: small on-device models handle 90% of agent calls (tool use, structured extraction, document Q&A, summarization, multi-step orchestration on local data). Large cloud models handle the 10% that needs frontier reasoning (planning novel tasks, generating 500-line code modules, complex multi-document synthesis). The cost structure flips from "everyone pays per-token to a cloud API" to "cloud bills drop 80%, on-device compute costs are amortized to zero per call."
LFM2.5-2.6B is the first model in 2026 that makes the 90% line credible. Not "credible for demos" — credible for production. A small model trained on synthetic tool-call traces benchmarks fine and deploys badly. A small model trained inside the actual harnesses it ships in benchmarks fine and deploys with zero friction.
The closed labs cannot do this. Anthropic, OpenAI, Google do not have the option to ship a 2.6B model with 30 tok/s on a phone — their unit economics demand cloud-only delivery. Liquid AI can, and did, this week. Every harness update Liquid makes to OpenClaw or Hermes Agent is implicitly a training surface for the next LFM2.5 checkpoint. Every new tool-call schema in the harness ecosystem becomes a capability LFM2.5 picks up faster than a model trained on offline traces. The on-device agent era started August 4. You ship it.
What To Ship This Week
1. Pull LiquidAI/LFM2.5-2.6B-GGUF and serve it with llama serve -hf. Point your existing OpenClaw or Hermes Agent at the OpenAI-compatible endpoint. Expect tool calls to parse without adapter code — that is the part the harness-trained RL paid for.
2. Profile the on-device case. Audit your agent call volume by use case. Identify the 80% that are tool use, structured extraction, or local-data orchestration. That is your on-device candidate set.
3. Run the harness benchmark. Compare tool-call success rate, multi-turn completion rate, and recovery-from-error rate vs your current endpoint. The numbers should match or exceed your current setup. If they do not, look at your system prompts — the model was trained on the harness defaults, not on custom system prompts.
4. Re-price your 2027 inference budget. If your agent call distribution is 90/10 small/large, your 2027 inference bill drops 60-80% just from the routing change. The savings pay for the deployment engineering in one quarter. The hard part is not the model — it is the on-device rollout, the memory budget, and the prompt-tuning pass to drop your custom system prompts back into the harness defaults where they belong.
— Mr. Technology
Sources: