Leanstral 1.5 Cracked 587 Putnam Problems for $4 Each. Formal Verification Just Got Cheap.

Hey guys, Mr. Technology here.

Mistral shipped Leanstral 1.5 on July 3, 2026 — a 119B-total / 6.5B-active sparse MoE, Apache 2.0 licensed, free in Mistral Labs, weights on Hugging Face. The headline number is 587 of 672 problems solved on PutnamBench, which is the formalization of the Putnam Math Competition, which is the closest thing the North American university system has to "find the derivative of x³." That is 87.4% of one of the hardest formal-math benchmarks on the planet.

The line that should make every founder sit up is the second one: ~$4 per Putnam problem. DeepMind's Seed-Prover 1.5 high — the closed-source leader Leanstral just edged — runs at 10 H20-days per problem, which works out to roughly $300+ per problem if you have to pay for the compute at all. Leanstral is ~75× cheaper than the best closed model on the benchmark, and it ships the weights for free.

Read those two sentences again. The frontier of formal mathematical reasoning just went from "you need a closed frontier lab's allocation to touch this" to "you can curl it on the free Mistral Labs tier tonight, or git clone it in the morning."

The benchmark table, because you will want it

This is the leaderboard Leanstral 1.5 ships with, against the other named provers Mistral benchmarked against. Costs are vendor-cited where available, my best estimate otherwise:

Two qualifiers Mistral is honest about: Aleph Prover scores higher on PutnamBench, but it costs 13–17× more per problem and only runs behind an API. Seed-Prover 1.5 high was reportedly nudged with natural-language proof hints that Leanstral does not get. Under fair conditions — same hints, same compute — Leanstral is the open-source leader and the cost-per-solved-problem leader. Period.

What 119B / 6.5B MoE actually buys you

Leanstral 1.5 is a sparse mixture-of-experts with 128 experts per layer, 4 active per token. That is the same architecture Mistral has been refining since the March Leanstral-2603 release (which itself grew out of Mistral Small 4). The point is not the parameter count. The point is that per-token compute is bounded — so you can spin it up on a single DGX Spark, batch it cheaply in inference, and let a proof attempt run for 2.7 million tokens without imploding.

Leanstral did exactly that on an AVL-tree insertion/deletion time-complexity proof: 2.7 million tokens, 22 context compactions, and the model kept editing the proof, running lean against it, reading the compiler error, and iterating. The proof it produced is structurally tight — it unfolds the TimeM monad layer by layer, mirrors the tree's recursive structure via structural induction, and ends with a clean O(log n) bound on insertion and deletion. That is the kind of artifact that, until two weeks ago, required a closed frontier lab's budget and a human Lean expert on call.

Test-time scaling is the real story

Here is the line from Mistral's blog that buried the lede:

Pass@8 on PutnamBench vs token budget per attempt:
  50k   tokens ->   44 problems solved
  200k  tokens ->  244 problems solved
  1M    tokens ->  493 problems solved
  4M    tokens ->  587 problems solved

That is monotonic scaling across four orders of magnitude. Every doubling of compute buys you more solved problems, all the way out to 4M tokens per attempt. Most LLMs hit a wall around 200k — the reasoning gets worse, not better, when you let it ramble. Leanstral is the first model I have seen where the curve does not bend. The cost math flips on its head: spending more compute per attempt is strictly worth it, which is the opposite of how you normally price frontier inference.

This is partly a property of the RL loop. Mistral trained with CISPO — a clipped surrogate variant designed for sparse, long-horizon reasoning tasks where the reward signal only fires when the proof compiles. The model gets a Lean compiler error every turn, refines, and tries again. It is the same agentic-training topology GLM-5.2 and OpenPipe-ART/GRPO use, but applied to formal proofs where the verifier is the Lean kernel rather than a unit test. The kernel does not grade on vibes. The kernel either accepts the proof or it does not.

The bug-finding is the part developers should care about

Mistral ran Leanstral against 57 open-source Rust repositories via an Aeneas (Rust→Lean) translation pipeline. The model inferred the developer's intent, generated correctness properties, and tried to prove them. Out of the 47 properties Leanstral flagged as violated, 11 pointed to real bugs — 5 of them previously unknown to the maintainers.

One concrete example: in datrs/varinteger, the zigzag decoder's sign function overflowed on Std.U64.MAX, because (value + 1) wraps. In debug mode it crashes; in release mode it silently corrupts output. That is the kind of edge case fuzzing and standard property tests routinely miss. A Lean-fluent model that can flip and try to prove the negation of a property after the proof fails is the first bug-finding tool I would actually trust on a security-critical Rust crate.

Why this is bigger than the headline number

Two things are happening at the same time and they reinforce each other:

1. The cost of formal verification fell by ~75× in one release. Same magnitude of collapse as Grok 4.5 vs Opus 4.8 on token pricing, but applied to a category that was previously gated entirely by closed labs. 2. The model is Apache 2.0. The weights are on Hugging Face right now. You can fine-tune, distill, or self-host Leanstral without calling Mistral, paying Mistral, or asking Mistral. That is the same open-weights move that made GLM-5.2 from Zai the default coding model in a dozen Asian enterprise stacks over the last six weeks.

The compounding effect: every Rust / Lean / Coq / Isabelle team that was on the fence about formal verification just got handed a free, fine-tunable model that hits near-frontier on the math benchmarks and demonstrably finds real bugs. The "we will get to formal methods in v3" roadmap slides at your company are no longer defensible.

What to watch

One, whether Mistral ships the CISPO recipe in a paper. RL for long-horizon formal proofs is the most important open-research question in agentic training right now. If CISPO turns out to be a clean 20-line patch to GRPO, every other frontier lab will copy it within a quarter.

Two, whether Anthropic responds with an Opus-class prover. Claude Sonnet has been the de facto "reasoning about code" model since the March 2603 Leanstral release showed the open-source gap. If Anthropic ships a Mythos-shaped formal-verification model, the open-source advantage evaporates. If they do not, Leanstral is the default.

Three, the bug-pipeline tooling. Aeneas (Rust→Lean) plus a property-generation model plus Leanstral is a turnkey formal-verification product the moment someone wraps it in a GitHub Action or a Claude Code hook. Expect this to ship as a YC startup before the end of Q3. I would bet on it.

— Mr. Technology

Released: July 1-3, 2026. Model: Mistral Leanstral 1.5 (119B total / 6.5B active MoE, 256k context, Apache 2.0). Benchmarks: 100% miniF2F, 587/672 PutnamBench, 87% FATE-H, 34% FATE-X, FLTEval pass@8 43.2 (beats Opus 4.6's 39.6 at ~1/7 cost). Training: mid-training + SFT + CISPO RL. Cost: ~$4 per Putnam problem vs ~$300+ for Seed-Prover 1.5 high. Free in Mistral Labs and Hugging Face. Found 5 previously unknown bugs across 57 Rust repos via an Aeneas pipeline; proved AVL-tree time complexity over 2.7M tokens and 22 context compactions. Sources: Mistral — Leanstral 1.5: Proof Abundance for All, The Decoder — Mistral's open-source Leanstral 1.5 aces formal math benchmarks, MarkTechPost — Mistral AI Releases Leanstral 1.5, DevOps.com — Mistral Releases Leanstral 1.5, Hugging Face — Leanstral-1.5-119B-A6B model card, AI Weekly — Mistral ships Leanstral 1.5 free in Labs.