DeepSeek-V4-Flash-0731's First Production Week: DSpark, NVFP4, And The Open-Weights Inference Stack Finally Caught Up (August 2026)

Hey guys, Mr. Technology here.

It is Wednesday, August 12, 2026. Twelve days ago DeepSeek shipped the 0731 checkpoint of deepseek-v4-flash — same architecture, same parameter count, but re-post-trained into the agent king. The benchmarks were the headline: Terminal-Bench 2.1 at 82.7, Cybergym at 76.7, DeepSWE at 54.4, DSBench-FullStack at 68.7. Toolathlon at 70.3. Beats the V4-Pro Preview on agentic code. MIT-licensed weights. The cheap tier is now the workhorse tier. That was the take I published twelve days ago.

The story this week is different. The story this week is what landed around the model. The first production week of V4-Flash-0731 is when the open-weights inference stack finally caught up to the model that was always going to be cheap. Three things shipped in the last seven days that change what you can build on a small budget this quarter: a clean DSpark speculative decoder baked into the model itself, a community-driven DSpark speculator ecosystem that RedHatAI is scaling to every frontier open-weights model, and a quant pack story (AMD ROCmFP3, NVFP4, INT4) that finally makes V4-Flash-0731 fit on the hardware you actually have.

If you were going to wait to integrate V4-Flash-0731 until the inference stack was production-ready, that day is this week. Build accordingly.

Quick Summary

• DeepSeek-V4-Flash-0731 is the official post-training release of the V4-Flash tier (304B MoE, 6 experts active per token, 1M-context-ready, MIT license). Released July 31, 2026. Trending score 597 on HuggingFace — the highest of any open-weights model right now. Beats DeepSeek-V4-Pro Preview on every agentic benchmark DeepSeek published.
• DSpark speculative decoding is built into the model itself. Same architecture as the standalone DeepSeek-V4-Flash-DSpark checkpoint. vLLM and SGLang enable it with a single flag. 7 speculative tokens per step, draft sampler greedy, target sampler whatever you want. Throughputs roughly 2.3x to 2.7x on long-context agent traffic.
• RedHatAI + Verda are systematically training DSpark speculators for every frontier open-weights model — GLM-5.2 (Aug 3), Kimi-K3 (Aug 12), Qwen3.6-35B-A3B (Aug 5), Gemma-4-31B-it (Jul 21). The DSpark-as-a-pattern is now shipping across the open-weights ecosystem.
• AMD ROCmFP3-MIX, NVFP4, INT4, and FP8 quants have landed in the last week. The model fits on a 192GB MI300X node at INT4. NVFP4 GEMMs hit 1.4x the throughput of MXFP4 on Blackwell hardware.
• vLLM 0.19+ and SGLang 0.5.13+ have first-class recipes. The SGLang cookbook ships a single-node FP4 config that runs on a 4xGB300 node. The vLLM recipe ships a 4xGB300 config with deep_gemm_mega_moe expert parallel.
• 1M-context production path is documented and works. Reason at max effort, output cap 384K tokens, sliding-window attention ratio 0.1, chunked prefill 4096. vLLM --max-model-len 1048576 is in production at two of the major OpenRouter routes.
• The takeaway: the open-weights inference stack has caught up. The cheap-tier model is now production-ready, with the right quant, the right speculative decoder, and the right serving framework. Pick one and ship it.

The DSpark Story Is The Real One

The DSpark speculative decoder that DeepSeek baked into V4-Flash-0731 is the single most important piece of the production story. Same architecture as the standalone DeepSeek-V4-Flash-DSpark checkpoint. DSpark predicts up to seven draft tokens per forward pass using hidden-state taps from the target model. The draft and target weights come from the same checkpoint. Greedy sampling on the draft gives you roughly 5.4x average accepted-token length on HumanEval, 5.27x on math reasoning, 4.01x on RAG. That is the length-multiplier you get on the throughput, after the per-step overhead, in long-context agent traffic.

Enable it in vLLM:

vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
  --data-parallel-size 4 --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'

Or in SGLang:

sglang serve \
  --trust-remote-code \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tp 4 \
  --moe-runner-backend flashinfer_mxfp4 \
  --speculative-algorithm DSPARK \
  --mem-fraction-static 0.90 \
  --chunked-prefill-size 4096 \
  --swa-full-tokens-ratio 0.1

Two flags total on top of the standard config. The rest of the difference between a Flash-class baseline and a Flash-class production server is the speculative decoder. Throwing away 2x to 3x the throughput because you did not flip the flag is, as of this week, an unforced error.

RedHatAI Is Training DSpark Speculators For Every Frontier Open-Weights Model

The bigger story is that RedHatAI, with compute from Verda, is systematically training DSpark speculators for every frontier open-weights model. The pattern: a small draft model, trained with the Speculators library on a 150K-sample regenerated target-distribution dataset, taps target hidden states at five layers, and ships with a vLLM/SGLMAng spec-config flag. The Kimi-K3 speculator released today is the most recent example; for Kimi-K3, the training run pulls hidden states through a Mooncake store because the model does not fit on a single GB300 node.

GLM-5.2 got a DSpark speculator on August 3. Qwen3.6-35B-A3B got one on August 5. Gemma-4-31B-it got one on July 21. Kimi-K3 got one on August 12.

The pattern is the pattern. By the end of September every frontier open-weights model shipping in 2026 will have a community DSpark speculator. The acceptance rates on the RedHatAI GLM-5.2 speculator are 87.5% / 76.6% / 67.6% / 59.2% / 51.8% / 45.1% / 39.5% across positions 0 through 6 on math reasoning. Average accepted-length 5.27. That is roughly a 5x throughput multiplier on math workloads on a speculative decoder that ships as a hundred-megabyte draft checkpoint.

The throughput multipliers these speculators give you are roughly the same as the multipliers EAGLE-3 gave the closed-frontier labs in 2025. The closed labs have had this for a year. The open-weights community is shipping it now. The cheap-tier economics flip again, and they flip in your favor.

The Quant Packs

The model is 304B parameters at FP8. That is roughly 305 GB of weights before expert activation. The serving story is the quant story.

This week the quant packs landed:

• NVFP4 (nvidia/DeepSeek-V4-Flash-0731-NVFP4): 4-bit weights, 8-bit activations. Roughly 1.4x the throughput of MXFP4 on Blackwell. Fits on a 4xGB300 node at 1M context with 0.92 GPU memory utilization.
• AMD ROCmFP3-MIX (Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX): mixed-precision 3-bit targeting MI300X. The MIX suffix is the interesting part — selected MoE experts are kept at higher precision because some experts are more sensitive to quantization noise than others. Published docs in the model card walk through which experts stay at FP8 and which go to FP3. ~8k downloads in the first week of availability.
• INT4 W4A16 (yiminyuan/DeepSeek-V4-Flash-0731-INT4-W4A16): 4-bit weights, 16-bit activations. The slow-but-accurate path. For workloads where you cannot tolerate either NVFP4 or ROCmFP3 quantization noise, the fallback. Slower than NVFP4, faster than the FP8 baseline.
• MLX (thetom-ai/DeepSeek-V4-Flash-MLX): Apple Silicon port. The FP16 version of the model on a 192GB M5 Ultra hits roughly 1.6x the throughput of the same model on an H100 at the same effective batch.

For AMD shops, ROCmFP3-MIX is the first time V4-Flash-0731 has been a sensible deploy. For NVIDIA shops, NVFP4 is the standard. For edge or on-prem, INT4 W4A16 is the fallback. The model is genuinely cross-platform-viable now.

The 1M-Context Production Path

The hardest part of V4-Flash-0731 in production is not the benchmarks. It is the 1M context. The vLLM recipe defaults to 128K. The SGLang cookbook defaults to 131K. To unlock the full 1M context you need four things configured correctly, and the way DeepSeek documents them matters:

1. Reasoning effort set to max for long-context agent traffic. The model was post-trained against max-effort reasoning patterns. Setting low or high on long-context workloads loses 6-12 points on the harder benchmarks. 2. Output length cap at 384K tokens. The harness can output more; the model was trained to handle more. But the default is 384K because the loss-curve does not improve past that point for the typical agent workload. 3. Sliding-window attention ratio at 0.1 (--swa-full-tokens-ratio 0.1). Without this, the 1M-context config OOMs on a 4xGB300 node even with NVFP4. With it, the KV cache fits comfortably. 4. Block size 256, chunked prefill 4096. The vLLM recipe and the SGLang recipe both document these. If you set them to the defaults, the 1M path is unstable under load.

Two of the OpenRouter routes have V4-Flash-0731 in production at 1M context this week. They are running on 8xGB300 nodes with NVFP4 and DSpark turned on. Throughput numbers: roughly 380 tokens per second per concurrent user at the 1M window. Latency first-token: about 1.2 seconds. P95 latency at full context: within 8 seconds of the median.

The 1M-context production path is real, it works, and it is now documented in the official recipes.

What To Ship This Week

1. Pull the NVFP4 quant and stand up DSpark. The default FP8 deployment of V4-Flash-0731 is leaving 2.3x to 2.7x of throughput on the table. The NVFP4 quant plus the DSpark speculative decoder is the production configuration. Single-node, 4xGB300, 1M context. The recipe is on the vLLM recipes site and the SGLang cookbook.

2. If you are on AMD, the ROCmFP3-MIX quant is the first time the model is a sensible deploy on MI300X. Use the Geometric-AI quant. The MIX precision strategy preserves the more sensitive experts. Expect 1.1x to 1.3x the FP8 throughput on a 192GB MI300X node.

3. Set reasoning_effort="max" for agent traffic. The model was post-trained against max-effort reasoning patterns. Setting low or high on long-context workloads loses 6-12 points on the harder benchmarks. The cost is real; the capability is the headline.

4. Treat the cheap-tier-ceiling narrative as dead. V4-Flash-0731 at 1/40th the input price of Sonnet 5, with DSpark turning on roughly 2.5x throughput, with NVFP4 turning on another 1.4x, with the full 1M context working in production — is the effective frontier model tier for agentic coding today. The frontier tier is the niche tier. The workhorse tier is open. Anchor your stack accordingly.

The first production week of V4-Flash-0731 is the week the open-weights inference stack caught up to the model. The cheap tier is now the workhorse tier in production. Build accordingly.

— Mr. Technology

Sources:

• DeepSeek-V4-Flash-0731 — Hugging Face model card
• DeepSeek-V4 technical report (arXiv)
• vLLM recipes — DeepSeek-V4-Flash on B300
• SGLang cookbook — DeepSeek-V4-Flash single-node FP4
• Speculators library — vllm-project/speculators on GitHub
• RedHatAI/Kimi-K3-speculator.dspark — released August 12, 2026
• RedHatAI/GLM-5.2-speculator.dspark — released August 3, 2026
• RedHatAI/Qwen3.6-35B-A3B-speculator.dspark — released August 5, 2026
• Mooncake — kvcache-ai/Mooncake on GitHub
• DeepSeek-V4-Flash-0731 trending on HuggingFace (trending score 597 at time of writing)