Xiaohongshu's dots-studio Just Dropped a 280B Multimodal MoE With Apache 2.0 Weights. dots3-note Preview Is Real.

Hey guys, Mr. Technology here.

It is Wednesday, August 19, 2026. Five days ago, dots-studio — Xiaohongshu's AI lab (the company behind RedNote) — shipped dots3-note preview on Hugging Face. 280B sparse MoE, 16B activated per token, text/image/video/audio → text, Apache 2.0 licensed, 512K context, MTP speculative decoding, FP8 on 8×H100 via SGLang or vLLM same day as release. The architecture is the story. Not the benchmarks.

The Architecture Is The Actual Story

The dots3 family uses a hybrid attention pattern that is structurally different from standard full-attention stacks. 13 DSA layers (Deep Sparse Attention, top-2048 tokens) plus 33 SWA layers (Sliding Window Attention) at a roughly 1:3 ratio. The DSA layers solve the "find the signal in the whole context" problem. The SWA layers handle local context without O(n²) cost.

This is the same pattern DeepSeek deployed in V3 and V4. DSA+SWA at 1:3 is becoming the canonical architecture for long-context sparse MoE inference. By end of 2026 you will see it in every serious open-weights release.

The MoE routing is clean: 256 routed experts, top-8 activation, plus 1 shared expert that fires on every token — handling instruction following, format, and safety while the routed experts specialize. The vision encoder is a MoE ViT (7B/1.2B activated per image token). Audio is a dense 800M encoder. Both are multimodal-pretrained and joint-tuned with the language model.

The Deployment Story Is Right

dots-studio shipped inference stacks the same day as the weights. That is the correct move. The FP8 vLLM recipe on 8×H100:

vllm serve dots-studio/dots3-note-prev-fp8 \
  --served-model-name dots3-note-prev \
  --tensor-parallel-size 8 --enable-expert-parallel \
  --moe-backend deep_gemm --max-model-len 262144

The SGLang Docker image is lmsysorg/sglang:dev-dots3-note. The key flags: --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-num-draft-tokens 4. Three speculative tokens per step, 4 draft tokens, using the single shared MTP layer as the draft model. TP=8, EP=8. This is the production config.

For local single-GPU use:

from transformers import AutoModelForMultimodalLM, AutoProcessor
model = AutoModelForMultimodalLM.from_pretrained("dots-studio/dots3-note-prev-fp8", dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("dots-studio/dots3-note-prev-fp8")
inputs = processor.tokenizer.apply_chat_template(messages, return_tensors="pt", enable_thinking=False).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)

enable_thinking=True enables reasoning mode.

The Honest Caveat: Community Says The Model Is Weak

The top HuggingFace discussion on the model card, opened two days after release, is titled "The model is very weak." Community reception is negative. dots3-note is the lightweight member of the dots3 family, positioned for cheap deployment, not capability leadership. No independent third-party evals (Artificial Analysis, SWE-Bench, LMSYS) have circulated. The model card numbers are self-reported and should be treated as directional only.

This is a first-release open-weights model from a new lab. It is interesting as a technical artifact and as a signal of Xiaohongshu's ambitions. It is not a frontier capability claim.

Why It Matters Anyway

Apache 2.0 on a multimodal MoE from a Chinese consumer internet company is a signal. Alibaba, Baidu, and ByteDance have shipped open weights under custom or research licenses with various commercial restrictions. dots3-note is Apache 2.0 — any team can build on it commercially without a legal review. The model is weak today. The license is not.

The DSA+SWA hybrid is now canonical. This is the clearest public reference implementation with full open weights and documented inference recipes. If you are evaluating long-context attention patterns for your stack, this is worth a look.

This is Xiaohongshu's opening move. RedNote has 300M+ daily active users and a deep multimodal content graph. dots3-note is the first public statement of their model architecture. The preview is weak. The trajectory of a lab with this data moat, shipping open weights, is worth watching.

What To Do With It Today

Do not use it for production agentic work. The community signal is clear.

If you are studying multimodal MoE inference stacks: run the SGLang Docker image. lmsysorg/sglang:dev-dots3-note, TP=8, EP=8, NEXTN speculative decoding. Clean reference deployment.

File it under models to watch. The dots3 family will expand. This checkpoint is not the story. The trajectory is.

The Take

dots3-note preview is not the story of a strong frontier model. It is the story of a major Chinese consumer platform entering the open-weights race with a clean multimodal MoE, an Apache 2.0 license, and the right inference stack shipped the same day. The DSA+SWA architecture is canonical. The MTP speculative decoding path is correct. The model itself is weak relative to V4-Pro-0813 or Qwen3.8-Max.

What makes it worth writing about is what it represents: dots-studio has the data moat (RedNote's multimodal content graph), the inference engineering (SGLang Docker image, vLLM recipe, NEXTN), and the Apache 2.0 posture to become a serious open-weights player by Q1 2027.

Watch the dots3 family. Not this checkpoint.

— Mr. Technology

Model: dots3-note preview · Lab: dots-studio (Xiaohongshu/RedNote) · Released: Aug 14, 2026 · Architecture: 280B sparse MoE (16B activated), 1 dense + 45 MoE layers, 256 routed + 1 shared expert, top-8, 13 DSA + 33 SWA layers (1:3 ratio) · Vision: MoE ViT (7B/1.2B) · Audio: Dense 800M · Modalities: text + image + video + audio → text · Context: 512K · MTP: 1 shared layer, 3-token speculative decoding via NEXTN · License: Apache 2.0 · Deployment: FP8 on 8×H100 (SGLang or vLLM) · Sources: Hugging Face model card · SGLang cookbook · OpenRouter free tier