Xiaohongshu Dots3: Self-Evaluation Unlocks Days-Long AI Agents

A Chinese lifestyle app just shipped the most instructive open model of the month, and it has almost nothing to do with the app itself. Xiaohongshu's Dots Lab open-sourced dots3-note Preview, a 280-billion-parameter multimodal model with a 512K context window, under Apache 2.0. The model is impressive. The training recipe behind it is the real story.

What happened

On August 15, Xiaohongshu's AI research arm, dots.studio, released dots3-note Preview on GitHub and Hugging Face, with weights under the permissive Apache 2.0 license and the architecture submitted to Transformers. It is the lightest member of a planned dots3 family, alongside jazz and aria, that will span different points on the complexity-versus-cost curve.

The model is a mixture-of-experts design with 280B total and 16B active parameters, 256 routed experts plus a shared one, and native multimodal input across text, images, video, and audio. The headline number is context: 512K tokens, long enough to hold days of accumulated agent state.

The release also ships two evaluation environments aimed squarely at what most benchmarks ignore. VibeSearchBench tests an agent's ability to iteratively uncover ambiguous user intent through conversational search. VibeLifeBench measures week-long task execution, where flight availability and weather shift mid-run. On Terminal-Bench 2.1, dots3-note posts a 75.1, roughly five points above the best US open-weight model on the same chart, according to figures shared by SemiAnalysis and reported by PANews.

How TEMPO works

The genuinely new idea is TEMPO, short for Test-time-scaled Value Estimation with Macro-step Policy Optimization. Long-horizon agent tasks break classic reinforcement learning in two ways. First, a single rollout can take ten or more hours, making training prohibitively slow. Second, a sparse reward signal makes it nearly impossible to assign credit to any individual action in the chain.

TEMPO's fix is to let the same model act as its own critic. It decomposes a long task into macro-steps, each containing several rounds of interaction with the environment. At the end of each macro-step, the agent switches roles, from actor to critic, and uses test-time-scaled reasoning to estimate the expected remaining return from its current state. That produces a learning signal before the task ever finishes, so the policy can be updated mid-episode.

The results are striking. On ARC-AGI-3, TEMPO-trained models score 31.5% above a baseline checkpoint and 20.6% above GRPO, a widely used value-free method. Dots Lab also reports the finding that makes this approach tractable: evaluation is easier than generation. Even when the model cannot yet solve a problem, it can reliably tell a genuine breakthrough from a superficially similar dead end. That asymmetry is what makes a self-critiquing agent trainable at all.

Why it matters

This is the strongest evidence yet that long-horizon agency is a training problem rather than a scaling problem. Xiaohongshu is a social-commerce company, not a frontier lab with a nine-figure data-center budget, yet its model sits near the top of agent benchmarks alongside systems several times its size.

There is a broader context here. The field is shifting its definition of what a capable agent even is, from "answer the question correctly" to "operate autonomously over time." ARC Prize's ARC-AGI-3, introduced this year, is the clearest signal: it drops static puzzles in favor of unfamiliar virtual environments where agents must perform thousands of interactions across 40 to 50 hours. Dots Lab explicitly built dots3-note against that benchmark, training on thousands of novel environments to teach the model to learn online and update its memory as the world changes. That is a meaningfully different target than the next coding eval.

The same self-evaluation capability shows up at inference. At IMO 2026 last month, a harness built around a branch of dots3-note earned a certified perfect score of 42/42 and a gold medal by generating proofs recursively and using tools to critique and improve them. The lesson transfers directly: spend inference compute on self-checking, not just on generation.

The honest caveat is benchmark over-optimization. SemiAnalysis has said its team is still testing the model to assess real quality and whether the scores reflect genuine capability or tuning to the test. Dots Lab itself says the full technical report, with architecture, training, and evaluation detail, lands within a week. Until then, the numbers are promising but provisional.

What builders should do

For builders, this is a blueprint as much as a download. The Apache 2.0 weights mean you can self-host a multimodal agent model and inspect the harness. The TEMPO recipe is described clearly enough to reimplement. And VibeSearchBench and VibeLifeBench give you a way to test whether your own agents genuinely handle ambiguity and long time horizons, rather than just passing a static eval.

Three lessons stand out. First, long-horizon agents need value estimation before the task ends; if your agent cannot self-assess mid-task, it cannot be trained efficiently. Second, evaluation is a cheaper capability than generation, so lean on self-critique to steer long rollouts. Third, memory that outlives the context window is learnable: when a task runs longer than the context, RL teaches the model to write memories that support its future decisions.

What's next

The dots3 family rolls out jazz and aria over the coming weeks, and the technical report should settle whether the benchmark gains are real or partially tuned. The bigger question is whether TEMPO generalizes beyond Dots Lab's own environments, and whether open-weight labs answer with their own self-evaluating training recipes, the way they did after DeepSeek open-sourced its coding harness.

Long-horizon agency is now an open, competitive problem. The race to build agents that can think across days, not minutes, just got a concrete, reproducible starting point.