A 4B Agent Ties the Giants by Verifying, Not Searching Harder

Featured

01 Verification Is Cheaper Than Discovery

Deep research looks for an answer that satisfies many constraints at once. Finding that answer is expensive; verifying a candidate is cheap, splitting into a checklist you can walk one constraint at a time. AREX exploits the asymmetry. Instead of searching longer, it keeps the partly-verified intermediate state and uses it to guide the next round toward only the constraints that haven't held.

It runs two nested loops. The inner one gathers evidence and drafts a tentative answer; the outer one audits that draft claim by claim, flags the unverified ones, and launches targeted follow-up research. To sustain long task chains, AREX learns a self-compression tool that squeezes the growing interaction log into a state holding only verified evidence and open constraints — no external summarizer.

Two models get this treatment: a 4B dense model and a 122B-parameter MoE (mixture-of-experts) with about 10B active. On deep-retrieval and reasoning benchmarks like BrowseComp and HLE, both beat same-class baselines and stay competitive with models that activate more parameters. What's worth keeping isn't the phrase "recursively self-improving" — the part most likely to be overrated. It's where the lever sits: on the structural gap where verification costs less than discovery, not on a stronger base model.

Key takeaways: - For any task where verification breaks into cheap per-item checks while discovery stays expensive (deep retrieval, multi-constraint solving), search once and then patch the constraints that failed. That beats searching longer. - To keep a long-chain agent from blowing its context, compress history into a state that holds only verified evidence and open constraints. The self-compression trick is worth borrowing on its own. - Don't get pulled in by "recursively self-improving." The lever is the structural gap where verifying costs less than finding, not a stronger base model, and the 4B model tying larger ones is the proof.

Source: AREX: Towards a Recursively Self-Improving Agent for Deep Research

02 Force Spatial Answers Into Words, Test the Wrong Thing

Almost every spatial-reasoning benchmark forces the answer into coordinates, multiple choice, or a paragraph of text. But spatial cognition is about pointing a direction, circling a region, tracing a path. Serialize that into text and you may be measuring the model's text-translation ability, not its spatial ability.

ProVisE changes the interface. It lets an image-generation model answer directly in the pixels it draws, then parses that drawn answer into a structured prediction aligned with the original metrics. The surprise: when spatial answers can be drawn this way, image generators pull even with text-output VLMs (vision-language models). Only on tasks that need compositional spatial reasoning do text models take a clear lead.

For anyone building evals, the takeaway is blunt: the interface you ask through largely decides what you actually measure.

Key takeaways: - The eval interface isn't neutral. Force spatial answers into text and you may be scoring translation, not spatial skill. - When the answer can be drawn directly, image generators match text VLMs. Compositional spatial reasoning stays the text models' turf. - For spatial or embodied evals, confirm the answer interface matches the ability you mean to test before trusting the scores.

Source: Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text

03 How Much of a Coding Leaderboard Is Memorized?

Most coding-agent benchmarks lightly edit public issue text — text the model has likely already seen in training. So leaderboard scores mix in memorization, and you can't tell real skill from recall.

Tencent's WorkBuddy Bench differs entirely in how tasks are built. Each one is reverse-engineered from a real commit, PR, or business scenario, then rewritten as a colloquial, role-play request so you can't search the original issue to recover the answer. It spans four domains: Code, Web, Office, and Security.

Its contamination resistance doesn't rely on secrecy. Tasks, environment images, tests, and reference solutions are all public; the defense is the construction method plus version control, so third parties can rerun and audit it. It even refuses to publish one overall average, since the four subsets score on different standards — restraint that reads as more credible than a clean composite. Next time you read a coding-agent leaderboard, ask first whether the tasks already leaked into training.

Key takeaways: - Read any coding-agent leaderboard with one question first: could these tasks already sit in the training set, and how much of the score is recall? - Contamination resistance doesn't need secrecy. Reverse-engineering real commits, full public release, and version control make a more auditable path. - Because the four subsets score on different standards, WorkBuddy refuses a single average — a reminder not to trust cross-domain "composite" numbers.

Source: Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

04 Mostly Linear Attention, With a Softmax Anchor

Linear attention is tempting for long-video generation: the longer the sequence, the more compute it saves. But it lacks softmax's global token interaction, so it always comes up a little short on expressiveness.

SANA-Video 2.0 doesn't go pure linear. It pairs three parts gated linear attention with one part softmax "anchor," using that 25% softmax to restore global dependencies. The paper's low-resolution proxy experiments call this ratio the best balance of quality and efficiency. This hybrid trains from scratch rather than bending a pretrained model into linear form. On a single H100 it reaches 720p, running inference 3.2x faster than a same-size full-softmax baseline — and the gap widens as the video runs longer.

The line to watch is "matches full-softmax quality." A close aggregate like VBench 84.30 doesn't mean the real gaps in motion coherence and long-range consistency are gone, and that's exactly where efficiency papers tend to discount. The point isn't the single GPU. It's whether the "mostly linear, a little softmax" trade-off holds up in real testing.

Key takeaways: - Hybrid linear-softmax attention (about 3:1, a quarter softmax) is a practical way around pure linear's expressiveness gap. Watch this line if you want long-sequence compute savings without losing quality. - Don't take "matches full-softmax quality" from a VBench aggregate alone. Motion coherence and long-range consistency are where efficiency papers discount most. - The efficiency edge grows with video length, so long clips pay off more than short ones. But it trains from scratch rather than adapting an existing model, so count the reuse cost in.

Source: SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation