Today's Overview
- Constrained Harness Editing Delivers Limited but Transferable Gains. RRSI gained up to 14.1 points on its evolution split and 4.7 points across five out-of-distribution benchmarks. It also consumed 30% fewer policy tokens.
- WorldCrafter Compresses History for the Target Camera View. It turns multiview observations into a fixed number of view-specific tokens without explicit depth-based correspondence. The paper reports better long-term consistency and camera control during minute-long exploration.
- Token-Level Correction May Cut Annotation Costs While Preserving the Model Distribution. onPanda keeps most of the model’s own output. A small controlled study found a 52% reduction in median annotation time versus human post-editing.
Featured
01 A High-Scoring Harness May Just Know the Benchmark
Strong in-distribution results can hide a harness that has written the training tasks into the system. RRSI constrains automated iteration instead of generating ever more candidate edits. A time-varying budget limits how many edits can ship together, while the evolution history rewards unexplored paths.
At selection time, a critic rejects proposals tailored too closely to a specific benchmark. A pruner removes edits with small gains, excessive costs, or mechanisms that no longer work. The surviving changes should be more reusable.
Results split sharply by evaluation setting. Scores rose by up to 14.1 points on the split used for evolution, but only 4.7 points across five out-of-distribution benchmarks. The regularized harness also consumed 30% fewer policy tokens than unconstrained evolution. Teams automating agent prompts, tools, and control flow should copy the constraints, not just the recursive rewriting.
Key takeaways:
- Report in-distribution and out-of-distribution scores separately when evaluating automated harness optimization.
- Limit per-round edits and track exploration history to reduce benchmark-specific accumulation.
- Base deployment decisions on transfer gains and policy-token costs, not the highest score increase alone.
Source: RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
02 Ask the Camera, Then Compress the Memory
Keeping more history does not always produce better memory. The next camera view determines which parts of that history matter.
WorldCrafter conditions history compression on the target camera. It turns multiview observations into a fixed number of view-specific tokens for the video generator’s denoising process. A memory encoder and view-conditioned reader jointly learn what to retrieve, without explicit depth-based correspondence.
Combined with recent temporal context and few-step distillation, the system can explore scenes from one image or a text prompt. The abstract reports better long-term consistency and camera control across static scenes and scenes with motion. It also maintains visual quality during minute-long exploration. Broader generalization and the size of the advantage still need full-paper evidence and testing across more scenes.
Key takeaways:
- Long-horizon world models can compress history around the target viewpoint instead of preserving it unchanged.
- A fixed number of view-specific tokens provides a clear interface for limited context budgets.
- Minute-long exploration looks promising, but the abstract does not establish performance across all environments with motion.
Source: WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
03 Fix the First Bad Token, Not the Whole Answer
Rewriting an entire model response is slow. It can also turn alignment data into samples of human writing.
onPanda asks annotators to find the first unsuitable token and correct it. The model then continues from the revised prefix. This cycle repeats until the result is usable.
Most of the final text still comes from model sampling, so the data stays closer to the model’s generation distribution. That makes it suitable for on-policy SFT, preference data, and agent interaction traces. Each correction also records the wrong token, its replacement, and the exact location.
A small controlled study found a 52% reduction in median annotation time versus human post-editing. Treat that as an early efficiency signal, not a universal estimate.
Key takeaways:
- Token-level correction may reduce human effort while preserving most of the model’s generation distribution.
- Correction logs naturally provide positive and negative supervision at exact token positions.
- The study found 52% lower median annotation time than post-editing, but larger studies must test whether that result generalizes.
