Teacher Policy Hits 93.57; 3,000 Samples Recover 37.3%

Featured

01 Robotics: Human Logs Also Limit What Cars Learn

Imitation learning can reproduce driving from logs, but it rarely explores behaviors that human drivers never demonstrated. DriveZero does not force perception and action into the same training loop. Its perception model learns from large, diverse visual datasets. The action model trains through closed-loop reinforcement learning in interactive environments converted from real logs.

The perception stack combines frozen vision foundation models, including DINOv3, SigLIP2, and SAM, into one backbone. This requires no task-specific labels. For control, PPO trains a teacher policy with access to additional environment information. Mistakes therefore affect what happens next.

The final camera-only planner learns from behavior rolled out by the teacher rather than directly imitating human trajectories. Changing the driving intent produces goal-consistent supervision absent from the logs. With value-guided test-time action search, the teacher averaged 93.57 across several nuPlan community splits. It beat the log-replay expert on every split. The final system also led on NAVSIMv1, NAVSIMv2, and HUGSIM, according to the abstract. These results show that training signals can move beyond log coverage. They do not prove comprehensive superhuman driving on real roads. Broader generalization and safety limits still need the full paper and more on-road validation.

Key takeaways:

• Treat perception pretraining and closed-loop action learning as separate problems, with different data and training methods.
• An interactive teacher can generate goal-consistent behavior outside the logs, expanding demonstration coverage.
• Beating log-replay experts does not mean beating human drivers everywhere. Keep testing real-road generalization and safety.

Source: DriveZero: End-to-End Driving Beyond Human Demonstrations

02 Agent: Correct Tools Can Still Fail

In multi-turn tool use, the easiest failures to miss live inside the arguments. A model may select the correct tool while supplying fabricated, stale, or unsupported values. SAP combines state guidance, provenance constraints, and turn-by-turn validation to build accurate traces with long dependencies. The resulting data teaches models to assemble executable calls from current state, not merely match requests to tool names.

According to the abstract, SAP-4B trained on this data competes with larger models across several benchmarks. Its exact advantage and tolerance for real-world noise still need confirmation from the full paper. Teams training agents should make argument traceability and cross-turn consistency core synthetic-data metrics.

Key takeaways:

• Correct tool selection does not guarantee an executable call. Bad arguments can still sink the task.
• Synthetic data should explicitly cover argument sources, updates, and dependencies across turns.
• Evaluate turn-by-turn state consistency, not just tool-name accuracy.

Source: SAP: State-Guided Data Synthesis with Argument Provenance for Multi-Turn Tool Use

03 Training: “Training-Free” Compression Is Only the Start

A checkpoint produced by MoE compression may be an initialization that needs repair, not a finished artifact. The study compares several adjustment methods under equal GPU budgets. One epoch on just 3,000 C4 samples let full-parameter tuning recover an average 37.3% of the performance lost to compression.

Against intuition, standard language-model training was more cost-effective than token-level knowledge distillation from a teacher model. Full-parameter tuning also outperformed narrower updates. The finding spans two MoE backbones, four compression methods, and 28 benchmarks. Generalization to other models and deployment settings still needs testing.

Deployment teams should optimize total performance per unit of GPU cost, not zero training cost. Count both compression and the small repair stage that follows it.

Key takeaways:

• Treat compressed checkpoints as repairable initializations rather than finished artifacts.
• One epoch on 3,000 samples recovered an average 37.3% of the performance lost to compression.
• Compare compression methods using the combined GPU cost of compression and subsequent adjustment.

Source: Beyond Retraining-Free MoE Compression: A Cost-Normalized Study of Post-Compression Adjustment

04 Evaluation: Full Context, Yet Discourse Stays Invisible

Giving annotators an entire document should expose cross-sentence inconsistencies. This study found that showing context does not mean evaluators use it. Researchers built counterfactual hybrid documents by combining passages from different translation systems, deliberately breaking discourse coherence.

Across 18,420 expert English-to-Korean annotations and 14 automatic metrics, hybrid and coherent documents received statistically equivalent scores, rankings, and error labels. Reviewers could still detect the difference. When shown paired texts, they identified the coherent version from one translator 87.3% of the time.

The document-level interface changed how annotators worked, but those differences never entered the recorded judgments. Translation teams should test evaluation protocols with controls that deliberately damage discourse coherence. Validate the metric before investing in document-level systems and annotation.

Key takeaways:

• Showing the full document does not guarantee that judgments capture document-level quality.
• Counterfactual hybrid documents can test whether metrics detect cross-sentence consistency.
• Confirm that an evaluation protocol measures the target capability before funding document-level systems and annotation.

Source: The Blindness of Document-Level Translation Evaluation