DAPD Gains Two Points as Diffusion Hits 1,500 Tokens/s

Featured

01 Training: More Privileged Information Can Hurt

Giving a teacher more information can make the self-distilled model worse. The problem is not just a capability gap. It is “privileged hallucination.”

The teacher produces correct behavior using extra context available only during training. The student never sees that context at inference, yet training treats it as available.

DAPD adds a student-conditioned bridge through dual-path anchoring. It aligns teacher and student only across paths with matching information. Dual-source anchoring adds bidirectional constraints, preserving correctness supervision while reducing dependence on the privileged teacher.

The paper reports a 2.00-point average gain over standard online self-distillation across Qwen3-4B tasks. Gains persist with scale, reaching 2.69 points at 4B and 2.78 points at 32B. Exact settings and costs still require the full paper.

Key takeaways:

• Treat the boundary between training-time and inference-time information as a primary distillation constraint.
• Stronger supervision can create irreproducible behavior when the teacher relies on information hidden from the student.
• Dual anchoring preserves correctness guidance while limiting transfer to information the student can access.

Source: DAPD: Dual-Anchored Policy Distillation

02 Robotics: Seeing the Answer Corrupts Driving Reasoning

Giving a teacher the vehicle’s actual future trajectory changes the task. It no longer decides from the current scene. Instead, it writes a justification for an outcome it already knows.

The paper calls this trajectory anchoring bias. The resulting chain of thought may sound plausible without being causally faithful. Hallucinations become worse in scenes that demand accurate causal distinctions.

Simply hiding the true trajectory does not solve the problem. Open-ended trajectory generation mixes high-level decisions with exact geometry and vehicle physics.

Deferred Exposure turns planning into candidate selection. The model chooses first, then sees the true trajectory as a reinforcement-learning verification target. The paper reports better driving reasoning while preserving or improving general visual ability.

Key takeaways:

• Audit whether future information leaks into the teacher’s context.
• Explaining an observed action does not prove the model can decide independently from the current scene.
• Candidate selection separates high-level planning from low-level trajectory generation, though broader validation is still needed.

Source: Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs

03 Efficiency: DiffusionGemma Converts a Mature Model

Diffusion text generation may matter less for processing 256 tokens in parallel than for its migration path. Mature autoregressive models can move to diffusion decoding without training from scratch.

DiffusionGemma starts from Gemma 4 and uses less than 10% of its original training-token budget. Its two-stage fine-tuning first teaches bidirectional denoising. Reinforcement learning and sampler distillation then balance quality and efficiency.

The model generates about 20 tokens per forward pass on one NVIDIA H100, averaging roughly 1,500 tokens/s. It also outpaces autoregressive models using advanced speculative decoding.

DiffusionGemma retains autoregressive generation with a small performance loss, leaving room for hybrid decoding. It is still an experimental open-weight model. Processing 256 tokens in parallel does not mean a 256-fold end-to-end speedup.

Key takeaways:

• Mature autoregressive models can move to diffusion decoding with far less training than a new model requires.
• Measure end-to-end throughput and quality, not just the number of tokens processed in parallel.
• Teams building generation infrastructure should track hybrid diffusion and autoregressive decoding.

Source: DiffusionGemma Technical Report

04 Image Gen: Deliver Layers, Not Flat Pixels

Flat images lose structure when they enter a design workflow. Moving, deleting, or reusing one element quickly exposes that limitation.

UniWorld-Design treats semantic RGBA layers as the basic unit for generation, understanding, and editing. Each layer is a complete object with transparency, delivered as an independent asset.

The model can generate assets from text or decompose an image into ordered, complete layers. Learning full objects instead of visible pixel fragments should keep other content usable when one layer moves or disappears.

On Crello, it cuts per-layer RGB error by 37% and improves Alpha Soft IoU by 34%. Whether those gains save time in real design workflows still needs validation.

Key takeaways:

• Evaluate editability and output structure alongside image quality.
• Semantic RGBA layers could reduce the cost of modification, reuse, and automated collaboration.
• Benchmark gains are promising, but usability in real design workflows still needs testing.

Source: UniWorld-Design: From Pixel Generation to Layer-Native Design