Adaptive Decoding Hits 4.2x, Joint Training 10x Faster

Featured

01 Efficiency Fixed Block Size Wastes Speedup

Speculative decoding — a small model drafts, a large model verifies, lossless — moved into the diffusion regime and pushed parallelism to SOTA by generating several tokens per block. These methods share one assumption: every input uses the same fixed block size and one shared decoding policy. BlockPilot argues that assumption is suboptimal. Different instances want different amounts of parallelism, so the optimal block size varies per sample. A fixed value splits some requests too aggressively and starves the parallelism of others.

The fix is light. After prefilling, BlockPilot uses the representation at that moment to predict the right block size for this input, then decodes normally without further intervention. The key observation is that these optimal values cluster near the training-time block size, so the decision space collapses to a low-dimensional problem and policy learning stays cheap.

On Qwen3-4B at temperature T=1, it hits an acceptance length of 5.92 and 4.20x speedup. The T=1 setting matters, because high-temperature sampling is usually the worst case for speculative decoding. Holding up here means the method is not sensitive to sampling randomness.

Key takeaways: - The fixed decoding hyperparameters in your pipeline may be leaving speedup on the table, and instance-adaptive prediction is a low-cost fix. - This is a plug-and-play policy-layer change. It leaves the draft model and verification logic untouched, so integration cost is low. - The cost of the prediction itself, and whether lossless speedup holds under real mixed workloads, needs production data to confirm.

Source: BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding

02 Safety Make Metacognition a Training Target

The standard defense against hallucination wraps the model in an extra layer: confidence thresholds, retrieval checks, second-pass review. Yale takes a different entry point. One root cause of LLM unreliability is a metacognitive deficit — confidently wrong answers, blurry knowledge boundaries, and a gap between internal uncertainty and what the model actually says.

Their RLMF (reinforcement learning with metacognitive feedback) does not calibrate that confidence number. It treats the quality of a model's self-judgment about its own performance as the RL signal, training the model so that a stated "I'm unsure" corresponds to its internal state. The paper reports up to a 63% gain on faithful calibration over standard RL without sacrificing accuracy. That 63% is a relative gain, so the baseline and task distribution matter before you weigh it.

For teams that care about trustworthiness, the shift in framing is the point. Uncertainty expression moves from a bolt-on filter to a capability you can optimize end-to-end.

Key takeaways: - A new path for hallucination control: make metacognition a trainable target rather than adding confidence filtering at the output. - RLMF uses the quality of the model's self-judgment as the RL signal, getting past the ceiling of earlier internal-feedback methods. - The 63% is relative to standard RL. Check the full paper's baseline and task range before judging real-world value.

Source: Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs

03 Image Gen The Tokenizer and Generator Stop Training Apart

Mainstream visual generation runs in two stages: train a tokenizer for reconstruction, freeze it, then train an AR generator on its discrete codes. The problem is that the tokenizer has no idea which codes the generator finds easy to predict, so the two are misaligned by construction. GEAR trains the VQ tokenizer and AR generator end-to-end, guided by representation alignment.

The hard part is that VQ's discrete indices are non-differentiable, and straight-through gradients collapse. GEAR uses a dual-readout design. A hard one-hot branch trains the AR with next-token as usual. A differentiable soft branch passes only the alignment loss back to the tokenizer, letting the AR pull the tokenizer toward a code distribution it predicts better. The alignment burden shifts from tokenizer to AR: the tokenizer's own features become less DINOv2-like while the AR's features grow more semantic — the opposite of the diffusion camp's recipe of making the latent itself semantic.

Against a LlamaGen-REPA baseline, ImageNet gFID converges up to 10x faster, and the method generalizes to quantizers like LFQ and IBQ and to text-to-image.

Key takeaways: - On the "tokenizer-generator mismatch" problem, the pixel-AR camp drops the tokenizer while GEAR makes the two learn jointly. If you build image generation, read the two paths side by side. - Dual-readout is the key engineering trick for the non-differentiable VQ. The soft branch only feeds the tokenizer and never contaminates the AR's next-token training. - 10x is convergence speed, not final quality. Final FID needs the full paper, but generalizing across quantizers and to text-to-image is a plus.

Source: GEAR: Guided End-to-End AutoRegression for Image Synthesis

04 Evaluation Check Your Intermediate Rewards Before Training

A long-horizon agent trajectory runs hundreds to thousands of actions, and rewarding only the final outcome is too sparse. So dense supervision — scoring intermediate steps — keeps appearing, from model confidence to self-distillation to embedding similarity. Validating any of these usually means wiring it into a full training pipeline and measuring downstream results. That is expensive, and it tangles the signal's own quality with training engineering, so different approaches can't be compared fairly.

QVal skips training entirely. Given a state-action pair, it checks how well a scoring method agrees with a strong reference policy's Q-value ranking, so you can judge whether a dense signal is trustworthy before any run starts. The authors ran a head-to-head over 21 methods, 7 families, and 6 open models. After more than 1,200 experiments, the conclusion is unflattering: a simple prompting baseline reliably beats the fancy dense-supervision methods from recent literature, and scores cluster strongly by family.

If you design intermediate rewards for agents, this is a low-cost selection tool. It measures whether a signal aligns with Q-value ranking, and whether that equals real training value still needs the full paper's validation.

Key takeaways: - Evaluating dense supervision doesn't require a full downstream run. Q-value agreement can screen out unreliable signals early. - A simple prompting baseline beats most recent methods. Don't assume the complex method is stronger before you select. - This is a training-free open testbed. Anyone designing agent rewards can plug in their own environment and try it.

Source: QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents