Stop When Reasoning Converges, Save 26% of Tokens

Featured

01 Watch Reasoning Convergence, Not Answer Stability

Existing early-exit for reasoning models reads the answer side — a confidence threshold or several consistent trial answers in a row. PUMA argues this signal only says "a candidate answer is ready," not "reasoning has actually converged." A trial answer can stabilize mid-chain, then get overturned by later self-correction. Cut too early on an answer-level rule and accuracy drops along with the semantic completeness of the truncated chain.

PUMA swaps the signal. A lightweight Redundancy Detector watches whether the next steps still produce new progress. If a few steps just restate the existing conclusion, the trace counts as converged, and answer-level verification catches the safe stop. Across 5 LRMs and 5 reasoning benchmarks, the average token saving is 26.2%. Accuracy and the surviving CoT quality hold. The same trend reproduces on code generation and VLM tasks.

The Redundancy Detector has its own compute cost. A single trace may not show ROI. The value shows up in batched serving, where long-tail traces accumulate the savings. Profile detector overhead against token savings before you ship.

Key takeaways: - Answer-level signals fire when a trial answer stabilizes, but reasoning may still self-correct once. An early stop bets on no reversal. - Two-layer judgment (semantic redundancy plus answer verification) is plug-and-play. Not tied to a specific LRM, and transfers to code and vision-language tasks. - Savings hide in per-trace numbers. The real payoff is the long-tail aggregate in batched serving. Profile detector cost against savings.

Source: Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models

02 The Bottleneck in Video LLMs Isn't the LLM Anymore

Compress visual tokens past a point and the next bottleneck isn't the LLM. It's the vision encoder that nobody was watching. LiteFrame's real contribution isn't the method itself — distillation teaches a small encoder the larger model's spatiotemporal compression. The contribution is the inversion. Two years of FastV, VisionThink, and other post-hoc token reduction work assumed "LLM inference is the bottleneck." Once tokens get thin enough, per-frame encoder time takes over.

In 2026, the optimization target for video LLMs has moved from the LLM side to the encoder side. Picking today's solution off the 2024 latency profile will get the math wrong.

Key takeaways: - Video LLM latency bottleneck has moved from the LLM to the vision encoder. 2024 optimization intuitions are already stale. - Before evaluating long-video methods, measure where time actually goes in each component. Then decide which end to optimize. - The next round of efficient video LLM work is moving to the encoder side.

Source: LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs

03 Scaling Law Estimation Without the Full Sweep

Choosing model size, architecture, and hyperparameters at scale starts with a grid of small training runs across (parameters × data × hyperparameters) to estimate the scaling law. Mid-size teams can't pay for the full sweep. Many model selection decisions end up driven by intuition.

This ICML work pairs Successive Halving with a surrogate model. During training, it predicts each configuration's learning curve and kills the bad ones early. The shift is from "scan the grid" to "ask in real time which configuration is worth continuing." Reported improvements over uniform allocation: 2.84% on real datasets, 5.47% on synthetic. Up to 98.7% compute savings versus running the full grid to estimate the same scaling law.

The 98.7% number compares against "run everything to completion," which most teams wouldn't actually do. A more honest comparison is against your current early-stopping recipe, and the abstract doesn't give that one. Even at a steep discount, this kind of active allocation could move "estimate the scaling law first, decide second" out of frontier-lab territory and into mid-size team budgets.

Key takeaways: - The total compute bill for scaling law estimation is a real reason small teams skip the proper model selection workflow. - 98.7% is an upper bound against a full-grid sweep. Compare against your current early-stopping before deciding. - If active-allocation tooling matures, "estimate then decide" moves down from frontier labs into mid-size team range.

Source: Active Budget Allocation for Efficient Scaling Law Estimation via Surrogate-Guided Pruning