Neural Functions Hit 83.6%, Random KV Eviction Boosts Throughput

Featured

01 Training: Compile Specifications Into Local Functions

Natural-language specifications no longer need a large model on every run. Compile by Training turns them into local neural functions that teams can store, version, and compose like software.

A teacher model first synthesizes task data from the specification. A compact interpreter then trains a small adapter on that data. Once compilation finishes, the teacher exits the loop.

On FuzzyBench-Hard cases where fast compilers cannot find exact matches, the method reaches 83.6% semantic accuracy. Compilation takes about one minute instead of seconds, but this one-time cost removes the teacher from later execution. Demos include website assistants, language-controlled 3D avatars, and bidirectional translators. The goal is not wholesale replacement of large models. It is extracting narrow, well-defined capabilities from remote services.

Key takeaways:

• Prioritize stable, frequent tasks that resist rule-based implementation.
• Trade about one minute of training for reusable, versioned local execution.
• Ask which capabilities can safely leave remote models, not whether small models can replace them.

Source: Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

02 Efficiency: Complex Selectors May Be Winning the Wrong Race

KV cache eviction methods compete over which tokens matter most. Random Attention finds that protecting the prompt and randomly evicting reasoning tokens within each attention head can match leading selectors.

Across four models and six reasoning tasks, it avoids importance scoring entirely. In vLLM deployments, the method raises throughput by 32% to 43%. The authors argue that reasoning traces resist eviction by repeating key information and retaining copies across attention heads. The original prompt is more fragile.

Many complex methods may owe their gains to prompt protection and per-head budgets, not better selection signals. The full ablations still need to confirm that interpretation. Deployment teams should separate selection quality from cache allocation before paying for added complexity.

Key takeaways:

• Ablate prompt protection separately from importance scoring.
• Text repetition and multi-head redundancy may limit the value of precise selection.
• Compare reasoning quality and scoring overhead. Always include a random baseline.

Source: Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

03 Image Gen: An Open Recipe Isn't Full Reproducibility

LLaDA-Image first builds generation priors through image-only pretraining and mid-training. It then combines the generator with a frozen vision-language understanding module based on LLaDA2.0-Mini.

This staged design separates image generation from language understanding early in training. Teams can adapt existing generators without depending on extensive paired data from the start.

The final 6B model scores 53.53 and 53.38 on Qwen-Image-Bench’s English and Chinese tracks. The abstract claims a new high among open models. Distilled variants reduce inference to two to four steps. The team released weights, code, and detailed recipes, but the pipeline uses 220 million samples. An open recipe does not make its data or results fully reproducible. Other teams now need to test the approach across different data and compute budgets.

Key takeaways:

• Build image priors before adding language understanding and instruction alignment.
• Staged training recipes can guide model adaptation better than a single benchmark score.
• Evaluate open projects by checking weights, code, training details, and data availability separately.

Source: LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes

04 Evaluation: Turn Translation Failures Into Test Cases

Translation evaluations cannot guide repairs unless they express failures as repeatable test conditions. LTB collects peer-reviewed multimodal examples designed to trip up leading models. Each example includes a manually verified rule describing a specific reproducible failure.

The evaluation unit shifts from overall translation quality to whether a capability fails under defined conditions. That leaves less room to optimize against automated metrics. It also makes error attribution and regression testing easier.

LTB continuously accepts contributions, and LTBv1 covers cases accepted through September 1, 2026. Rule consistency, representative coverage, and cross-version comparability still require confirmation through the full paper and future releases. Translation teams can use its model to turn production failures into reviewable test cases that guide data and model updates.

Key takeaways:

• Replace average scores with capability tests when aggregate metrics cannot guide iteration.
• Put reproducible failure rules into regression tests and quality pipelines.
• Live benchmarks absorb real failures, but progress claims must control for version and sample bias.

Source: Last Translation Benchmark