Repositories Become Skills as Models Manage Their Own Context

Featured

01 Agent | Repositories Become Reusable Skills

Research agents usually inspect repositories only after a task begins. They must quickly discover configurations, execution order, and common failure modes. DisCo turns that scattered operational knowledge into reusable, verifiable skills.

It can compress frequently used repositories in batches or build skills for a specific task. Its AREX-Skill Library contains over 5,000 skills from 1,000 machine learning repositories. The collection spans 20 domains and 178 capability categories.

With GPT-5.5, research scaffolding, and execution budgets held constant, these skills raised MLE-bench scores by 134.3%. PaperBench scores improved by 34.4%. Better reproduction may not require a stronger model or more trial and error.

The abstract only shows that the skill layer works under a fixed setup. It does not establish how well skills transfer across datasets, frameworks, or tasks. Teams should measure whether one successful reproduction becomes a dependable capability for the next project.

Key takeaways:

• Distilling repository experience into skills can reduce repeated work on environments, parameters, and execution flows.
• Large gains under fixed model and execution budgets suggest reusable skills can become a separate source of agent capability.
• Check transfer reliability across repositories, frameworks, and new tasks before adopting this approach.

Source: Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills

02 Efficiency | Models Choose Where to Look

Million-token contexts are wasteful when a model needs only a small part of the history. Yet each generated token may still scan the entire KV cache.

Declarative Attention lets the model act as its own retrieval controller. During inference, it declares whether to scan globally, focus on a region, or read only recent output. The inference engine then skips irrelevant content.

In zero-shot tests, two existing models reduced total attended tokens by 52.0% and 31.1%. Accuracy dropped by only 1.27 and 2.75 percentage points, with smaller losses for larger models.

This design moves long-context control from an external scorer into the model. Choosing the wrong region can still hide essential information. Serving teams should test latency, throughput, and cache bandwidth under real request lengths and concurrency.

Key takeaways:

• Long-context models can control their own retrieval instead of depending on external step-by-step scanning.
• Attention reads fall sharply, but region-selection errors must be included in quality evaluations.
• Test real traffic, latency, throughput, and training costs before trusting theoretical savings.

Source: Language Models Can Control Their Own Attention

03 Evaluation | EarlyEval Predicts Outcomes Mid-Trajectory

An agent trajectory often reveals its likely outcome before the run ends. EarlyEval treats evaluation as an early-stopping problem with calibrated confidence.

Two lightweight LightGBM classifiers predict success and failure from behavioral, textual, and reference-solution features. The system stops a run once confidence crosses a calibrated threshold.

Across three benchmarks, EarlyEval reduced execution steps by 13%–26%. It saved up to 44.1% of input tokens and 29.4% of output tokens. Prediction accuracy stayed between 89% and 97%, while agent solve rates shifted by only one to two percentage points on average.

Those averages can hide damaging errors. A premature failure prediction erases late recoveries and may underrate models that excel at long-horizon correction. Calibrate thresholds by task type and retain full trajectories as controls.

Key takeaways:

• Early stopping can complement smaller evaluation sets and further reduce agent evaluation costs.
• Threshold selection should focus on false failure predictions and late recoveries, not only overall accuracy.
• Keep full-trajectory controls to confirm that small solve-rate shifts do not change model rankings.

Source: EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction

04 Safety | Pruning Can Shift Persona Bias

Pruning appears to remove weights, but it can also amplify existing model biases. The same question may receive different answers when a prompt includes different persona cues.

Debias-SparseGPT adds debiasing directly to the pruning process. Paired demographic-group inputs constrain internal representations, limiting the amplification of group differences during compression.

Across several generative LLM families and 25%, 50%, and 2:4 structured sparsity settings, it consistently reduced pruning-induced bias versus SparseGPT. Perplexity and zero-shot accuracy stayed intact.

Even under 2:4 sparsity, which damaged model quality the most, longer and richer calibration samples improved both task performance and fairness. Compression checks should therefore treat bias changes as a release criterion, not a later repair.

Key takeaways:

• Pruning is not value-neutral. Removing weights can redistribute how persona cues affect outputs.
• Compression evaluations should measure performance, sparsity, and fairness together.
• Longer, more informative calibration data deserves extra investment for structured sparsity.

Source: Debias-SparseGPT: Bias-Aware Pruning for Large Language Models