Do Agent Skills Help? What Peer-Reviewed Research Shows

Quick answer: Curated agent skills improved task success in the studied configurations. The evidence also includes regressions, a benchmark with no established retrieval benefit, and successful attacks through malicious skill files. These are conditional results, rather than a guarantee that installing a skill improves an agent. SkillsBench conference record, SkillFlow, Skill-Inject conference record.

Our review of CLAUDE.md and AGENTS.md examined repository instruction files. This companion article asks a different question: what happened when researchers supplied agents with task-specific skill bundles?

It covers three papers with verified acceptance at peer-reviewed conferences. Their experiments measure different outcomes, so the results below keep the benchmark, model and condition attached to each number.

Publication status and the versions behind the numbers

SkillsBench and Skill-Inject appear in the official NeurIPS 2026 accepted-paper list. SkillsBench's author record identifies the Evaluations and Datasets Track. SkillFlow appears in the official COLM 2026 accepted-paper program.

The SkillsBench and Skill-Inject sections use their official conference abstracts. SkillFlow's results use its linked v3 author manuscript, headed as a COLM 2026 conference paper. Its exact final accepted PDF and individual reviewer reports were inaccessible during this review; the version link identifies the manuscript examined.

SkillsBench's conference abstract and its expanded author manuscript report different benchmark snapshots. This article uses the conference snapshot consistently, rather than presenting expanded manuscript results as independently verified reviewed findings.

SkillsBench: curated bundles improved task success

The accepted conference record describes 84 tasks across eight domains, seven agent–model configurations, and 7,326 trajectories. It compares three conditions: no skills, curated skills and self-generated skills, using deterministic verifiers.

The reported average pass-rate improvement from curated skills is 16.4 percentage points. Domain-level gains range from 7.2 to 24.8 percentage points. Yet 18 of the 84 tasks regress. The aggregate benefit is therefore compatible with worse outcomes on particular tasks. SkillsBench, official conference abstract.

Self-generated skills show a net difference of −1.3 percentage points in the same abstract. That measures the tested generation method; it does not prove that every automated skill-generation approach fails.

These are results for skill packages in the evaluated tasks. They do not isolate the effect of Markdown prose, establish a universal coding-success gain, or quantify savings for a developer's own workflow.

SkillFlow: retrieval helped on one benchmark, but did not establish a gain on another

SkillFlow retrieves skills from approximately 35,000 community bundles. Its downstream evaluation retained 65 SkillsBench tasks and 88 Terminal-Bench tasks after exclusions.

On SkillsBench, Claude Code with Haiku 4.5 improved from 6.7% to 17.4%: +10.7 percentage points, with adjusted p=.0002. Codex CLI with GPT-5-mini improved from 9.2% to 16.4%: +7.2 percentage points, with adjusted p=.0364. SkillFlow v3, Table 1.

On Terminal-Bench, GPT-5-mini changed from 34.8% to 31.8%. That difference was not statistically significant, so this experiment establishes neither a benefit nor a harmful effect. Claude was tested on SkillsBench only.

The retrieval library includes SkillsBench's task-matched oracle bundles. That matters when interpreting the positive result. SkillFlow also shares the SkillsBench task suite with the preceding study; these are not two independent demonstrations of transfer to a new domain. SkillFlow v3, §4.

Skill-Inject: skill files can influence agents in unwanted ways

The accepted conference record describes 202 injection–task pairs, including both obvious attacks and subtle instructions embedded in otherwise legitimate skill files.

It reports attack success reaching 80% in the tested frontier-model configurations. The attacks include unwanted data transfer and destructive actions. This is the upper result reported across tested configurations, not a universal rate for Claude or every agent. Skill-Inject, official conference abstract.

These are results from deliberately malicious skill files under experimental conditions. They are not estimates of how often ordinary public skills cause incidents, or measurements for every later model version. The study evaluates a security outcome; it does not measure the ordinary productivity benefit of installing a skill.

What this establishes about skills and CLAUDE.md

The experiments support a bounded finding: particular curated bundles and a particular retrieval system improved outcomes in tested settings. They also expose conditions where skills failed to help and a security surface introduced by skill files.

None of these studies compares a skill with a minimal CLAUDE.md containing equivalent information. They therefore do not establish that relocating the same instructions into SKILL.md causes an improvement. The observed bundle and retrieval results remain useful without making that stronger claim.

This is a focused review of three accepted papers, not an independent replication or a universal skill-authoring recipe. It does not establish an optimal skill count, a guaranteed saving in tokens, or a return on investment for a particular repository.