CLAUDE.md and AGENTS.md: What Peer-Reviewed Studies Actually Found

Quick answer: The reviewed evidence does not establish that adding repository instructions generally improves coding-agent performance. One direct evaluation reports no improvement in task success and more than 20% higher inference cost. Other studies document authoring practices, rather than demonstrate better outcomes. Performance study.

The question behind this article is straightforward: what has actually been measured about CLAUDE.md and AGENTS.md?

A collection of popular instruction files cannot answer that question on its own. Neither can an attractive template. Here, each result is attached to a reviewed publication, with the experiment or observation identified. The distinction matters when deciding what deserves a place in a developer's workflow.

The direct performance result: more cost without a general success gain

Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? compares coding-agent outcomes with repository context files in two settings: SWE-bench tasks with model-generated files, and repository issues with developer-written files.

The authors' publication summary reports no task-success improvement across multiple agents and models, alongside more than 20% higher inference cost. Both kinds of files encouraged more exploration and testing; agents generally followed their instructions. More activity and instruction adherence did not establish better task completion. Study and reported findings.

This was reviewed at MemAgents, a workshop at ICLR 2026, rather than the ICLR main conference. The workshop specifies double-blind review. Its reviewed manuscript and individual reviews were inaccessible during this source check, so this article uses the authors' publication summary, without importing detailed results from later revisions. Workshop review policy.

The authors recommend restricting human-written files to minimal requirements. That is their conclusion, not a tested optimal template or proof that every instruction file hurts performance. Authors' conclusion.

The Claude-specific study describes files, not their effectiveness

On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code, published in the PROFES 2025 proceedings, analyzed 253 files from 242 repositories.

The published abstract describes shallow heading structures and content dominated by operational commands, implementation details and architecture. This tells us what developers were putting into Claude instruction files in that sample. It does not demonstrate that those contents made Claude more accurate, faster or cheaper: this is a corpus study, not a comparison of task outcomes with and without those instructions. Published chapter and abstract.

“Common in repositories” and “shown to improve performance” are different findings.

Configuration problems are common, but their performance effect was not measured

Configuration Smells in AGENTS.md Files: Common Mistakes in Configuring Coding Agents, a SCAM 2026 research-track paper, examined 100 popular open-source repositories containing AGENTS.md or CLAUDE.md.

Using proposed detection heuristics, the researchers reported Lint Leakage in 62%, Context Bloat in 42%, and Skill Leakage in 35% of files. These are frequencies under the study's categories and detectors. They are not percentages of agents failing, or measured improvements available from deleting the flagged material. Conference record and findings.

The study did not test whether removing each detected smell improved task success or cost. It therefore cannot establish an optimal file length or a causal benefit from moving content elsewhere.

Testing instructions are widespread; that does not prove a benefit

Uncovering AGENTS.md: What Testing Guidance Open Source Projects Provide to Coding Agents, also in the SCAM 2026 research track, manually analyzed testing guidance in 300 repositories, covering 626 Markdown files and more than 3,300 coded quotations.

249 of the 300 repositories contained test commands. Other testing expectations appeared less consistently. This documents how projects expose their testing infrastructure; it does not measure whether adding those commands improves an agent's results. Conference record and findings.

What these findings leave unresolved

These studies answer different questions. The direct evaluation measures outcomes and cost. The corpus studies describe file contents, and the smell study applies proposed classifications. Combining them does not turn observed conventions into experimentally validated best practices.

None of the observational studies above establishes a winning template. The direct evaluation's conclusion about minimal requirements also does not settle an exact rule that only information an agent cannot infer should remain. That would require a comparison designed to test that rule.

This article is a focused review of four publications, not an exhaustive systematic review or an independent replication. It makes no claim about an optimal number of instruction files, a universal line limit, or the performance effect of replacing a particular CLAUDE.md with skills. Those questions remain outside the findings summarized here.