Lead · 7 chapters
Repo conventions as agent input
Config files, skills and hooks applied to a real codebase
Poorly scoped or stale guidance makes agents confidently produce convention-breaking changes, so this is a practical reliability control rather than documentation polish.
Read these first
What you will know
Apply skills and config files to a real repository and measure the difference
Use a before-and-after task set and record compliance, test outcomes, latency, and token or tool-use overhead.
Show an agent following a repo convention it would otherwise miss
A useful demonstration includes a convention-sensitive change and a verifier that fails when the rule is violated.
Find the instruction that is being ignored, and why
Trace the loaded instruction sources, scope, ordering, and conflicts rather than blaming model randomness.
Keep repo-level guidance current as the codebase changes
Treat ownership, review triggers, and versioning as part of the interface, or stale rules silently steer edits.
Measure how a Stage 12 technique changes daily coding-agent quality on a fixed task set
Use acceptance rate, rework, and failure categories to separate a real quality gain from anecdotal improvement.
Chapters
- 1
More sources reach the model than the config file you wrote
Before diagnosing anything, take the inventory: a root config, the nearest package-level config, installed skills, hooks that fire around the run, the task prompt, and the retrieved file content itself, which teaches by example whether anyone intended it to or not. Each source has its own discovery and scoping rule — nearest file up the tree, per-package scope, whatever happens at a monorepo package boundary — and the harness assembles them into one request in an order nobody chose deliberately. Knowing that inventory and that order is what turns "was this rule even in scope for that package" into a question with an answer.
- 2
Check what was loaded before you rewrite the rule
The root config says in bold that currency uses the repository's Money type and never a float, and the agent keeps writing float arithmetic in billing code. Restating it louder is the last resort dressed up as the first. Trace instead: was the root file in scope for that package at all, did a nearer config override it, where in the assembled prompt did the line land, and what do forty existing files of float arithmetic demonstrate?
- 3
Precedence has to be written down, not left to prompt assembly
In a monorepo a root config, a package config, an installed skill and the task prompt all say something about error handling and none of them agree, so engineers describe the agent as inconsistent. It is resolving a conflict nobody specified. Make the rule explicit — most specific source wins, task overrides convention except where a rule is marked non-overridable — and log the resolved instruction set per run so surprising behaviour traces back to its source.
- 4
Compliance is an impression until identical tasks run both ways
Adding a conventions file and asserting that it helps is not evidence of anything. Assemble a task set whose changes are convention-sensitive, run it with the file and without it, and score it with a check that fails when the rule is broken, so the result is a compliance rate on identical tasks. What is held constant, what varies and what is scored are the three decisions that make the number mean something.
- 5
A rule with a check behind it holds; prose holds about half the time
Measured that way, two conventions in one file separate sharply: "use the structured logger rather than print" has a lint rule and holds in 97% of agent patches, while "prefer small, focused functions" has nothing and holds in 54%. This chapter separates what a paragraph buys — reasoning the model generalises from — from what an enforced check buys, which is a compliance number and a signal when it slips. A legitimate exception, such as the migrations package, is handled very differently by each.
- 6
Thirty-one points of compliance still costs tokens on every request
The comparison returns: convention compliance up 31 points, first-attempt CI pass unchanged, tokens per task up 18%, latency up 9%. Compliance is a proxy for review friction and maintainability, and neither of those was measured, while the cost side moved the wrong way on every request from now on. You will learn the metrics that separate a real quality gain from an anecdote — acceptance rate, rework, failure categories — and what result would say the work was wasted.
- 7
Stale guidance overrides the correct evidence sitting in the code
A rule written a year ago routes outbound calls through the v1 HTTP client wrapper; the codebase migrated to v2 six months ago; the agent confidently writes imports that no longer resolve. Guidance that is wrong is worse than guidance that is absent, because it outranks the code. Every rule therefore needs an owner and a trigger that forces review when the thing it describes changes. A 900-line file nobody owns gets audited rather than deleted, and the audit is a process rather than a reading: group the rules by the code they touch, check each against what the codebase does today, attribute each to an owner or the incident that produced it, and retire the ones with neither a check nor a measurable effect.
External resources
- Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?arxiv.org
Measures whether repo convention files actually change agent outcomes
- How to Use AI Agents in Large Codebases: Anthropic's 7-Strategy Framework | MindStudiomindstudio.ai
Despite intrusive product promotion and Claude-specific framing, it provides a substantive explanation of using root and nested CLAUDE.md files to encode architecture, conventions, testing requirements, boundaries, and module-specific guidance for agents.
- Standardize project context with AGENTS.md and Agent Skillsdevelopers.redhat.com
Step-by-step setup of convention files and skills in a real project
- Implementing CLAUDE.md and Agent Skills In Your Repositorygroff.dev
What belongs in a convention file versus a skill versus a hook
- Master 95% of Claude Code Skills in 28 MinutesNate Herk | AI Automation
Deep dive on skills as reusable repo-scoped agent instructions