The platform is live. A private workspace for every engagement, and an agent that answers from your project or says it does not know.

Architecture essay · 14 min read

The five files that run this codebase

An AI writes code here every day. This is the system that stops it drifting, and lets the codebase get smarter instead of noisier.

Standing instructionsDecision ledgerGuardrailsThe loop

This is the system running on this platform, a multi-tenant product with client data and real money moving through it. The numbers are measured in that repository. Where something has a hole, the hole is named, because a system you only hear the wins about is not one you can copy.

The codebase has a new kind of colleague

The situation

An AI writes a meaningful share of the code in this repository. It reads the codebase, follows the conventions, and produces work that goes to production. That is now normal, and most of the discussion about it is about the model.

The model is the least interesting part. What decides whether this works is much older and much duller: whether the place it is working in has written down what it needs to know, and whether anything checks that those instructions are still true.

A new engineer who joins a company gets an induction, asks questions for a fortnight, and slowly builds a sense of how things are done here. An AI gets none of that. It arrives every single morning with no memory of yesterday, reads whatever is written down, and acts on it with total confidence, including when it is wrong.

So the written-down part stops being documentation and starts being infrastructure. This is what that infrastructure looks like here: five artifacts, what each one is for, and the loop that keeps them honest.

What the check found

measured in this repository, not estimated

937claims in the docs, checked against the code

16of them had drifted

100%of drifted rules were held by memory

0checked rules had drifted

42.5ktokens read before every task, before scoping

176decisions on the ledger, none rewritten

I checked my own. Every claim in the documents that govern this codebase, tested against the actual code. Nine hundred and thirty-seven of them.

Sixteen were wrong. The rules describing how client data is separated named three database tables that had never existed, while leaving out sixteen that did. Two index files listed twelve sets of rules when there were fourteen, so two whole areas of the system were invisible to anyone reading them.

None of that was sloppiness. Every line was correct on the day it was written. What none of them had was anything that would notice when it stopped being true. And a stale instruction is worse than a missing one, because an AI does not hesitate over it. It follows it.

The split was total, and it is the reason this system has the shape it does. Every instruction with an automatic check behind it had survived intact. Every instruction that relied on a person remembering had drifted. Not most of them. All of them.

Held by a mechanism5 of 5 still true

The decision log stays in date orderA test
Archived designs carry a warningA template
Every config pointer resolvesA test
Every model has a priceA type
Database writes read their errorA lint rule

Held by memory0 of 5 still true

The index lists every rules fileMemory
One directory never imports from anotherMemory
Query keys have one sourceMemory
The schema file range is currentMemory
The tenant table list is completeMemory

The only variable. Both columns were written by the same person, in the same weeks, with the same care. One column had something watching it.

Five parts, and what each one is for

what each artifact is, and what stays out of it

Read the top of each card and you have the whole system. The block at the bottom of each one opens the files, the numbers and the mechanism, for anyone who wants to build it.

The standing instructions

CLAUDE.md

The short document the AI reads before it does anything at all, every single time.

The induction pack you hand a contractor on their first morning: the handful of things about this place they could not reasonably work out for themselves.

What goes in

The rules that cost money or trust if broken
The traps a newcomer falls into, and why
Where to find the deeper rules for each area
What must never happen without asking

What stays out

Anything the code already says clearly
A tour of the folder structure
Lists of files, which go stale the day you write them
Explanations of how the technology works in general

The temptation is to make this file complete. Resist it. Every line here is read before every task, whether or not it is relevant, and attention is finite for a machine in much the same way it is for a person. A document that explains everything gets followed less carefully than one that explains four things.

The test for whether something belongs is simple: could a careful reader work this out by looking at the code? If yes, leave it out. If no, and getting it wrong is expensive, it belongs here.

Mine had grown to 357 lines and I had never measured what it cost. When I did, it and the deeper rules alongside it were consuming roughly 42,500 tokens before a single instruction from me arrived. Against a 200,000-token window that is a fifth of it, spent before the work starts. It is now about half that.

Show the mechanismfor engineers

The 2026 research is unusually pointed here. A study across 438 tasks and four coding agents found that context files did not improve task success rates while raising inference cost by over 20 percent, and that repository overviews specifically, the architecture tour most of these files open with, were measurably unhelpful. The value concentrated in non-standard practices: the gotchas an agent cannot infer.

Anthropic removed more than 80 percent of Claude Code's own system prompt for the Claude 5 generation with no measurable loss on their coding evaluations. Separately, instruction-following degrades as instructions stack: one 2026 study measured follow rates falling from roughly 96 percent to 20 percent across 24 stacked instructions.

The practical move is loading by relevance rather than by default. Most of the domain rules here now carry a frontmatter glob, so a file governing the billing code is injected when you touch the billing code and not otherwise. Five cross-cutting files stay resident because they apply everywhere.

Before scoping: 2,571 lines, ~42,500 tokens, every session
Share of the window: 21%, before any instruction from me
After scoping: ~21,600 tokens resident, the rest on demand
Kept resident: 5 of 15 files

The domain rules

.claude/rules/*.md

One deeper document per area of the system, loaded only when the work actually touches that area.

The specialist manuals on the shelf. Nobody reads all of them. You pull the one for the machine you are standing in front of.

What goes in

How this specific area works and why it is built that way
The invariants that must hold, stated as rules
The mistakes already made here, so they are not repeated
What to re-run before calling a change done

What stays out

The story of how it came to be built that way, which belongs in the ledger
Anything already stated in another rules file
Copies of facts the code or the folder listing already owns

Splitting the rules by area does two things. It keeps each document short enough to actually be followed, and it means the AI reads about billing when it is working on billing, rather than carrying every rule about every subject at all times.

The discipline that matters most here is subtraction. A rules file that only ever grows becomes a file nobody follows, including the machine. So the standing instruction is that these files must shrink as often as they grow: when a rule is replaced by an automatic check, the rule comes out.

I am not especially good at this yet, and the measurement says so. Across the life of this system the rules layer has added about 3,500 lines and removed about 1,000, and there has been exactly one genuine deletion day.

Show the mechanismfor engineers

Each file carries a `paths:` frontmatter list of globs. A file with no globs is treated as cross-cutting and loads every session; the rest load when the agent touches a matching path. Getting a glob wrong is a quiet failure, since the rule simply never loads, so a check asserts that no glob matches zero files.

The rule that keeps these honest is that a rules file may not restate a fact the filesystem already owns. No file enumerations, no copied constants, no table lists. Every hand-maintained list in this repository had drifted when it was checked, so the lists were replaced with pointers and a test now fails if one comes back.

---
paths:
  - "features/billing/**"
  - "app/api/**/checkout/**"
  - "supabase/schemas/14-billing.sql"
---

# Billing rules

Every surface that spends on inference is gated...

The frontmatter is the whole mechanism. This file is invisible until someone opens the billing code, and unavoidable once they do.

The decision ledger

docs/DECISIONS.md

A dated, append-only record of every non-obvious choice and the reasoning behind it. Entries are never edited or deleted.

The minutes book. You do not go back and rewrite last year's minutes because you changed your mind. You record the new decision and note that it replaces the old one.

What goes in

Why this approach and not the obvious alternative
What the trade-off was, stated honestly
What was rejected and the reason
The date, always

What stays out

How to do the thing, which belongs in the rules
Edits to old entries, ever
Decisions that were obvious, which is most of them

This is the part I would keep if I could keep only one. Code shows you what a system does. It almost never shows you what was considered and rejected, and that is the thing that gets re-argued every few months by someone who was not in the room.

The rule that makes it work is that entries are never rewritten. When a decision is overturned, the old entry stays exactly where it is and gets a marker saying it is dead and which decision replaced it. Deleting it would be tidier and would destroy the thing that makes the log worth having, which is that you can see how the thinking moved.

There are 176 entries here now. The AI reads them, which means it can tell me that something I am about to do contradicts a decision I made in July, and point at it. That is the moment this stopped being documentation and became memory.

It also has a real weakness worth naming: at over 6,000 lines, nobody reads it end to end, machine included. It needed an index before it was useful, which is generated rather than maintained by hand.

Show the mechanismfor engineers

Append-only, newest first. A test enforces the ordering, so an entry filed out of sequence fails the build rather than quietly hiding itself below the fold.

Supersession is a stamp with an exact format, so one search finds every dead claim in the log. The format is enforced, and it has to be: an earlier check found that half the stamps had drifted to a different casing, which meant no single search found them all.

The index is generated from the headings and pinned by the same test, so it cannot go stale. It turns a roughly 118,000-token read into a 5,500-token scan followed by a targeted read of one entry.

## 2026-08-11 — A guard that cannot fail is worse than no guard

Three tests written to prevent a defect class could not
detect it. Two matched their own explanatory comments...

> SUPERSEDED 2026-09-02 by "Source scans move to AST"
> — the lexer approach did not survive minified fixtures.

An entry and, underneath it, what happens when it is later overturned. The original stays. Nothing is quietly rewritten.

Entries: 176, spanning two months
Superseded, stamped: 9 fully, 2 more superseded in part
Read whole: ~118,000 tokens, which is why the index exists
Via the index: ~5,500 tokens to find the right entry

The guardrails

eslint-rules/, tests/, next.config.ts

The automatic checks that make an important rule impossible to break, rather than merely discouraged.

The interlock on a machine press. The sign saying keep your hands clear is a rule. The switch that stops the press when the guard is open is a guardrail. You want the sign, but you need the switch.

What goes in

Anything where being wrong costs money, leaks data, or is silent
Anything that has already gone wrong once
Anything a person has to remember at exactly the moment they are busy

What stays out

Matters of taste
Rules nobody has ever actually broken
Anything where the check would cost more than the mistake

This is the difference between a rule and a guarantee, and it is the whole reason the checked instructions survived while the remembered ones did not.

A guardrail does not have to be sophisticated. The most-repeated rule in my documentation was never commit anything without asking me first, written in twenty-one different places, which is a strong signal that writing it down was not felt to be enough. It is now a permission rule: the command still works, it just raises a prompt that has to be accepted. Four lines of configuration do what twenty-one paragraphs could not. And then the part that matters more, because it is the step this whole system is supposed to force: the paragraphs are still there. Nineteen files still carry the prose the mechanism made redundant. Deleting them is the loop, and I have not run it yet.

The important discipline is the unglamorous one: state what each guardrail does not cover. A check whose limits are unwritten gets trusted past them, and that is its own kind of failure. Every one of mine carries a note saying what it misses.

Show the mechanismfor engineers

The ladder, cheapest first: prose, a comment, a convention plus review, a test, a lint rule, a type derived from the data so a mistake cannot compile, a build-time guard, a permission rule. Most conventions belong on the lower rungs. The move is finding the lowest rung that makes the mistake impossible rather than discouraged.

Worked examples running here. An unpriced model used to log an alarm after the call had already billed at zero; the model identifier type is now derived from the price table, so naming a model with no price does not compile. A missing environment variable used to fail in front of a user; the build now fails instead, with no opt-out flag. A data export that silently omitted a table used to pass a test that asserted a hand-written list; the test now reads the schema and fails on any table neither exported nor explicitly excluded.

And the part worth saying out loud: a green test is a claim, not a result. Three checks written here in one session turned out to be unable to fail, two of them because they scanned source code and matched their own explanatory comments, asserting that the documentation existed rather than that the code did. Every guardrail now gets broken on purpose once before it is believed.

// Before trusting it, break it.
//   delete the argument the guard exists to protect
//   -> the test must fail
//   restore it
//   -> the test must pass
//
// Two of the three guards written this session
// passed the first step. They were checking
// their own comments.

Mutation is the only evidence that a check works. It takes minutes, and it is the step almost everyone skips.

The loop

The habit, not a file

What happens after every round of work: what gets written down, what gets deleted, and what gets promoted from a rule into a check.

The end of a shift. You log what happened, you fix the thing that nearly went wrong, and occasionally you install a switch so it cannot happen again.

What goes in

Record why, in the ledger, in the same change
Ask whether the rule needs to exist at all
When something goes wrong twice, build the check
Delete the rule the check replaced

What stays out

Adding a paragraph and calling it fixed
Leaving a rule in place after a mechanism replaced it
Closing an item without deleting it from wherever it was tracked

Everything above is inert without this. The loop is what turns a set of documents into something that improves.

The rule I hold myself to is that a defect never becomes another paragraph. When something goes wrong, the fix is the cheapest mechanism that makes it not happen again, and if a rule already existed and did not hold, that is evidence the rule was on the wrong rung, not that people need to try harder.

The counter-intuitive result is that a system running this loop gets shorter over time while getting stronger, because each new check retires the prose it replaced. That is the test of whether the loop is actually running. If the documentation only ever grows, it is not.

I run a version of this deliberately: periodically, several AI reviewers go through the codebase in parallel, each told to attack a different aspect, and then a second round tries to refute what the first round found. That is how the 937 claims got checked, and how I found out that three of my own new guards were useless.

Show the mechanismfor engineers

Concretely, per round: append the decision entry in the same change as the code, never afterwards. Check whether an existing rule covers it, and if one did and failed, promote rather than restate. Delete the superseded prose in the same commit. If a hand-maintained list appears anywhere, replace it with a derivation.

The review round is worth running adversarially rather than confirmatively. Reviewers told to find problems find some; reviewers told to refute a specific claim, and to report what they attacked and could not break, produce something you can act on. Two of mine reached opposite conclusions on a double-charge path, which was more useful than either verdict alone, because it located exactly the line worth reading.

Claims checked in one round: 937
Confirmed drift found: 16, every one a hand-maintained list
New guards written: 5
Of those, unable to fail: 3, found by the next round

The loop

the habit that keeps the other four honest

1Something breaksor nearly does
2Write down whythe ledger, same change, dated
3Does it need a rule?most things do not
4Did the rule hold?check, do not assume
5Build the mechanismcheapest rung that works
6Delete the rule it replacedthis is the step everyone skips

and round again

The loop. Step 6 is the one almost nobody runs, and it is the one that decides whether the system gets smaller as it gets stronger. If the instructions only ever grow, the loop is not running.

The prompt

run it against your own repository

Take this with you

Make this repository AI-native

Point a coding agent at any repository and run this. It leaves behind the five parts described above: the standing instructions, the scoped rules, the ledger, the guardrails and the loop. It audits first and changes nothing until you say so, refuses to write a rule for something that has not actually gone wrong, and proves every guardrail by breaking it on purpose. Works in any language or stack.

00
What done looks likeFive things in the repository, none of them containing a claim that nothing checks. If some exist, repair them. If none do, build them.
01
Audit, change nothingWhat exists, whether it is still true with counts, what the code actually does, and what has already gone wrong. Then stop and wait.
02
Propose the cheapest mechanismOne guardrail per thing that has actually broken. No hand-maintained lists. Say what each one will not catch.
03
Build them, break each oneCopy the file, introduce the exact defect, confirm the check fails, restore byte-identical. A guardrail that has never failed is decoration.
04
Write the instruction layerThe standing instructions, as short as they can be while true everywhere, and a rules file per area declaring the paths it governs.
05
Start the ledger and the loopAppend-only decisions, an index generated from the log itself, and the rule that a fix becomes a mechanism rather than a paragraph.

Run it on Opus 5 with maximum effort. Phase 1 is a judgement task across a whole codebase, and a weaker setting produces a plausible list rather than an evidenced one.

# Make this repository AI-native

You are working in a repository you did not write. Your job is to leave behind a
small system that lets an AI agent do useful work here without drifting.

## What done looks like

When you finish, this repository has five things, and not one of them contains a
claim that nothing checks:

1. **Standing instructions.** One short file an agent reads before every task,
   holding only what it cannot infer from the code.
2. **Domain rules.** Deeper files, one per area, each declaring the paths it
   governs so it loads when that area is touched and not otherwise.
3. **A decision ledger.** Append-only, dated, never rewritten. Why things are the
   way they are.
4. **Guardrails.** Automatic checks for the invariants that actually cost
   something, each one proven by breaking it on purpose.
5. **A loop.** Written down, so the next defect becomes a mechanism rather than
   another paragraph.

If some of these already exist, audit and repair them. If none do, build them.
Either way the governing idea is the same: an instruction nobody checks will go
stale, and a stale instruction is worse than a missing one, because an agent
follows it without hesitating. Every claim you write must either be verifiable
against the code today, or be replaced by a mechanism that verifies itself.
Prefer the mechanism.

## Absolute constraints

- Never run git commit, git push, git reset --hard, or any history rewrite.
- Never run migrations, deploys, or any command that touches a live environment.
  If a command's blast radius is unclear, do not run it. Ask.
- Never delete or overwrite an existing file without showing the diff and asking.
- Do not install dependencies without asking.
- Report what you could not verify. An unverified claim stated as fact is the
  exact failure this system exists to prevent.

## Phase 1: audit. Change nothing.

Read only. Produce a written report, then stop and wait.

**1a. What already exists.** Look for agent instruction files at the repo root
and in any tool config directory, plus README, CONTRIBUTING, ARCHITECTURE, docs,
and ADR directories. For each: path, line count, rough token cost (words divided
by 0.75), last modified, and how much of it restates something the code already
says. If there is nothing, say so plainly and move on. That is a normal starting
point, not a failure.

Then check whatever exists for truth. Sample every concrete claim: does that file
exist, does that function have that name, does that command run, is that list
complete? Report how many you checked and how many were wrong. Do not soften it.

**1b. What conventions the code actually follows.** Do not ask anyone. Read.
The layering, the entry points, the error-handling shape, how modules are wired,
what gets validated where. Note where the pattern is consistent and where it has
forked into two dialects. A forked pattern is worth writing down. A consistent
one usually is not, because the code already teaches it.

**1c. What has already gone wrong.** The most valuable part, and the easiest to
skip. Evidence, not intuition:

- Commit messages shaped like incidents: fix, revert, hotfix, regression,
  "again", "actually". A bug fixed twice is a guardrail waiting to be written.
- Defensive comments: "do not remove", "must run before", "looks redundant but".
  Each is an invariant someone learned the hard way and could only write in prose.
- Clusters of TODO, FIXME, HACK. Where they cluster matters more than how many.
- Anything touching money, authentication, tenant or user isolation, data
  deletion, external calls, retries, background jobs. Silent failure costs most here.
- The same defensive check repeated in many places. Repetition under duress is an
  unmechanised invariant.

If the history is squashed or too short to be useful, say so and lean on the
other sources. Do not invent incidents to fill the list. A short evidenced list
is the correct output; a long speculative one is worse than none.

**1d. What verification exists.** Test framework and how it runs. Linter, and
whether it gates or merely reports. Type checking. CI: what actually blocks a
merge. Then the question that matters: of the invariants in 1c, which are caught
automatically today, and which depend on someone remembering?

**Stop here.** Report what exists, what is stale with counts, the invariants with
their evidence, and the gap between the two. Wait for a decision.

## Phase 2: propose

For each invariant from 1c, propose the cheapest mechanism that would catch a
violation. Cheapest first, because the cheapest one that works is the one that
survives:

1. A type or signature that makes the wrong thing fail to compile.
2. A test that derives what it expects from the codebase rather than from a list
   you typed: it walks the tree, parses the config, reads the schema. This is the
   one that does not go stale, because there is nothing to update.
3. A lint rule, custom if the linter supports authoring one.
4. An assertion at build or startup, so a misconfigured deploy fails before it
   serves traffic.
5. A CI step or hook.
6. Prose. Last resort, for what genuinely cannot be mechanised.

Rules for the list:

- **One guardrail per thing that has actually broken.** If you cannot name the
  incident, the commit, or the defensive comment behind it, do not propose it.
- **No hand-maintained list, ever.** If it needs someone to add a line when they
  add a file, it is already broken. Derive it.
- Say what each will not catch. A guardrail whose limits are unwritten gets
  trusted past them.

Present it. Wait for approval before building.

## Phase 3: build the guardrails, and break each one

Confirm the working tree is clean, or that every file you will touch is
unmodified. If not, stop and say so. This phase edits source and must not run
over uncommitted work.

One at a time:

1. Write it. Run it. Confirm it passes on the correct codebase.
2. **Break the invariant deliberately.** Copy the file to a temporary location
   outside the repo first. Introduce the exact defect the guardrail exists to
   catch. Run again. It must fail, and the message must name the file and say
   what is wrong.
3. Restore from the copy, not from memory. Confirm byte-identical. Re-run, green.
4. Record what you broke, the failure output, and the restore.

Not optional, and not skippable for being obvious. A guardrail that has never
failed is decoration, and worse than nothing because it manufactures confidence.
Expect some of your first attempts to pass while broken. That is the most common
defect in this kind of work. If you cannot make one fail on purpose, delete it.

## Phase 4: write the instruction layer

Now write the standing instructions and the domain rules, with the guardrails in
place so the prose no longer has to carry what a check now covers.

**The standing instructions.** One file, as short as it can be while remaining
true everywhere. It holds: what an agent cannot infer from reading the code, what
is deliberately not obvious, what is dangerous, the commands to run, and a
pointer to where the domain rules live. Nothing else. Specifically it does not
hold a tour of the directory structure, an explanation of the framework, or any
list of files, modules or tables. Point at the directory instead.

**The domain rules.** For each area with rules of its own, one file, named for
the area. At the top, declare the paths it governs, in whatever form the local
tooling reads, so it loads when that area is touched and stays out of the way
otherwise. Inside: how this area works and why, the invariants that must hold,
the mistakes already made here, and what to re-run before calling a change done.
Do not restate anything another rules file owns.

If an instruction file already existed, this phase is mostly deletion: remove
everything the code, the filesystem, or a new guardrail now states, and move
whatever is area-specific out into its own file. Report the token cost before
and after. If it grew, justify it.

## Phase 5: the ledger and the loop

Create an append-only decision log, newest first. One entry per non-obvious
decision: date, what was decided, what was rejected, why. It records reasoning,
not instructions. The rules say what to do; the ledger says why that was chosen,
so the argument is not had again from a worse position.

Two properties keep it usable. An entry later overturned is marked in place with
one unmissable line pointing at what replaced it, so a reader going top-down
cannot implement a dead decision. And once it is too long to read whole, generate
an index of headings and line numbers from the log itself, with a check that
fails when the index is stale. Never hand-maintain that index.

Then write the loop into the standing instructions, as the closing section:

- A decision gets its ledger entry in the same change as the code, never after.
- When something breaks twice, build the check. Do not add a paragraph.
- When a check replaces a rule, delete the rule in the same change.
- These files must shrink as often as they grow. If they only grow, the loop is
  not running.

## Final report

State plainly: which of the five parts existed before and which you created; what
each guardrail catches and what it does not; which you proved by breaking and
which you could not; what you deleted; and what you found in the audit but
deliberately did not act on. List every claim you could not verify. Recommend
nothing you have not tested.

What I actually think, stated plainly

the position

A rule is a request. A guardrail is a guarantee. Almost everything a team calls a standard is a request wearing the language of a guarantee, and the gap between those two is where the incidents live.

So the move is not to write the rules more carefully. It is to take the handful that would actually cost you something and push each one down a rung, into a test, a type, a build step, something that fails without anyone remembering. Then delete the prose it replaced. That last part is the one I am worst at, and the measurement says so: across this system's life the rules layer added about 3,500 lines and removed about 1,000, on exactly one day.

The honest version of the claim is narrower than the one I would like to make. I cannot show you that this produces better code. Nothing here measures output quality, and the strongest study on repository context files found no improvement in task success rates at all. What I can show is what happens to a written instruction that nothing checks, which is that it stops being true and nobody notices, and that an agent then follows it with complete confidence.

There is one more thing I believe now that I did not a week ago. I wrote five of these mechanisms in a day and recorded in the commit that each had been verified by breaking it first. A later review with instructions to refute rather than confirm found that three of them could not fail. Two were matching their own explanatory comments, asserting that the documentation existed rather than that the code did.

So: an author cannot meaningfully verify their own guardrail, for the same reason nobody proofreads their own writing well. You see what you meant. Whatever checks the checks has to be something other than the person who wrote them, and on this platform that is a second pass whose instructions are to refute the first.

What this is built on

verified against the primary sources, not from memory

01Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?Gloaguen, Mundler, Muller, Raychev, Vechev (ETH Zurich / LogicStar) · arXiv:2602.11988v2, MemAgents workshop at ICLR 2026 · 23 June 2026supports

Across 300 SWE-bench Lite tasks and 138 CTXBENCH tasks with four agents, repository context files did not improve task success rates, and repository overviews in particular showed no benefit. The caveat matters as much as the headline: when existing documentation is REMOVED from the repository, model-generated context files then improve performance by 2.7 percent. The honest reading is that these files were largely restating documentation the agent could already reach, not that written context is useless.

Read the source

02The new rules of context engineering for Claude 5 generation modelsShihipar (Anthropic) · claude.com engineering blog · 24 July 2026supports

Anthropic removed more than 80 percent of Claude Code's own system prompt for the Claude 5 generation with no measurable loss on their coding evaluations, and recommends keeping instruction files light while spending the tokens on gotchas inside the codebase. First-party, and the evaluations behind it are internal and unpublished, so treat it as a strong signal rather than an independent result.

Read the source

03Instruction Stacking CollapseAnand, Chattaraj · arXiv:2608.02639 · 31 July 2026supports

Instruction following degrades as instructions stack, tested up to 20 simultaneously stacked instructions. The widely quoted 20 percent floor is GPT-5-mini specifically; Sonnet 4.6 holds 60.4 percent and Gemini 43.3 percent at the same load. The direction is robust across models. The magnitude is not, and quoting the worst number as though it were general is the mistake to avoid.

Read the source

04Repository context files and coding-agent efficiencyLulla et al. · arXiv:2601.20404 · January 2026against

Measured a 28.6 percent reduction in median runtime and 16.6 percent fewer output tokens with context files present. This runs directly against the cost finding above, and the two have not been reconciled. Citing only the result that suits you is not reporting the state of the evidence.

Read the source

05Bimodal effects of agent context files on merge ratesRepository-level study of contribution outcomes · arXiv:2606.13449 · June 2026against

27.7 percent of projects saw merge rates improve by 20 percent or more, while 26.35 percent got worse. The average effect is close to nothing, and the average is the least interesting number in the study. Whether these files help appears to depend on the repository rather than on the practice.

Read the source

06Probe-and-refine guidance for coding agentsShepard, Albrecht · arXiv:2606.20512 · June 2026context

Guidance tuned by probing the repository reached 33.0 percent on SWE-bench Verified against 25.5 percent unguided. The gain came from locating the right file rather than from writing a better patch, which is a useful frame: this kind of context helps an agent find, not decide.

Read the source

Two of these argue against the position taken here and are marked as such. The evidence on instruction files is genuinely unsettled: one study finds them costly, another finds them cheaper, a third finds the effect splits almost evenly between repositories that improved and repositories that got worse. What none of them contradicts is the narrower claim this piece actually makes, which is about what happens to a written instruction that nothing checks.