Open a fresh session in Claude Code or Cursor on a repository you have worked in for months, and watch what the agent does before it writes a single line.
It lists directories. It greps for a symbol and gets forty hits. It opens six files whole to find out which of the forty matter. It follows an import chain by hand. It reads a config to work out how the thing is wired. Somewhere in there it asks you a question you answered on Tuesday.
None of that is the task. All of it is billed. And tomorrow it happens again, identically, because nothing the agent worked out today survived the end of the session.
I call this the cold-start tax, and once you start measuring it rather than feeling it, it turns out to be the largest single line item in most agent workflows — larger than the actual editing, often by a wide margin.
Where the tokens actually go
Strip a few dozen agent transcripts down to what was being investigated rather than changed, and the same four questions come back in nearly every session:
- What is the shape of this thing? Directory layout, module boundaries, what lives where. Re-derived by listing and opening.
- Where is X defined, and who calls it? Answered by grep, which returns every string match including comments, tests, and the unrelated variable with a similar name. The agent then opens files to disambiguate.
- What does this library actually do in this version? Answered from training data, which is frequently a major version behind, then corrected the hard way when the call signature is wrong.
- What was decided here, and why? Usually not answered at all. The agent proposes something that was tried and rejected six months ago, and either you catch it in review or you do not. Making that answerable is the decision graph.
Each of those is cheap once. The problem is that none of them are once. They are per session, per agent, and — if you run several agents in parallel — per worker. Five agents on one repository pay the cold-start tax five times, in parallel, for identical information.
Why a bigger context window does not fix it
The intuitive response is to throw the whole repository at a large-context model and stop worrying. It does not work, for three separate reasons.
Cost scales with what you send, not with what you needed. Loading 400,000 tokens of repository to answer a question that needed 3,000 is not thoroughness, it is a bill. Multiply by every session and every worker.
Attention degrades before the window does. A model with a very large window is not uniformly good across it. Material in the middle of an enormous context gets attended to less reliably than material near the edges — so the answer arrives less accurate and more expensive, which is the worst of both directions.
It is stale by construction. A repository dump is a snapshot. The moment another worker commits, or you switch branches, the context describes a tree that no longer exists — and nothing in the prompt tells the model that. Confident answers about code that changed are worse than no answers.
The fix is not more context. It is the right context, resolved, at the moment it is needed — and, for the things that do not change per task, not re-deriving them at all.
Three layers that actually remove it
Layer 1 — the instruction layer (cheapest, do this first)
Most repositories tell an agent nothing about how to work in them. Everything the team knows — where things live, what the conventions are, which directories are load-bearing and which are archived — lives in people's heads and in review comments.
Writing it down is unglamorous and has the best return of anything on this list, because it is pure text a model reads once per session at negligible cost:
- An
AGENTS.md/CLAUDE.mdat the root with the rules that actually matter: commit conventions, what must never be edited, which commands verify a change, the language and formatting policy. - Navigation indexes — one per directory of any size, saying what lives there. This replaces the "list and open six files" reflex with a single read.
- Per-directory rules where a subtree has its own conventions, with a clear precedence rule (nearest ancestor wins).
- A reading manifest for common task types: "before touching billing, read these four documents."
A useful discipline here: keep genres separate. A file that says what lives here is not the file that says how to work here, and neither is the human-facing README. Mixing them produces a document nobody maintains and the agent half-reads.
Layer 2 — the retrieval layer
Grep is a string matcher pretending to be a code-intelligence tool. It cannot distinguish a definition from a mention in a comment, it does not know that a method is an implementation of an interface, and it returns forty results when one is correct.
Replace the common investigations with structural operations: go-to-definition, find-references, call hierarchy, project structure. These are resolved answers rather than string matches, and they return in tens of tokens what a grep-and-open cycle spends thousands on.
For the fuzzy questions where you do not know the exact string, the retrieval layer needs a budget rather than a top_k. That is a topic of its own — see token-budgeted retrieval for how selection under a token ceiling is built and measured.
Layer 3 — the memory layer
Layers 1 and 2 make investigation cheap. Layer 3 removes it, for everything that does not change per task: decisions and their reasoning, pitfalls already paid for, conventions the code does not make obvious, and the current state of what is in flight.
The trap is that a naive notes file rots within weeks — near-duplicates fragment retrieval, and stale entries outrank current ones because they are wordier. The rules have to run at write time rather than read time. Persistent agent memory covers the design in full.
And if you are running more than one agent, the coordination failures are their own category — double claims, silent hangs, output lost with the process. That is many agents, one repository.
Measure it before you fix it
The argument for any of this collapses without numbers, and the numbers are easy to collect. Take ten representative tasks and record, per task:
| Metric | How to read it |
|---|---|
| Tokens to first useful edit | The cold-start tax itself. The gap between session start and the first line of real work. |
| Total tokens per completed task | The headline cost. Should fall alongside the first metric, and by more. |
| Wrong-file edits | How often the agent modified something it should not have. A structure problem, not a model problem. |
| Repeated questions | Questions the agent asks that it asked before. Directly measures whether memory is working. |
| Wall-clock to first review-ready diff | What you actually feel. Includes the investigation the token count hides. |
Then apply Layer 1 only — a day of writing, no tooling — and re-run the same ten tasks. In my experience the first two metrics move immediately and substantially, which is usually enough to justify Layers 2 and 3 to whoever controls the budget.
The order that works
- Write the instruction layer. A day. No new dependencies. Largest return per hour spent.
- Measure. Ten tasks, before and after. You need the baseline to argue for anything further.
- Add structural code intelligence. Definition, references, call hierarchy over MCP, so the agent stops grepping.
- Add memory. Decisions, pitfalls, conventions, state — read automatically, written deliberately.
- Only then parallelise. Fanning out before the cold-start tax is fixed multiplies the tax by the number of workers.
That order is not arbitrary. Every step makes the next one cheaper, and skipping to step five — which is what most teams do, because parallel agents look like the exciting part — buys you five agents each paying full price for the same rediscovery.
Where this comes from
The three layers above are the shape of Hilum Tools, the MCP platform I build and run daily against a multi-repository workspace: nine tool domains over a single server, giving agents resolved structure, budgeted retrieval, durable memory, decision provenance and coordination — instead of grep and a guess.
Underneath all three layers sits one rule worth reading on its own: when the consumer of a service is an agent rather than a human, an answer has to state its own reliability. Everything above is a consequence of taking that seriously.
If you would rather have this done to your codebase than build it yourself, that is the AI development engagement — an audit of how agents currently fail on your repository, the instruction layer written, the tooling wired, and a written handover of what changed and why.