Give a coding agent a good model and good tools and it will still ask you, for the fourth time this week, why the retry wrapper exists. It is not being slow. It genuinely does not know, because nothing in its world survived the end of the last session.
The obvious fix is a notes file, and the obvious fix rots in about three weeks. This post is about why it rots, and what to build instead. It comes out of the memory layer in Hilum Tools, which I run daily against a multi-repository workspace — including the one this site lives in.
The failure mode is not what you think
When people design a memory store they optimise for recall: will the agent find the note? That is the wrong worry. Retrieval failure is loud — the agent asks a question you already answered, you notice, you re-answer, life continues. Annoying, bounded, self-correcting.
The failure that actually hurts is silent. The agent finds a note, the note is wrong or stale or one of five near-copies that disagree, and the agent proceeds with total confidence. You do not get a question. You get a pull request built on a convention that was abandoned two months ago, and you find out in review — or you do not.
Two mechanisms produce almost all of it:
Near-duplicates fragment retrieval. Five slightly different notes about the same convention split the signal. Each individually looks authoritative and each carries a different detail. Whichever one wins the ranking becomes the truth for that session, and it is a coin flip which one that is.
Stale entries outrank current ones. A note written when the decision was fresh is detailed, well-phrased and rich in the exact vocabulary of the question. Its replacement is terse, because by then everybody knew. The old one wins on every similarity metric you can name. Recency is the one signal that would save you, and it is the one most stores weight lowest.
Both failures are created at write time and merely revealed at read time. That single observation dictates the whole design.
Rule one: the rules run at write time
If you enforce hygiene when reading, you are trying to repair a corpus you already corrupted, on the hot path, with a latency budget. It does not work, and it is expensive to attempt.
Enforce at write instead, where you have time, the full corpus, and the author still in the loop:
- Detect the near-duplicate as it is written, and say so. Not silently merge — report. "This is 0.91 similar to a record from June; update that one instead?" The author is right there and knows which is true. A merge decided by cosine similarity is a guess made by the component least qualified to make it.
- Archive, never delete. A superseded record still answers "why did we do it the old way", which is exactly the question that comes up when someone proposes doing it the old way again. Archived records leave the default retrieval path and stay reachable on request.
- Require a type. An untyped bag of text cannot be reasoned about. Four types cover almost everything in practice, and each has different lifetime and different retrieval weight.
| Type | Holds | Lifetime |
|---|---|---|
| Decision | A choice made, the alternatives, and why. The why is the payload — a decision without it is just a fact. | Long. Superseded, not deleted. |
| Pitfall | A trap already paid for, and the symptom that identifies it. | Long. The most valuable class per byte. |
| Convention | How things are done here, where the code does not make it obvious. | Medium. Drifts; needs review. |
| State | What is in flight, what is last-green, what is blocked. | Short. Must expire or it becomes a lie. |
The State type is the one people get wrong. A "current status" record with no expiry becomes actively harmful within days — it is a confident, specific, wrong description of the world, and it is the sort of thing an agent will act on without hesitation. Either it carries an expiry, or it is derived from ground truth (git, CI, the tracker) rather than stored.
Rule two: reading is automatic, writing is deliberate
Asymmetry is the point.
Reading must cost nothing. If the agent has to decide whether to consult memory, it will skip it exactly when it is under pressure, which is exactly when memory would have helped. The briefing arrives at session start whether it was asked for or not: conventions in force, decisions recently made, known pitfalls in the area being touched, current state. One call, one budgeted response.
Writing must be a decision. An agent that writes a note after every task produces a corpus made mostly of restated task descriptions, and that corpus is worse than no corpus — it dilutes the signal and inflates every retrieval. The write bar is a single question: would a competent engineer joining tomorrow be wrong without this? Answer no, do not write.
Which produces a short list of things never to store, and this list does more for quality than any ranking improvement:
- Anything the repository already records. Code structure, file layout, the dependency list, what a function does. That is what code intelligence is for; memory duplicating it guarantees the copy goes stale while the original does not.
- Anything git already records. Who changed what, when, and the message they left.
- Session narrative. "Tried X, it failed, tried Y" is context for the current conversation, not knowledge. It dies with the session, correctly.
- Anything already in a rule file or an ADR. Point at it; do not restate it. A restated fact is a second source of truth, and a second source of truth is a future contradiction.
Rule three: scope is part of the record
A note about "our commit-message convention" is true in one repository and false next door. A note about "how I like reviews structured" is true across every repository this person touches. Storing both in the same undifferentiated pile means one of them will leak into the wrong context and be wrong there with full confidence.
Three scopes are enough:
- Repository — travels with the repo, shared with whoever clones it.
- Host-global — this machine, this operator, every project. Preferences, environment quirks, personal workflow.
- Team / organisation — shared across members and repositories, and the only scope that needs access control.
Get this wrong and you get the two classic complaints about agent memory: "it keeps applying another project's rules here", and "I have to teach it the same thing in every repository".
Rule four: the store reports its own health
Memory is the one subsystem where being trusted is the whole product. So it has to be inspectable, and the inspection has to be cheap enough that it actually happens.
Track and surface: total records by type and scope; the near-duplicate cluster count; the archived-to-active ratio; the age distribution, specifically what fraction of State records are past their expiry; and the contradiction count — pairs of active records that assert opposite things about the same subject.
That last one is the alarm. A store with contradictions is not a store with a small quality problem; it is a store whose every answer is now a coin flip, and it will keep serving those answers with the same confident tone as before.
What to measure
The honest metrics for a memory layer are behavioural, not internal:
- Repeated-question rate. How often the agent asks something the corpus already answers. Falls fast when memory starts working; the easiest win to demonstrate.
- Repeat-mistake rate. How often a pitfall already recorded gets stepped in again. Slower to move, worth far more.
- Cold-start tokens. Tokens spent between session start and first useful edit. A briefing should collapse this dramatically — that is the whole economic argument.
- Contradiction count. Should be zero, and any non-zero value should be visible without anyone going looking.
The short version
- Design against confident wrongness, not against not-found. Retrieval failure is loud and self-correcting; stale confidence is silent and compounds.
- Enforce hygiene at write time, with the author present. Read time is too late and too expensive.
- Report near-duplicates rather than merging them. Archive rather than delete.
- Type every record. Give State an expiry or derive it from ground truth.
- Read automatically, write deliberately. Never store what the repository or git already records.
- Scope every record: repo, host, team.
- Make the store report its own health, and treat contradictions as an alarm.
Memory handles what should never have needed searching. The material that genuinely does need searching is the subject of token-budgeted retrieval; running several agents against one repository without them tripping over each other is the next problem along. If you want this layer built into your own stack, that is the AI development engagement.