The record and the territory
Stale by Design
Every record about a system is verified, stale, or was never true at all. Two of those three have detectors. Set who is holding the artifact, then check it against the repository and watch which line nothing was ever going to catch.
The month I spent blocked on nothing
For four weeks my own project state named one build as the blocking critical path. It was gated on two things. One had been cleared in June. The other was a branch that existed in no ref, local or remote.
These are the actual lines. Nothing was blocked, and blocker language made unscheduled look like blocked for a month.
Who is reading the record
Check it against the system
The planning artifact
STATE.md
four weeks
- 1
## Status
- 2
- 3
Critical path: the build, blocked.
- 4
- 5
- Gate 1 cleared on June 19.
- 6
- Gate 2 blocked on branch feat/cell-provenance.
- 7
- Receipt 010 still pending.
- 8
- Receipts 011, 012 and 013 outstanding.
Four claims, one hand, one confidence. Not one of them carries a date, a source, or a note about who checked it last. Nothing on this screen tells you which two are false.
A person is holding it
Caught eventually, and only by putting the notes down
Four consecutive Friday reviews read this file and repeated it, each one reading the last. The second line looked unfamiliar every single week and never quite enough to go and check. It broke on the week I stopped reading my notes and read the repository instead.
Time the wrong line survived
four weeks
Three states, two detectors
Staleness has a shape you can hunt. The phantom claim does not.
Nothing to catch.
A date, a diff, an expiry, a forced review. Every freshness tool ever built.
Nothing. It has no timestamp problem. It has a truth problem.
Every freshness mechanism ever built rests on one assumption. The record was right at least once, so the only question is whether the world moved since. The phantom claim breaks that assumption at the root. Nothing moved. It was wrong the moment it was written, it reads exactly like a verified one, and the entire class of tooling looks straight past it.
This is measured, not just anecdotal
The silent category is the one the machines are worst at.
Identification F1, by category
higher is better
- Test-Breakingfails loudly59.9
- Test-Missingleaves a visible gap52.9
- Test-Stalestill passes, so nothing raises a hand36
16.9 points lost on the category that stays quiet.
The ordering is the whole finding. Agents are worst by a wide margin at the one category that does not announce itself.
Breaking, stale, or missing
TEBench benchmarks coding agents on 314 tasks drawn from 10 Java projects. A stale test, in the paper's own definition, still passes after a change but no longer reflects what the code now means. A breaking test fails. A missing test leaves a gap. Only the stale one is silent.
Shang et al., arXiv 2605.06125 ↗Writing more down does not fix it
ETH Zurich and LogicStar.ai evaluated the AGENTS.md context files every agent vendor now recommends. They “do not generally improve task success rates, while increasing inference cost by over 20% on average.” That held for files written by humans as well as files generated by models. The document existed. It was not load-bearing.
Gloaguen et al., arXiv 2602.11988 ↗The claim
A stale record went wrong. A phantom claim was never right, and nothing you have installed was ever going to tell you.
Rather than treat a written record as evidence, treat it as a claim with a timestamp and a source, and hold artifacts to one rule. An artifact asserts only what is verifiable on a clean checkout or by re-running something. Everything else is a note, and notes are for people. That rule is cheap for a human team and load-bearing for an agentic one, because time to trusted action is not only about how fast the number arrives. It is about how long a confident wrong answer survives before anything contradicts it.
Read the full article
Stale by Design, on SubstackGet the next one
One essay a week on making data and AI systems trustworthy enough to act on.