Explore
Find the second number
Every governed system has the number it publishes and the number its code actually compares against. This one had both, five points apart, and nobody noticed for months. Drag a score into the gap and watch an agent that did exactly what it was told come back marked wrong.
One score, two readers
The disagreement
On the left is the envelope the agent receives. On the right is the record the platform writes after the action commits. Same score, two rulers, built from different constants.
The marked stretches are every score where the two readers disagree.
Trust contract · envelope
served 09:14:02
Net Revenue
FY26 Q3 · Controllership workspace
Composite trust score
82
Thresholds published to the agent
A score of 82 lands in the Act zone of this ruler.
The agent reads this envelope, sees 82 against an act threshold of 80, and chooses act.
retrieval r-7f3a91 · as served
Grading record
written 09:14:06
action a7f3-22
reallocate 2.4M against Net Revenue · committed
- Agent chose
- Act
- Ground truth
- Caveat
Bands the grader used
A score of 82 lands in the Caveat zone of this ruler.
The dashed line is where the envelope said to act. The grader's own cut sits 5 points to the right of it.
Recorded incorrect
The agent followed the envelope and was recorded as wrong, against a number it was never shown.
Every score from 80 to 84 lands the same way.
That switch is the whole fix. The verdict scale still exists and still describes how good a number is. It no longer decides whether a customer's agent behaved.
One band, three deployments
Whose number is it
Three workspaces read the same metric and carry genuinely different costs of acting wrongly. Each ruler below is that team's own. The dashed line is the vendor's single global band cutting across all three.
The score marker is the one you set above, still at 82.
Controllership · during close
A wrong number in a filed statement is a restatement.
A score of 82 lands in the Caveat zone of this ruler.
Acts at 92. On this score the global band happens to agree.
FP&A · rolling forecast
A wrong forecast is revised next week. Waiting costs a planning cycle.
A score of 82 lands in the Act zone of this ruler.
Acts at 80. On this score the global band at 85 calls it caveat and this team called it act.
Growth · campaign sizing
Acting wrongly costs spend. Not acting at all costs the quarter.
A score of 82 lands in the Act zone of this ruler.
Acts at 64. On this score the global band at 85 calls it caveat and this team called it act.
There is no position that satisfies all three, because their costs are not the same. A single shipped cutoff is not conservative. It fits the median customer, and nobody is the median customer.
Why nobody caught it
Where the suite looked
The fixture generated a score and asserted the agent was right to act on it. Drag the pin and watch which part of the axis it can reach.
grading.fixture.spec
suite green// the pin is what makes the
// act-is-correct premise hold
score = randomInt(85, 100)
Pinned at 85, no test case ever carries a score between 80 and 84. That is the band where the code was wrong, and the suite reports green because it never goes there. Shipped value was 85.
The suite was not passing because the code was right. It was passing because it never once went there.
Read the full article
Graded for Obeying, on SubstackThe essay carries the defect this page is built on, the oldest rule in machine learning deployment that explains it, and why a vendor cannot know what acting wrongly costs you. Adjacent, and worth a look if you came for who checks the checker, is Who rates the reviewer?.
Get the next one
One essay a week on making data and AI systems trustworthy enough to act on.