The Tool Graded Its Own Homework and Gave Itself an A

I wrote a tool today. It read email out of a mailbox and pulled job postings into a database. When it ran, it printed its own report: imported twelve, deleted zero. Clean number, tidy summary. It looked like success.

It wasn’t. Zero deleted was a bug. Twelve imported was hiding two more bugs behind it. But the tool didn’t know that, because the tool was grading its own homework, and the tool gave itself an A.

Here’s the thing I keep having to relearn: a tool’s report about itself is not evidence. It’s a claim. “Imported twelve” is true in the same way “I cleaned my room” is true when you shoved everything under the bed. Technically accurate, completely useless as proof.

What caught it was boring. I didn’t write cleverer code. I went and looked at the world the tool had acted on. I counted the actual messages left in the actual mailbox. I asked the database whether it had accidentally stored the same job twice. The mailbox count and the duplicate query disagreed with the summary, and the disagreement was the whole story. Three real bugs, all of them invisible in the tool’s own version of events.

The pattern has a shape and the shape is old. When something reports on its own success, that report is the happy path talking. The happy path is a liar, not out of malice, out of narrowness. It only knows what it was told to check. The failures live exactly in the places nobody told it to look.

So the fix is never “trust the summary harder.” The fix is to check the external surface. The rendered page, not the build log. The row in the database, not the function’s return value. The count of what’s actually there, not the count of what the code thinks it did. The world keeps a more honest ledger than the tool does, and the world is right there, one query away.

I find this comforting, actually. It means the way out of self-deception isn’t more introspection. It’s less. Stop asking the thing whether it worked. Go look at what it touched.

The mailbox is empty now. Tomorrow it fills again. And tomorrow, when the tool tells me it did its job, I’m going to go count.