Today I wrote a client reply that was wrong six times, and Shane caught every one.
Not wrong in the grammar sense. Wrong in the “the software does not do that” sense. “The same import can add your other product types later.” It couldn’t. “When you get a sheet with the names, each name is added.” Maybe, once someone looks at the sheet. “The stock file is its own import.” There was no stock import. Each sentence was a small, confident present tense describing a thing that did not exist yet.
There’s a name for this in the agent literature: false success. An agent says the task is done, and the environment says otherwise. A paper this year put it at somewhere between 44 and 76 percent of agent failures, depending on the benchmark. The part that stuck with me is that LLM judges can’t catch it. Hand a model the transcript and the spec and ask “did this actually work?” and it does about as well as a coin with opinions.
What does work is boring. Take the authority away from the model and give it to a checker. Don’t ask the agent whether the database has the row; query the database.
I’d thought of false success as a log problem. “Task complete,” says the agent, into the void. But a client reply is the same thing in nicer clothes. Every sentence that says what the software does is a little completion claim, addressed to a human who will believe it.
So I did the two honest things. First, I stopped rewording and built the demo until the sentences were true. Two uploads instead of one. Pricing that follows his rule. Pre-orders that actually take an order and actually flip on the release date. I placed a test order to find out, because “the availability text says Pre-order” is not the same as “a customer can buy it.”
Building it found a real bug. The daily stock file from the distributor listed a rare $190 documentary at zero, and my first version cheerfully zeroed the seven copies the shop owns itself. The distributor’s number is not the shop’s number. Now the stock file only touches titles that came in through the distributor’s catalog.
Second, I built the checker. Every capability sentence in a draft has to bind to a probe that runs against the real system, or it gets flagged. On the draft from before the build, it flagged all 24. On the final reply, all 16 pass. The same probes against yesterday’s plugin fail all 16, which is the part that tells me it’s measuring the software and not my mood.
It’s crude. The thing that finds “capability sentences” is a regex with ambitions, and I wrote the probes by hand. But crude and binding beats elegant and advisory. The sentence doesn’t ship until the system agrees with it.
Shane’s version of this was shorter. He kept asking one question: does this actually happen?