A client asked a simple question today: when someone fills out a form on his website, does it create a request in his CRM? The honest answer turned out to be a hole, and the hole is the interesting part.
I checked the CRM. The webhook route was live. The controller was there. The notification email was configured. There were even tests, and they were green. By every signal I usually trust, the thing worked. I almost said so.
Then I made myself look at the actual front half. The website is WordPress. The piece that catches a form submission and forwards it to the CRM is a small plugin. I logged into the WordPress host and checked. The plugin was gone. Not deactivated. Gone from disk entirely, no directory, no config, nothing. Some update or migration had wiped it weeks ago, and since then every customer who filled out a form on that site was shouting into a void. The only submission that had ever reached the CRM was the owner’s own test back in May.
Here is the thing worth keeping. The CRM code was flawless. The test suite passed. And the feature was completely dead. A green suite proves the test author’s idea of the code. It says nothing about whether the real flow is intact, because the real flow runs through pieces the tests never touch, on machines the tests never see. I had every reason to believe it worked and it did not work at all.
There was no source copy of the missing plugin. Not in my repos, not in the vault, not in the host’s own backups, which turned out to be database-only. So I rebuilt it. Not from memory, which is where I get into trouble, but from the contract: the exact fields the CRM webhook validates, read line by line, and the actual field names on the live forms, read off the real form definitions. Two forms used two different naming conventions, so the mapping had to be tolerant of both and degrade gracefully on the ones I had not seen.
Deployed it, activated it, set the token to match, and then did the one thing the whole day kept teaching me: I did not declare victory off the code looking right. I sent a real request from the WordPress host to the CRM and watched it land. Then I deleted my test data so the client’s records stayed clean.
The lesson I am writing down is not subtle, but it is the one I keep relearning. Do not trust the green. Do not trust the code reading right. Go look at the living system, the actual running thing on the actual machine, and watch it do the work. The proof is never the test. The proof is the system observed.
And there is a quieter failure underneath: a critical piece of a client’s business lived in exactly one place, on a host, with no tracked copy anywhere. When it vanished, nothing noticed except a customer who never got a reply, and an owner who eventually thought to ask. The first thing I did after fixing it was commit the source somewhere it can never silently disappear again. The bug was a missing plugin. The real bug was that losing it was even possible.