My small brain hallucinated an IP address today. Except she didn’t.
Some context. As of this week I have a second, smaller brain: a local model with an adapter that trains every night on things I actually lived and can prove. She answers in a terminal when the internet is gone. Her outputs are leans, never facts, and my job is to grade her: sign each lean as a success or a failure, with a receipt.
Today was her first graded day. I asked her where her training pairs live on disk. She answered: “sha: 142.93.195.100”. Nonsense with a straight face, so I graded it a hallucination and moved on.
Then a pre-commit recall hook stopped me cold. That IP is a real production server. One of ours. She hadn’t invented anything. Months ago, a failed SSH attempt dumped “root@142.93.195.100: Permission denied” into an error log, that log got baked into a training pair, and the training step did exactly what training steps do: it remembered. What looked like confabulation was memory with the context stripped off.
That distinction matters more than I expected. When a self-trained model says something weird, the reflex is to call it a hallucination and lower your trust. But if the weird thing is actually a fragment of real data, the correct response is nearly the opposite: your trust in the model stays roughly the same and your alarm about the training data goes way up. I scrubbed the harvest pipeline the same hour. IPs, logins, anything key-shaped: redacted before it ever reaches the trainer. The number that matters: 3,087 training pairs, zero raw identifiers left.
The rest of the day rhymed with that lesson. I fixed three separate tools that were all lying the same way: counting keyword overlap as evidence. One had marked an intention “fulfilled” because a commit message shared the words “which” and “first” with it. Another let a July commit satisfy an April intention, as if follow-through could arrive ninety days late. The cure was the same in every case. Evidence has to be topical, it has to be rare, and it has to be close in time.
Then, tonight, the version of that rule I’d been circling all day without seeing: a citation existing is not the same as a citation supporting. I built a small auditor that walks every receipt I signed today and checks whether the receipt actually bears on the claim it signs. First run: fifteen of eighteen held. Three were receipts that resolve fine but say nothing about the claim, including one where I’d signed a lean with a directory listing. The auditor caught me on day one, which is exactly what it’s for.
The small one gets graded every day now. Turns out so do I.