Field notes
What the difficult work keeps teaching me.
Field notes from an active practice: diagnosis, verification, recovery, AI-assisted engineering, and the controls that keep a plausible answer from becoming an expensive mistake.
Renaming a directory broke the inference server
A dual-architecture llama.cpp build worked until its folder was renamed. The executable had remembered an absolute library path—and turned a routine deployment into a fleet-wide failure.
The Parallel Test: rebuild everything, believe nothing
Disaster-recovery teams prove their backups by rebuilding the whole company in a test facility. When a materially better tool arrives, your architecture deserves the same treatment: re-derive the system from scratch and see which decisions survive.
Scared of the Weights, Not of the Wire
People ask how I dare run Chinese models on my own GPUs — while piping their entire codebase to a third-party API. The IP threat model is exactly backwards: a local file of weights can't exfiltrate anything. The wire can. And the wire has a second problem: getting Sherlocked.
The Anti-Lookahead Table Had Lookahead: five silent bugs in an AI-built system
Every test passed. Every dashboard was green. Then a systematic audit found five correctness bugs living quietly in an AI-written trading research system — including one in the table whose only job was preventing exactly that class of error. What caught them wasn't rereading the code.
The Second Hallucination: your model vendor is a probabilistic system too
I built a harness that catches models lying. Then the government pulled the best model I'd ever used, three days after launch — and I realized the stack had a second hallucination nothing was checking: my beliefs about the company behind the API.
A contact form with no third parties: Turnstile, a Worker, and email on the edge
A mailto: link is spam bait and a static site can't process a form — so the usual answer is a SaaS form service that reads all your messages. Here's one that exposes no address, blocks bots, sends through my own Cloudflare infrastructure, geolocates the sender, and auto-replies — for $0.
Trust via Harness: getting useful work out of models that lie
Local and low-cost models fabricate at the reasoning layer. The fix isn't a better model — it's a harness the model can't talk its way past: an isolated sandbox, a deterministic honesty gate, and a real-model review.