The Second Hallucination: your model vendor is a probabilistic system too
On June 12, Anthropic launched Fable 5 — the most capable model I had ever put to work. Three days later it was gone: a U.S. export-control order suspended access, Anthropic complied under protest, and every workflow anyone had built on that model stopped cold. The merits of the dispute — a claimed jailbreak, a company saying it was narrow, a government saying it wasn’t — are genuinely interesting and completely beside the point. What matters to an engineer is this: a dependency at the center of the workflow was revoked overnight, by a third party, for reasons that had nothing to do with anything I did.
I’ve written before about the first hallucination: a model hands you a confident, well-formatted, fabricated answer, and nothing in its tone warns you. The fix is a harness — deterministic checks the model can’t talk its way past. My whole system is built on that idea.
June taught me the failure mode has a big brother.
The second hallucination
The first hallucination is the model inventing facts about the world. The second is you inventing facts about the vendor — and holding them with the same unearned confidence:
- The company will keep existing.
- The model I depend on will keep being served.
- The price I planned around will keep being the price.
- I will keep being a permitted customer.
Every one of those is a probabilistic claim presented as fact. Confident, plausible, unverified — the exact shape of output my harness was built to reject, one level up where no harness was looking.
And the evidence that these beliefs deserve skepticism is public. Fable 5 shows the permission assumption failing. The solvency assumption is on file with the SEC: OpenAI filed confidentially for an IPO in May against a backdrop of multibillion-dollar annual losses, with profitability not projected until around 2030 and a funding gap that analysts measure in hundreds of billions. That’s not a prediction of failure — it’s a reminder that an API contract is, underneath, a bet on someone else’s balance sheet. The pricing assumption fails in the other direction: Chinese labs are serving frontier-adjacent models at a fraction of Western prices, which sounds like good news until you notice what a price war does to the vendors your architecture quietly assumed were permanent.
You cannot unit-test a vendor’s continuity
Here’s what makes the second hallucination nastier than the first: there is no deterministic check for it. I can verify a model’s claim that a file exists. I cannot verify a company’s claim that it will exist. No test suite covers “the Commerce Department changes its mind.”
When there’s no check, the only defense is independence — designing so the belief doesn’t have to be true:
- Own the weights that matter. The models doing my system’s bulk work run on GPUs I own. A file of weights on my own disk has no kill switch, no deprecation schedule, and no quarterly earnings call.
- Treat every model as replaceable. Everything routes through one chokepoint, and models are certified for roles empirically, by an evaluation battery — so when a model vanishes, requalifying a substitute is hours of compute, not a rewrite. My white paper calls this model-independence: the most capable tiers are the system’s architects and escalation reviewers, never its required operators.
- Rent capability; own continuity. I still use frontier APIs — they’re excellent, and for some jobs unmatched. But they get the work that can stop without stopping me.
The uncomfortable symmetry
We spent two years learning not to trust a model’s confident prose. The industry now needs the same lesson about its own dependency diagrams. Your vendor’s continuity is a sampled output from a distribution you don’t control — political, financial, competitive. Treat it the way you’d treat any other unverified claim from a probabilistic system: usable, useful, and never load-bearing without a fallback.
The model lies sometimes. The org chart above the model lies too — not out of malice, but because the future is out of everyone’s distribution. Build for the day the answer is wrong.
The architecture this essay comes from — the local farm, the crucible that certifies replacements, the routing chokepoint — is documented in the HMAS white paper.