Null Is Not a Value: Verification, Vacuity, and the Empty-Shell Failure Mode

CryptoCobie
Guide

In the spring of 2026 I watched a machine lie by telling the exact truth, and it took me three days to understand why it bothered me so much.

The setup was ordinary. A two-stage research pipeline: stage one decomposes a source article into structured facts — title, claims, entities, a list of information points; stage two takes those facts and runs a nine-dimension protocol analysis under an explicit evidence standard we call COLD-EXEC. Standard stuff. Decomposition is cheap and parallelizable, so you fan it out across a queue; analysis is expensive and needs a clean substrate, so you gate it behind a schema. Every "AI research agent" shipping this quarter has some version of this architecture, and most of them are proud of it.

Stage one returned this: every field present, every field empty. "Not provided." "Not assessed." The information-point list, empty. A schema-shaped zero.

And stage two — the analysis agent — refused. Not with a crash. With an argument. Feed an empty template into an analysis engine and the only possible output is an empty template. If I start filling in "technical positioning: N/A," "TVL: N/A," "team background: N/A" and walk the nine dimensions, that is not analysis. That is the shape of analysis, forged. The agent then named its own state — BLOCKED, interrupt condition b, critical input missing and not bypassable — and asked for the source text.

I have been in this industry long enough to know that a system refusing to fabricate is not the interesting part. The interesting part is that stage one produced something that looked like a completed analysis and was, in fact, null. And that this is not an edge case. It is the default behavior of nearly every pipeline I have audited in the last eighteen months.

Why the split exists — and why it fails

The two-stage split exists for good reasons. Decomposition is a different kind of work from judgment: it is closer to parsing, it parallelizes cleanly, and it produces a cacheable artifact you can reuse across many downstream questions. The interface between the stages has to be a contract, and in 2026 the only contract we know how to write between two models is a schema. So we write one. Title, source, claims, entities, information points, each with a type.

And here is where the design quietly betrays itself. A schema can describe presence. It cannot describe substance. title: str is satisfied by title: "Not provided". information_points: list[str] is satisfied by information_points: []. The type checker runs green. The orchestrator logs success. The job moves to the next stage carrying a payload that contains exactly zero bits of the thing it was supposed to contain, wrapped in a structure that says everything is fine.

This is the failure mode I now call the empty shell: a payload that is structurally complete and semantically void, where the structure is what every downstream system checks.

I want to be precise about why this is worse than a crash. A crash is information. It propagates at the speed of the failure, it costs you a retry, and — crucially — it costs you the retry at the point of origin, where the missing input still exists and can still be supplied. The empty shell propagates silently and then costs you everything at the far end, where the context is gone. In our case the cost was one refused job. In a production agent economy, where stage two is not a cautious analyst but a trading bot or a collateral oracle or an autonomous procurement agent, the cost is whatever the vacuous output is allowed to authorize.

I have a specific memory that keeps surfacing here, and I'll get to it, but first let me show you the mechanics, because the mechanics are where the blockchain people need to pay attention.

The type signature was lying, and it lied in a way we taught it to

The deepest version of this problem is not about prompts. It is about types.

Consider two ways to represent "I have no source text." The honest one is a null or an absence: Option

= None. In Rust or Haskell or Swift, the compiler forces every consumer to handle that case, because the type system refuses to let you pretend a None is an Article. The dishonest one is what we actually shipped: a fully populated Article whose fields happen to contain the string "N/A."

That string is a catastrophe in three parts. It is truthy in most languages, so if article: passes. It is a valid str, so the schema validator passes. And it is human-readable, which means when a human finally looks at the logs they read "Not provided" and conclude the system behaved correctly — because it did, in the only sense the system can measure.

The fix at the smallest scale is embarrassingly simple: