A Web3 news feed published a benchmark table last week. The provenance is the story.
Terminal-Bench 4 β a suite that grades an autonomous agent on driving a shell, patching files, and recovering from its own execution errors β allegedly scored Anthropic's Sonnet 5.5 at 70.6 percent. Its immediate predecessor, Sonnet 5, reportedly scored 10.3 percent. Same benchmark family, same vendor, one generation apart. A 6.9x discontinuity.
The same table places Sonnet 5.5 above Anthropic's own flagship, Opus 5.5, at 66.4 percent.
No named author. No publication date. No model card. No link to a reproducible harness. The post's architecture β vendor claims, benchmark roll-up, pricing comparison β reads like a positioning document reformatted into news grammar.
I have spent eighteen years reading launch copy for a living, most of it into a ledger. The formatting changes by sector. The epistemology does not.
Anthropic's Sonnet tier exists to solve a commercial problem, not a research one: the flagship is too expensive to run continuously. The pitch here is that Sonnet 5.5 delivers flagship-adjacent coding and tool-use performance at two dollars per million input tokens and ten dollars per million output tokens β reportedly half the operating cost of Opus 5.5.
Three operational claims ride along with that price. Inference speed improves by more than thirty percent. Token consumption per completed task falls by roughly one third. Programming and terminal-operation scores lead OpenAI's GPT-6 Astra and Sol line.
If all three hold, the arithmetic compounds. A model that runs faster and burns fewer tokens per finished task does not merely cost less per token β it costs less per unit of delivered work. That is a categorically different claim from a price cut, and it is the only claim in the entire document that a buyer can verify without trusting anyone's word.
Everything else is soft tissue: connective narrative that converts a repricing event into a capability event. This pattern is not unique to AI. Hype is leverage in reverse β it borrows against future credibility to move present allocation.
The discontinuity is the tell.
Model iteration curves are smooth. That is not an aesthetic observation; it is what the optimization process produces. Parameter updates, data mixtures, and post-training objectives move adjacent metrics in adjacent increments. When a single number moves 6.9x across one generation while every neighboring metric moves in single digits, the correct default assumption is that the measurement apparatus changed, not the model.
Terminal-Bench 4 is versioned. Benchmarks rarely reach a fourth major revision without a reason β earlier versions saturate, task composition gets rebalanced, or grading rubrics tighten around partial credit. Any of those three breaks cross-generation comparability. The 10.3 percent figure could have been produced on a harness with a different task mix, a different strictness threshold, or a different failure taxonomy. Publishing 10.3 percent and 70.6 percent in the same table without stating that the two numbers were not generated by comparable instruments is not a lie. It is a presentation choice, and presentation choices are where audits start.
I ran this exact exercise on the 0x exchange protocol in 2018. The market was euphoric; the arithmetic was not. Six weeks of edge-case modeling surfaced an integer overflow in the settlement logic that the deployment schedule had no room for. The defect was real. So was the probability that nobody would have looked.
Then there is the third-party divergence.
Artificial Analysis reportedly places Sonnet 5.5 at 63.6 percent and 59.6 percent across two configurations, against GPT-6 Astra at 59.1 percent. A lead of roughly four and a half points at that scale is genuine but thin β it sits inside the band that prompt formatting, tool scaffolding, and retry policy can manufacture without touching the weights. The vendor's own figure of 70.6 percent sits seven points above the independent result. Seven points is not measurement noise. Seven points is pipeline.
GDPval-AA tells a different story. Sonnet 5.5 at 1844, Opus 5.5 at 1846. Two points. That is the noise floor of any instrument I have used, which means two things simultaneously: the mid-tier model has reached parity with the flagship on that axis, and the source's implied hierarchy β mid-tier beats flagship on terminal operations, ties on general evaluation, costs half as much β describes not a product lineup but a repricing event dressed as one.
Which raises the question the document never asks. If Sonnet 5.5 genuinely matches Opus 5.5 across the measurable surface at half the cost, what is the flagship's remaining function? Historically, the answer is either an undisclosed capability gap, a safety or alignment differential, or a contractual tier that exists for procurement rather than performance reasons. None of those appear in the source material.
The token figure is the only claim worth underwriting.
A one-third reduction in tokens per task, with nominal pricing held constant, is a 33 percent effective price cut that never appears on a rate card. It is the least dramatic number in the post and by far the most consequential. Vendors prefer efficiency to cuts because efficiency preserves the anchor price. I have watched this maneuver run for years in crypto: rollups routing fee reductions through compression and batching rather than repricing schedules, so headline gas figures stay legible while realized cost per transaction collapses underneath. Blob space created that arbitrage. Based on my own modeling of blob demand curves, it has a shelf life measured in quarters, not years, and the repricing that follows will be abrupt because the compression headroom is finite.
Token efficiency is the same structure. It buys market share without triggering a price war narrative. It also means the effective margin depends entirely on whether the efficiency is architectural β cheaper attention, better caching, speculative decoding, aggressive output compression β or merely editorial, achieved by teaching the model to emit shorter answers. Those two look identical on a benchmark and completely different on a balance sheet at scale.
What the document does not say is the more useful evidence.
No architecture. No parameter count. No context window. No multimodal scope. No function-calling contract. No training data provenance. No refusal rates. No red-team results. No prompt-injection posture.
That last omission is not academic for a model being marketed on terminal execution. An agent that drives a shell, reads files, and patches code has an attack surface that includes every untrusted string it ingests. I spent part of 2024 evaluating Chainlink's CCIP routing mechanism and found a reentrancy path in the new message-handling logic β a bridging primitive that was already handling institutional volume. The lesson generalizes. Capability expansion in an execution environment outpaces the security review budget attached to it, every time, because the security review does not produce a benchmark number.
A terminal-operating agent with no published injection threat model is an unpriced liability, and the post that markets it does not mention the category.
The distribution channel matters more than the claim.
A Web3 feed is not a technology publication. Its editorial incentive is attention arbitrage. AI model claims are the highest-velocity content available; crypto audiences are the highest-conviction audience available. Publishing one to the other costs nothing and captures the readership of both. No reporter stakes their name, no editor accepts liability, and the vendor's positioning language survives intact through two reshares. This is how a marketing document becomes a report without anyone deciding to lie.
The bulls are not wrong about the part that matters.
The efficiency claim is probably real. Anthropic has no durable incentive to fake it, because token consumption is metered inside every customer's own dashboard and surfaces within a week of API access. Benchmark inflation depreciates in days. Token burn appears on someone else's invoice. If the one-third reduction survives contact with a production workload, the mid-tier-substitutes-for-flagship argument is correct on economics regardless of whether 70.6 percent is reproducible.
And a benchmark being untrustworthy is not evidence of a bad model. My 0x work taught me the inverse lesson: the protocol's own materials overstated readiness, and the integer overflow was there anyway. A sloppy claim and a genuine defect coexist routinely. The Compound rate-model analysis I ran in 2020 was dismissed as theoretical until the Treasury drain executed precisely along the slippage tolerance I had published. Skepticism deployed as blanket dismissal is just contrarianism wearing the same costume as hype.
The Sonnet-Opus convergence may also be deliberate. If the revenue line is the mid-tier and the flagship is a marketing anchor, cannibalization is not a product-management failure. It is a hedge against a competitor pricing into the same band.
Reject the table. Keep the invoice.
The only self-verifying number in this entire source is cost per completed task, and that is readable from a metering dashboard, not a leaderboard. Everything else β 70.6 percent, 1844, outperforms β is a claim with an owner who benefits from it being believed.
Code is law, but capital is king. Benchmark claims settle arguments. Metered bills settle allocations.
If a model card, a versioned benchmark manifest, and an injection threat model appear, the claim set becomes auditable. Until then, what circulated last week was a repricing rumor published by a feed with no stake in being right β and in ninety days, the only figure from that table anyone will still be able to quote is the price.