the expensive way to find out nobody was measuring

in late july 2026, tom’s hardware reported on internal Amazon AI-usage metrics first surfaced by the financial times. the headline case: a Claude Sonnet deployment matching author details to book listings, a task the reporting calls menial, that ran $1.8M in costs against a budget it exceeded by 860%. the overrun reportedly sat undetected for about five months. the same reporting lists smaller siblings: roughly $541K on a financial auditing tool build and $134K on a delivery-time logistics system. it also notes Amazon dropped an internal AI-usage leaderboard after it began rewarding engineers for maximizing token counts rather than outcomes.

every number in the paragraph above belongs to the journalists and their sources. nothing here is ours to verify, and none of it is a claim about how Amazon runs today. the sources line at the foot carries the links.

read as a measurement story, the case fails at four separate joints. each one has a boring, known counter.

admission without a pilot. an expensive model was admitted to a cheap task with no calibrated read on cost against capability. the counter is a pilot with receipts: score a small stated sample, fix the operating point, and let the numbers say whether the tool fits the task before the task scales. gaige ships this shape as its admit verb: a stated pilot, a vetted reference, and a refusal when the data cannot support the ask.

scale without an outcome gate. the deployment reportedly failed at the task and burned the budget. those are two separate meltdowns, and the first is the cheaper one to catch: an outcome gate against a vetted reference before corpus scale. pass, or no scale.

drift without a clock. 860% over budget is not a subtle signal, and it reportedly took months to surface. a registered series with per-interval bounds turns that class of surprise into a dated alarm inside days. that is what drift practice on this site is for: not budgets, but any measured number that is supposed to stay put.

a leaderboard measuring the wrong thing. usage went up because usage was what got measured. metrics are instruments too: point one at tokens and you buy tokens; point one at outcome-per-cost and you buy outcomes. the leaderboard was reportedly dropped, which is its own kind of receipt.

gaige does not monitor invoices, and it did not catch this case; nothing on this page says otherwise. gaige as shipped today calibrates detectors and watches instrument drift, with receipts. the discipline it implements, calibrated pilots before admission, outcome gates before scale, drift series with dated intervals, metrics that measure outcomes, is the counter to this failure class. that discipline is older than this project and bigger than it. the project’s contribution is making the receipts cheap, reproducible, and honest about refusal.

so why does this page live on a small site like this one? because measurement is the part of AI deployment nobody wants to pay for, and we run a bench that does almost nothing else. when fuel reaches this bench it turns into wider corpora, more instruments under calibration, and drift series that run on longer clocks. every one of those lands as a receipt you can re-run yourself. that is the trade: you fuel the bench, the measurement stays honest and public, and nobody has to take our word for anything. the support page says exactly what a coffee’s worth becomes. the roadmap shows the queue it moves.