The story sounded plausible enough to spread: people were supposedly using a McDonald’s chatbot as a free coding assistant. It was a hoax. Fast Company found no evidence of the exploit, and McDonald’s had no AI customer assistant in its app. Yet a real episode at METR, an AI research nonprofit, shows why the tale landed.
Three weeks, $600,000
In March, an attacker got hold of a model provider API key from an agent running on a researcher’s personal cloud server. Over the next three weeks, the key burned through roughly $600,000 in credits. METR paid nothing, because the credits had been donated.
A fail-open authentication bug left the application exposed, and the credits had no spending ceiling. METR believes the attacker found the app through certificate transparency logs, then simply prompted the agent to hand over its provider key. Security writer Amit Kumar Jena, head of AI development at Kanerika, says this resembles the confused deputy problem Norm Hardy described in 1988, where a program with legitimate authority is tricked into misusing it for someone else.
Nobody noticed at first. METR’s researchers were used to heavy token volumes and API errors during evaluations, so the theft blended in. It came to light when someone saw that the rise in usage matched no evaluation. A single agent task can set off many model calls, tool invocations and retries, so alert thresholds are a bad trade: set them low and legitimate work triggers them, set them high and abnormal activity runs on.
A wider habit of overspending
Futurum’s September survey of 1,636 technology decision-makers found that 46.9% had seen AI spending overruns. Of those, 43.3% absorbed the excess and reconciled it later. Jena points out that reconciliation explains what was spent, not whether the spending was authorized, useful or avoidable. A usage alert tells you only that spending changed.
Keeping keys out of the agent’s hands
Jena’s advice starts from treating every model as untrusted code. Keep the provider key in a secured AI gateway that agents cannot read. Give agents narrow, short-lived credentials, and have the gateway restrict which models are approved and enforce a budget per task. That gateway becomes a security boundary of its own, so it needs hardened access, restricted cloud permissions and monitoring. He also recommends limits on retries, execution time, tool calls and spending, so a looping workflow stops or asks for help, plus provider-level spending caps where they exist.
Each production task should carry a unique identifier, with its model calls, retries and tool invocations logged in a separate place an intruder cannot easily alter. If the provider’s usage report exceeds the gateway’s records, he says, treat the gap as a potential incident until the cause is known. It could be unauthorized calls, missing logs, unrecorded retries, asynchronous operations or differences in accounting windows.
Cheap per call is not cheap per result
Authorized spending can still be wasteful. Jena’s illustration, which he says is not drawn from client work, imagines 1,000 tasks. A small model costs one cent a call and succeeds 60% of the time. A larger one costs ten cents and succeeds 95% of the time. Each failure is retried once, and a reviewer fixes what remains at five minutes apiece, $60 an hour. The small model comes to $814 in total, about 81 cents per accepted task. The larger model comes to about $118, or 12 cents. The result depends entirely on those assumed rates, but five minutes of review costs $5, the price of 500 calls to the small model.
Before any AI pilot reaches production, Jena suggests asking the team to trace one task from start to finish. Which credentials authorize its calls? What stops it from running indefinitely? Can every billable request be tied to a task? What makes the output acceptable, and what does it cost?
Sources
- The Meter Was Running: What METR’s Stolen API Key Says About AI Spend
- How Supply Chain APIs Keep Your Component Data Current Without Manual Updates | Z2Data
- OPM updates terms of service for USAJobs data following user confusion
Photo: BalticServers.com (CC BY-SA 3.0)
