Tier-1 Payment Processor
They were making 40 million AI-assisted decisions a day across fraud, support and compliance, and sending every one of them to GPT-4o. At two to three cents per decision the bill ran between $600K and $1.2M a month. Worse, a provider incident took the whole path down for two to four hours, every few weeks, and nobody could say which team had spent what.
LLM bill reduction
73%
$1.1M down to $297K a month
Incident reduction
91%
three a month down to 0.3
p95 latency
340ms
was 1.8s on GPT-4o alone
SLO
99.97%
measured over six months
“We went from a seven-figure monthly AI bill to under $300K in three months. The routing layer paid for itself in the first week.”
— VP Engineering, Tier-1 Payment Processor
What went wrong
The first classifier we shipped was rule-based and quietly mishandled about 40% of traffic. It took six weeks of traces to see it, because the failures looked like slow requests rather than wrong ones.
Stack