Home › Blog › Cost Regression Protection for LLM Apps

Cost Regression Protection for LLM Apps

By Victor, founder of apicuts

You cut your API bill, then a few releases later it's back. Cost regressions rarely come from one big change. They come from small ones nobody priced.

Quick answer

A cost regression is a release that makes each request more expensive. Catch it by tracking cost per request for every route after each deploy, alerting on jumps, and checking that your cache hit rate holds.

What causes cost regressions

A prompt grows: new instructions, more examples, a bigger tool list.
Something dynamic lands at the top of a cached prompt, like a timestamp or user ID, and caching silently stops working.
A route moves to a bigger model or a higher effort or reasoning setting.
An output limit gets raised or removed.
A retry bug or agent loop runs more steps than before.
Retrieval starts returning more chunks per question.
A new feature ships on a frontier model with no cost budget.

How to catch them

Set a baseline. Record the cost per request for each route while things are healthy.
Tag every call. Log the route, model and release version with each response's usage data.
Compare after each deploy. Check the first hours of traffic on a new release against the baseline, and alert when a route jumps past your threshold.
Watch the cache separately. If cached input tokens drop toward zero after a deploy, something at the start of a prompt changed.
Check prompts in CI. Count the tokens in your prompt templates on every pull request and fail the build when one grows past its budget.

What to do when an alert fires

Find the route that jumped, then diff its prompt, model, effort and output settings against the last good release. Check the cached token share. Roll back if the fix isn't obvious, and fix forward once you know the cause.

How apicuts handles it

Cost Regression Protection is part of working with apicuts. We watch cost per request after every deploy. If a release makes it jump, we flag it and fix it before it hits your invoice. Start by measuring your cost per request.

Want us to find your 20%?

Book a free 15-minute API cost teardown. Bring last month's provider usage and we'll show you your biggest waste and the first fix we'd ship.

Get My Free 15-Minute API Cost Teardown

Frequently asked questions

How big a jump should trigger an alert?

Start around 10 to 20% on a route's cost per request and tune from there. Set a minimum request volume so quiet routes don't create noise.

Can I catch regressions before deploying?

Partly. Prompt token counts can be checked in CI, but caching and agent behavior only show up with real traffic, so watch the first hours after each release too.

Is this the same as a spending limit?

No. A spending limit stops the bill at a number. Regression protection tells you which release made each request more expensive, so you can fix the cause.