Home › Blog › LLM Cost Optimization: The Complete Guide
LLM Cost Optimization: The Complete Guide
Your LLM bill comes down to four things: how many tokens you send, how many you get back, which model handles them, and how often you pay for the same work twice. This guide covers every lever, in the order that usually pays off first.
Measure cost per request for each route first. Then take the free wins (prompt caching, batch APIs for work that can wait, trimming input and output), and only then the tradeoffs (smaller models, lower reasoning effort). Protect the result with a cost check after every deploy.
What drives the bill
- Input tokens: system prompts, tool definitions, chat history and retrieved documents, resent on every call.
- Output tokens: replies and reasoning, usually priced several times higher than input.
- Model tier: the same request can cost many times more on a frontier model than on a small one.
- Repeated work: retries, duplicate requests and agent loops that bill you again for the same answer.
Step 1: Measure before you change anything
Log the usage data from every response with the route name and model, then work out cost per request for each route. Use the token counter to size prompts and the LLM cost calculator to see what a change is worth at your volume.
Step 2: Take the free wins
These lower the bill without touching output quality:
- Prompt caching: keep the stable part of each prompt identical and at the start, so repeated input bills at the lower cached rate.
- Batch APIs: OpenAI, Anthropic and Google all offer batch processing at roughly half price for work that can wait up to a day.
- Input hygiene: trim chat history, retrieve fewer chunks, and only send the tools a route needs.
- Output hygiene: ask for a specific output shape, and drop instructions that produce text nobody reads.
Step 3: Then the tradeoffs
These can save more but can affect quality, so test each one against your evals:
- Model routing: send classification, extraction and other simple tasks to a smaller model.
- Reasoning effort: reasoning tokens bill as output, so use the lowest effort that still passes.
- Cost per completed task: a cheaper model that needs extra retries or turns can cost more overall.
Step 4: Keep it cut
A single prompt change can undo your savings. Cost regression protection compares cost per request after every deploy and flags the release that made it jump.
Where to start
| Lever | Best for | Quality risk |
|---|---|---|
| Prompt caching | Long prompts reused across requests | None |
| Batch APIs | Evals, backfills, nightly jobs | None (slower results) |
| Output limits | Verbose replies and JSON | Low |
| Model routing | Classification, extraction, tagging | Medium: test with evals |
| Lower reasoning effort | Routine chat and simple tasks | Medium: test with evals |
Provider-specific steps: OpenAI, Anthropic and Google Gemini.
Want us to find your 20%?
Book a free 15-minute API cost teardown. Bring last month's provider usage and we'll show you your biggest waste and the first fix we'd ship.
Get My Free 15-Minute API Cost TeardownFrequently asked questions
What is LLM cost optimization?
Lowering what you pay for large language model APIs without making answers worse: sending fewer tokens, paying less per token, and not paying twice for the same work.
Which lever usually saves the most?
It depends on the app. Apps that resend long system prompts or documents usually gain most from prompt caching; apps with lots of background work gain most from batch APIs.
How long does LLM cost optimization take?
Finding the waste is usually the slow part. apicuts targets one working day from kickoff to a tested pull request, and guarantees at least 20% lower cost per request.