Home › Blog › How to Cut Your OpenAI API Costs

How to Cut Your OpenAI API Costs

By Victor, founder of apicuts

Most OpenAI bills carry waste: prompts that never hit the cache, background jobs on the real-time API, and big models doing small jobs. Here's how to find it and cut it.

1. Find out which calls cost the most

Your bill is the sum of every place your code calls the API. Log the usage object from each response together with a feature name, then add it up by feature. Spend tends to concentrate in a handful of call sites, and that's where every fix below pays off first.

2. Let prompt caching do its job

OpenAI caches the start of repeated prompts automatically once a prompt reaches 1,024 tokens, and bills those cached input tokens at a discount that depends on the model. The cache only matches an identical prefix, so the order of your prompt decides whether you get the discount.

• Put stable content first: system prompt, tool definitions, examples, reference docs.
• Put changing content last: the user's message, timestamps, request IDs.
• Check cached_tokens in the usage details. If it stays at zero on repeated calls, something near the top of your prompt changes on every request.

3. Move work that can wait to the Batch API

Evals, backfills, nightly summaries and data enrichment rarely need an answer in seconds. The Batch API runs them asynchronously within a 24-hour window at half the normal price. Keep anything a user is waiting on on the standard API.

4. Send simple tasks to a smaller model

Classification, extraction, tagging and routing rarely need a frontier model. Run the same requests through a smaller model, compare the results on your evals, and switch the routes where quality holds. Keep hard reasoning on the larger model.

5. Cap output and reasoning

Output tokens cost several times more than input tokens. Set a maximum output length per route, ask for structured output instead of long prose, and drop instructions that make the model explain itself when nobody reads the explanation. On reasoning models, reasoning tokens are billed as output, so use the lowest reasoning effort that still passes your evals.

6. Stop paying for the same answer twice

• Retry with exponential backoff, and only on errors that can succeed on a retry.
• Deduplicate identical requests that fire twice from your front end or a queue.
• Store answers to questions your users ask over and over, and reuse them.
• Put a hard limit on the number of steps an agent loop can take.

7. Send less context

Resending a full chat history on every turn, retrieving twenty chunks when five would do, and attaching tool definitions a route never uses all add input tokens. Summarize or trim old turns, tune your retrieval, and pass only the tools each route needs. For embeddings, hash your documents and skip the unchanged ones instead of re-embedding everything on a schedule.

Measure the result

Track cost per request for each route before and after each change, not just the monthly total. Your traffic will keep growing, and the total bill hides whether a fix worked. To estimate what a change is worth, try the LLM cost calculator.

Frequently asked questions

Related