Prompt Caching Not Working? Every Reason Your Cache Misses

Caching only pays when requests share an identical prefix. Almost every miss comes from something small that changes at the start of the prompt.

Quick answer

Your cache misses because the start of the prompt changes between requests: a timestamp, a user ID, reordered tools, or a document placed before the instructions.

The rule every provider shares

Caches match on the prompt prefix, token for token. One changed character near the start means everything after it is a miss.

The usual culprits

Timestamps or request IDs in the system prompt. Move them to the end of the user message.
User names or account details at the top. Put shared instructions first and per-user context last.
Tools in a different order on each call. Sort tool definitions the same way every time.
Retrieved documents before the instructions. Put static instructions first, then retrieved content.
Prompts too short to cache. Each provider has a minimum prompt length for caching.
The cache expired. Anthropic offers 5-minute and 1-hour caches; requests further apart than that miss.
Requests spread across providers or regions. Each holds its own cache, so routing that ignores cache affinity lowers hit rates.

Provider differences

OpenAI caches automatically and bills cached input at a fraction of the input price.
Anthropic uses explicit cache breakpoints. Writes cost 1.25x the base price for 5 minutes or 2x for an hour; reads cost a fraction, such as $0.10 per 1M tokens on Claude Sonnet 5.5.
Gemini has implicit caching and explicit context caches, which also charge for storage per hour.

How to check your hit rate

Every provider reports cached tokens in the response usage. Divide cached input tokens by total input tokens over a day of traffic. A support bot or agent with a long shared system prompt should be well above half.

Estimate the value with the prompt caching savings calculator, or read prompt caching explained.

Frequently asked questions

Why are my OpenAI cached tokens zero?

Usually because the start of the prompt changes on every request, or the prompt is shorter than the caching minimum.

Why isn't Anthropic prompt caching hitting?

Check that the cache breakpoint sits after everything static, that nothing before it changes, and that requests arrive within the cache lifetime.

Sources: OpenAI's API pricing page, Anthropic's pricing page, Google's Gemini API pricing page. Checked October 30, 2026.

Want us to find your 20%?

Book a free 15-minute API cost teardown. Bring last month's provider usage and we'll show you your biggest waste and the first fix we'd ship.

Get a Free 15-Minute API Cost Teardown