Home › Docs › Caching

Caching

Caching is usually the biggest single saving for apps that resend long prompts. apicuts uses two kinds: the providers' own prompt caching, and optional semantic caching.

Provider prompt caching

OpenAI, Anthropic and Google bill repeated prompt prefixes at a lower cached rate. apicuts helps you get that discount on every eligible request:

• Static first, dynamic last. We help order prompts so system prompts, tools and documents come before anything that changes per request.
• Cache-aware routing. Requests that share a prefix keep going to the provider holding that cache, so hit rates stay high.
• Measured, not assumed. Your dashboard shows cached tokens and what they saved.

Background: prompt caching, explained.

Semantic caching (optional)

For routes where many users ask near-identical questions, apicuts can answer a similar repeated request from cache instead of calling the model again.

• Off by default. You switch it on per route.
• Conservative matching. Only close matches are served from cache.
• Expiry. Cached responses expire, so answers stay fresh.
• Not for personal or time-sensitive answers. We keep it to routes where a shared answer is correct.

What caching means for your data

By default, prompts and outputs are never stored. Semantic caching is the one exception: on routes where you turn it on, cached responses are kept only until they expire. See security and data residency.