Prompt caching, with the arithmetic

Prompt caching is usually described as a discount. It is not. It is a premium on the write in exchange for a discount on the read, which means whether it saves you anything is a break-even calculation with a specific answer.

The calculation is short, and running it is the difference between cutting a bill by 85% and quietly raising it by 25%.

The published multipliers

On the Claude API, relative to that model's base input rate:

For Claude Sonnet 5 at $2/MTok base input, that is $2.50/MTok to write with the five-minute TTL, $4/MTok to write with the one-hour TTL, and $0.20/MTok to read.

Worked, on a realistic shape

Take a retrieval-augmented assistant with a stable 20,000-token prefix — system prompt, tool definitions, retrieved policy documents — followed by a short user turn.

No caching. 20,000 × $2 / 1,000,000 = $0.040 per request, every request.

Five-minute cache. The first request writes: 20,000 × $2.50 / 1,000,000 = $0.050. Every subsequent request inside the window reads: 20,000 × $0.20 / 1,000,000 = $0.004.

So for n requests that land inside one window:

uncached(n) = 0.040n
cached(n)   = 0.050 + 0.004(n − 1)

Setting them equal: 0.036n = 0.046, so n ≈ 1.28. One cache read is enough to be ahead. From there it compounds:

The ceiling is 90%, because a read costs a tenth of base input. Nobody reaches it, but a busy endpoint gets close.

The top row is the whole risk. One request per window costs 25% more than not caching at all — and that row is invisible on an invoice, because it looks like ordinary growth.

The failure mode is traffic shape, not configuration

The arithmetic above assumes requests arrive inside the window. The break-even is not really about n; it is about inter-arrival time versus TTL.

An endpoint receiving a request every 90 seconds keeps a five-minute cache warm indefinitely and lands in the bottom rows of that table. An endpoint receiving one every eight minutes never gets a hit: each request writes a fresh cache at 1.25×, and nothing ever reads it. That deployment is paying a 25% surcharge for a feature that is doing nothing, and every dashboard says caching is enabled.

Before turning it on, measure:

-- Median gap between consecutive requests, per feature.
-- If this exceeds your TTL, caching will cost you money.
select feature,
       percentile_cont(0.5) within group (order by gap) as median_gap_s,
       percentile_cont(0.9) within group (order by gap) as p90_gap_s
from (
  select feature,
         extract(epoch from at - lag(at) over (partition by feature order by at)) as gap
  from model_usage
  where environment = 'prod' and at > now() - interval '7 days'
) g
where gap is not null
group by 1;

If the median gap is comfortably under five minutes, use the 5m TTL. If it sits between five minutes and an hour, the 1h write at 2× needs two reads to break even — worth it for steady traffic, not worth it for bursty traffic with long gaps between bursts. If the median gap is over an hour, do not cache; fix something else.

Then verify with the response, which reports what actually happened:

const { usage } = await client.messages.create({ ... })

// The number that matters. If this is near zero while
// cache_creation_input_tokens is large, you are paying the premium
// and getting nothing back.
const hitRate =
  usage.cache_read_input_tokens /
  (usage.cache_read_input_tokens + usage.cache_creation_input_tokens)

Record both counts per call. A cache hit rate that falls off a cliff after a deploy is one of the more useful alerts you can have, for the reason in the next section.

Order the prompt so the cache can work

Caching is prefix-based. The cache matches from the beginning of the prompt up to the breakpoint, and it is exact — a single differing token anywhere in that prefix invalidates the whole thing.

Which makes this the standard way to destroy a cache without noticing:

// Every request has a different prefix. Hit rate: zero.
const system = `You are a support assistant.
Current time: ${new Date().toISOString()}
User: ${user.name} (${user.plan})
${POLICY_DOCUMENTS}`

A timestamp at the top of a system prompt means no two requests share a prefix. The fix is ordering, not removal — put everything stable first and everything variable after the breakpoint:

const messages = [
  {
    role: "system",
    content: [
      { type: "text", text: STATIC_INSTRUCTIONS },
      { type: "text", text: POLICY_DOCUMENTS, cache_control: { type: "ephemeral" } },
    ],
  },
  // Everything below here varies per request and is priced at base input.
  { role: "user", content: `Time: ${now}\nUser: ${user.name}\n\n${question}` },
]

Two related rules follow from the same property. Anything that changes on every deploy — a build hash, a version string — must not sit in the cached prefix, or every release resets your hit rate to zero. And growing conversation history should have its breakpoint moved as it grows, so the settled part of the transcript stays cached while only the new turns are charged at base input.

When it does not apply

Caching pays for repeated prefixes. Some workloads do not have one:

• Document extraction, one document per call. The document is the bulk of the input and it is different every time. Cache the instructions and schema if they are large, but the ceiling on savings is the fraction of input that is actually shared — often under 10%, at which point the saving is a rounding error.
• One-shot classification of short inputs. Nothing substantial to cache.
• Batch jobs with no latency requirement. The Batch API is a flat 50% off both input and output with no reuse requirement at all, and the two discounts stack — so for offline work, batch first, then cache.

The general shape: caching rewards a large stable prefix hit often. If your prompt is mostly variable, or your traffic is sparse, the honest answer is that this is not your cost problem, and finding out which feature actually moved the bill is the better use of the afternoon.

More in when the model bill triples and what an AI feature costs to build. Rates quoted are from Anthropic's published pricing, checked 27 August 2026.