I once shipped an agent that read a 40K-token system prompt on every single request. Hundreds of requests an hour, same prompt every time, billed at full input price each time. The fix was four lines. Prompt caching took that recurring cost down to roughly a tenth. Cache reads cost about 0.1× the base input price. That's the headline number, and it's real.
But here's what bit me first: I added the cache marker and nothing changed. cache_read_input_tokens stayed at zero. The cache was silently doing nothing, and I was paying a premium for the privilege. So before the how-to, you need the one rule that everything else follows from.
Prompt caching is a prefix match
That's it. The cache key is the exact bytes of your rendered prompt up to each cache breakpoint. Any change anywhere in that prefix invalidates the cache for everything after it. One byte. A reordered JSON key, a timestamp, a swapped tool — any of those and you're writing a fresh cache entry instead of reading the old one.
The render order is fixed: tools, then system, then messages. So whatever sits earliest has to be the most stable thing you've got, and whatever changes per-request has to live at the very end. Get that ordering right and most caching works for free. Get it wrong and no amount of markers will save you.
The thing that was killing my cache
It was this, at the top of my system prompt:
Current date: 2026-06-21 14:32:07
A timestamp. It changed every request, it sat near the front of the prefix, and it invalidated everything downstream of it. The entire 40K prompt was uncacheable because of a 19-character string I didn't think about.
This is the single most common caching bug I see. People interpolate dynamic stuff — current date, user name, session ID, a mode flag — into the system prompt, and it poisons the prefix. The fix is to freeze the system prompt and inject the dynamic bits later, in the messages array, where they invalidate nothing before them. A fact at turn 5 doesn't touch the cache for turns 1 through 4.
Here's my audit checklist when a cache won't hit. Grep your prompt-building code for:
datetime.now()/Date.now()anywhere in the system prompt or toolsuuid4()or request IDs early in the contentjson.dumps(d)withoutsort_keys=True— Python dict iteration order can shift the bytes- f-strings interpolating a user or session ID into the system prompt
- a tool set that's built per-user, so the tools block differs every request
If cache_read_input_tokens is zero across two requests you know share a prefix, one of those is the culprit. Diff the rendered bytes between two requests and you'll find it.
Placing the breakpoint
The actual marker is cache_control: {type: "ephemeral"} on a content block. Simplest case, a big shared system prompt:
"system": [{
"type": "text",
"text": "<your big stable prompt>",
"cache_control": {"type": "ephemeral"}
}]
Because tools render before system, a marker on the last system block caches tools and system together. You get four breakpoints max per request, so spend them wisely.
For multi-turn conversations, put the breakpoint on the last block of the most recent turn. Each new request reuses the whole prior conversation as a cached prefix, and hits accrue as the chat grows. For a shared preamble with a varying question — few-shot examples plus a different query each time — put the marker at the end of the shared part, never at the end of the whole prompt. If you cache through the varying question, every request writes a unique entry and reads nothing.
The economics, because they're not free
Reads are cheap (~0.1×) but writes cost more than a normal request: 1.25× for the default 5-minute TTL, 2× for the 1-hour TTL. So caching only pays off if you read more than you write. With the 5-minute TTL you break even at two requests. With the 1-hour TTL you need at least three, because the write premium is doubled.
Which TTL? Use 1-hour only when your traffic has gaps longer than five minutes. If requests arrive more often than that, they keep the cache warm on their own and the default 5-minute TTL is fine and cheaper. Don't reach for the 1-hour TTL by default — the doubled write cost will quietly eat your savings if traffic is steady.
Two gotchas that cost me a debugging afternoon each
The 20-block lookback. Each breakpoint walks backward at most 20 content blocks to find a prior cache entry. In an agentic loop with lots of tool_use/tool_result pairs, a single turn can add more than 20 blocks — and then the next request can't find the previous cache and silently misses. Fix: drop an intermediate breakpoint every ~15 blocks in long turns.
Concurrent requests. A cache entry only becomes readable after the first response starts streaming. Fire ten parallel requests with the same prefix and all ten pay full price, because none can read what the others are still writing. For fan-out, send one request, await the first streamed token, then fire the rest. They'll read the cache that first one wrote.
One newer trick worth knowing
On Opus 4.8 there's a clean way to inject a mid-conversation instruction without nuking your cache: append a {"role": "system", ...} message to the messages array instead of editing the top-level system. Editing top-level system changes the prefix ahead of your whole history, so every cached turn re-processes uncached. A system-role message sits after the history and leaves the cached prefix intact. It's also the non-spoofable operator channel, which is a nice bonus.
Verify everything with the usage object: cache_creation_input_tokens is what you wrote, cache_read_input_tokens is what you read at the cheap rate, input_tokens is the uncached remainder. If your agent ran for an hour and input_tokens reads 4K, don't panic — the rest came from cache. Check the sum, not the single field.
Four lines of config, one rule to respect. Respect the prefix and the bill follows.