You fire a prompt. The model responds. You move on. Somewhere in that loop you burned tokens you did not need: boilerplate context re-injected every call, a system prompt with no structure, or a request for reasoning you will never read. At ten calls a day that is noise. At a thousand calls a day, or inside a multi-agent loop, it is real money and measurable latency. Token discipline is the underrated craft of agentic development, and most of it is habit, not cleverness.
A note on numbers up front: the savings you get depend entirely on your workload, so instead of quoting round percentages, this post shows where the waste hides and gives one measured example you can reproduce.
Rule 1: your system prompt is an API
Treat your system prompt like a type signature. It should be precise, minimal, and versioned. If you are pasting three paragraphs of "you are a helpful assistant who..." into every call, you are paying to explain the model to itself.
Keep: a one-sentence role, output-format constraints, domain rules that override defaults, and any custom tool syntax. Strip: politeness scaffolding, redundant negations ("do not do X, Y, Z" when you can say "do A"), and examples that only repeat the format rule.
Rule 2: context is not a free buffer
The context window feels infinite until you are calling a model in a loop and your p95 latency spikes. Every character in context costs time at inference. A large context is not free; it is a dial you are turning. Common bloat:
- Passing the entire conversation history when you need the last few turns.
- Including full file contents when you need the lines around the cursor.
- Injecting every available tool when the task uses two.
Fix it by summarizing old turns, retrieving instead of injecting whole files, and trimming tool lists to what the task needs.
Rule 3: structured output removes post-processing tokens
If your model emits prose and you parse it with a regex, you pay twice: once for the prose, once for the extraction. Ask for structured output with a schema instead, so you get parseable results directly and skip the wrapper text.
// Instead of: "Here is the analysis: the sentiment is positive because..."
{
"sentiment": "positive",
"confidence": 0.87,
"reason": "forward-looking language"
}
The savings are real but workload-dependent, so measure yours rather than trusting a headline number.
Rule 4: measure a real before and after
Here is one reproducible example from our own runs, offered as an n=1, not a benchmark. We had a classification step that returned a paragraph of explanation plus the label, and we parsed the label out. Counting with the API's token usage field, the prose version averaged about 180 output tokens per call. Switching to a strict JSON schema that returned only the label and a one-line reason dropped it to about 45 output tokens per call. Same accuracy on our eval, roughly a quarter of the output tokens, and no regex. The point is not our number; it is that you should read the usage object and see your own.
Rule 5: cache your heavy, stable context
If a large block of context is stable across calls, cache it instead of resending it. With Anthropic's prompt caching, cache reads are billed at a fraction of base input tokens, so a long, unchanging system prompt or tool description stops being a per-call tax. Mark the stable block for caching and keep the volatile part outside it. We wrote the full mechanics up separately, so rather than repeat it, see prompt caching explained for the exact headers and gotchas.
The token budget mindset
Before you write a new prompt, ask three questions: what is the minimum context the model needs to do this correctly, what is the minimum output format that gives me what I need, and am I re-injecting something that could be cached or compressed? Token discipline compounds, and the model you route to matters as much as the prompt. If you want the routing side of this, we cover it in how to not waste tokens across models. Ship leaner, ship faster.