loading
loading
Tokens are the currency of agentic systems. Every prompt is a spending decision. Treat them with the same discipline you'd treat money in a production system — because at scale, they are money.
Context windows are large. Pricing per token is low. It feels like tokens are free.
They're not. At 100 calls per day they're nearly free. At 100,000 calls per day, a 20% reduction in average token use is the difference between a sustainable product and one that hemorrhages margin. And token count directly affects latency — more tokens in and out means slower responses, which means worse user experience.
Token discipline is fiscal discipline for AI systems.
It means every token in your system has a reason to be there. It means asking, for every component of your context: does this information increase the probability of a correct, useful output enough to justify its presence?
If the answer is "probably" or "maybe" or "it can't hurt", that's not good enough. Cut it.
System prompt: This is your highest-value token real estate. Every sentence should be load-bearing. Run the deletion test: remove a sentence. Does the model's behavior change? If not, cut it.
Conversation history: Most agent loops don't need the full history. They need the current task, the most recent 2-3 relevant exchanges, and a compressed summary of what's been decided. Keeping 20 turns of full conversation in context when the model only needs 3 is a common, expensive mistake.
Tool results: Your database query returned 200 rows. Your agent needs 5 of them. Filtering at the tool level — before the results enter the context — saves far more tokens than any prompt compression trick.
Any information that doesn't need to be verbatim can be compressed.
A 3-turn conversation summary takes 50 tokens instead of 500. A "relevant rows only" database result takes 100 tokens instead of 2,000. A structured JSON output takes 80 tokens instead of a prose paragraph at 180.
Compression at each stage compounds. A 30% reduction in system prompt tokens + 40% reduction in history + 50% reduction in tool results = an agent that's 40% cheaper and faster, with no loss in output quality.
The most disciplined builders set explicit token budgets before writing a prompt. "This agent gets 2,000 tokens of system prompt, 1,500 tokens of history, and tool results capped at 800 tokens per call." The budget forces tradeoffs, and tradeoffs produce better design decisions than unconstrained abundance.
When you hit the budget limit, you have to choose what matters. That choice is clarity. The agent with a tight budget and clear priorities outperforms the agent with an unlimited buffer and vague context.
"I'll add this context just in case" is the enemy. Just-in-case context is unfocused, expensive, and often distracting. Models with more context don't necessarily perform better — they can "lose" critical information in a large, undifferentiated window.
Give the model what it needs for this task. Not everything you have. Not everything that might be relevant. What it needs, for this task, right now.
Before you ship a new prompt:
Token discipline is not a one-time optimization. It's a continuous practice. The builders who internalize it ship faster, pay less, and build agents that scale.
Spend deliberately.
You've read all the doctrine. Now go build.