An agent working a long task tends to ask the same question more than once — it re-reads a file it already read three steps ago, re-runs a search it already ran, re-fetches an API response it already has. Some of that repetition is genuinely necessary because the underlying state changed; a lot of it is waste, because nothing changed and the agent simply forgot, or a subagent independently needed the same thing its parent already fetched. Caching that second call instead of re-executing it saves tokens, time, and — for a paid external API — money. The failure mode isn't caching itself; it's caching a result under a key that doesn't capture everything the result depended on, so a later call gets an answer that was true once and is presented as though it's still true now.
Idempotent tool design is about making a tool safe to call twice with the same effect — a retry after a timeout shouldn't double-charge a card or create two rows (see the idempotent tool design guide in this pack). It's about the tool's side effects, not about skipping the second call. Caching is the opposite move: it's about not making the second call at all, because the first call's result is still valid. Cost budgeting is about deciding when a task is spending too much and should stop (see cost and token budgets for autonomous agents); caching is one tactic that reduces spend, but a budget with no caching and a cache with no budget solve different problems — a cache can be perfectly correct and an agent can still blow its budget on cache misses for a task that's genuinely large.
The most common caching bug in agent tools isn't expiry, it's an incomplete key. A tool that caches "get file contents" by file path alone will serve stale content after the file is edited, because the path didn't change but the content did — the key needed to include a modification time or content hash, not just the path. A tool that caches a search by query string alone will serve wrong results if the search is scoped to a working directory or branch that changed between calls — the key needed the scope, not just the query. The question to ask for every cache is not "what arguments did the caller pass" but "what, in the world, does this result actually depend on" — and every one of those things belongs in the key, even the ones that are annoying to compute.
| Cached call | Tempting (incomplete) key | What actually determines the result |
|---|---|---|
| Read a file | file path | path + mtime or content hash — the file can change without the path changing |
| Run a search across a repo | query string | query + branch/commit + working directory — the same query means different things in different states |
| Call an external pricing API | product ID | product ID + currency + as-of time — prices move, and a currency mismatch cached under the same key silently returns the wrong number |
| List directory contents | directory path | path + a directory-level version marker — a stale listing looks identical to a fresh one until something tries to open a file that was actually deleted |
A cache entry with a five-minute TTL is making a bet: that whatever it's caching won't change in five minutes. For some things that bet is nearly always right — a repo's license file, a config schema. For others it's frequently wrong — a build status, a queue depth, anything another agent or process might be actively writing to. TTL-only caching treats every cached thing as equally volatile, which means it's either too aggressive for the stable data (wasting cache hits it could safely have kept longer) or too permissive for the volatile data (serving staleness inside the TTL window with no way for the agent to notice). Where the underlying system can say when something last changed — a version number, an ETag, a last-modified timestamp — invalidating on that signal beats guessing a TTL, because it's tied to whether the data actually changed rather than how long ago the cache entry was written.
The safest place to invalidate a cache entry is at the moment the underlying state changes — the same tool call that writes a file should evict that file's cache entry as part of the write, rather than leaving the next reader to discover staleness on its own. Read-triggered invalidation (checking staleness only when something is read again) works only if every possible reader goes through the same freshness check every time, which is hard to guarantee once multiple tools, subagents, or processes can read the same cache. Write-triggered invalidation doesn't have that dependency — it doesn't matter who reads next, because nothing stale is left behind for them to find.
A cache hit and a cache miss should not be indistinguishable in the response, even though the actual data returned might be identical — because the one time staleness matters, the agent needs a way to notice. A response that includes {"result": ..., "cache_status": "hit", "cached_at": "..."} lets an agent that's about to make a decision with real consequences — deploying, spending money, telling a user something is true — decide whether a cached answer from ten minutes ago is good enough or whether this particular call is worth the cost of a fresh one. A cache that hides its own existence takes that judgment call away from the agent and makes it silently, every time, on the tool author's behalf.
The test to apply: pick a tool result an agent might reuse across steps, and list every real-world fact that has to stay unchanged for the cached copy to still be correct. If any of those facts aren't part of the cache key and aren't what triggers invalidation, the cache isn't wrong yet — it's just waiting for the one call where that fact changes and nothing notices.
A cache that's wrong in a way nobody can retry their way out of is a narrower version of the same problem idempotent tool design solves for side effects — see that guide for the write side of this coin.
Caching decides whether a call happens at all. A related but separate question is whether several calls that are all going to happen anyway can safely happen at the same time — see parallel tool calls for agents.