Repeated recall, at a tenth of the price.
Opt in per request and WOS caches the search result under its query text, with the same prefix rules as LLM prompt caching. While the cache is warm, a repeated or extended query reuses the previous result, and the cached part is billed at 10% of the normal token rate.
One conversation, three turns
Here is what actually happens when an agent keeps talking to its memory. Each turn sends the conversation so far as the query, with cache_control on.
The full query is searched and cached: input at 2x (5-minute TTL).
Only Bob’s sentence is indexed and searched. The old part costs 0.1x, the new sentence 2x, and the cache now ends at it.
No engine call at all. Everything at 0.1x: the 90% discount.
The rates
| Operation | Token billing | What it means |
|---|---|---|
| Cache write - TTL 5 minutes | 2× | The first request. Its result is kept for 5 minutes, and every read slides the window forward. |
| Cache write - TTL 1 hour | 3× | The first request, kept for a full hour. |
| Cache read - hit or prefix hit | 0.1× | Every request after the write: the cached part costs a tenth of the normal token rate. |
What it saves
A concrete example: your agent sends a 3,000-token conversation as its query and repeats or continues it 10 times within five minutes. Without caching, that is 30,000 input tokens at full price. With a 5-minute cache it is 6,000 for the first write (2x) plus about 2,700 for the nine cached reads: 8,700 billed tokens, 71% less. The longer the conversation runs, the bigger the save.
The prefix rule
Matching is on the front of the query. If the front stays identical and new text is only appended, the cached part is reused and only the new part is searched. If anything before the end of the cached text changes, nothing can be reused.
cached [ A B C D E F G ] ○ [ A B C D E F G ] E ✗ [ B C D E F G ] E
○ hit - the front is unchanged, E is the only new part
✗ miss - the front changed, so the whole query is searched and cached again
Three rules to remember
- Extending re-caches through the new tail. After [A B C D E F G] + E, the cache now ends at E: the tail is billed once at the write rate, and the next turn can match all of A..E as its prefix again.
- One contiguous prefix per request. A query cannot be split into two cached segments; only its front can match.
- Writes invalidate instantly. Any store, store-turn, bulk-store, forget, supersede, or store deletion drops that store’s cache, so a cached answer can never be stale.
Turning it on
hits = mem.search(
"...the conversation so far...", user_id="alice",
cache_control={"ttl": "5m"}, # or "1h"
){ "memories": [ ... ],
"cache": { "status": "hit", // "write" | "hit" | "extend"
"ttl": "5m",
"cache_read_input_tokens": 412,
"cache_creation_input_tokens": 0 } }You do not need an SDK for any of this. Caching is one field on one HTTP call, so it works from every programming language. The curl tab is the universal recipe, and the Python, TypeScript, and Rust SDKs are convenience wrappers around the exact same call.