Using semantic caching, token spend can drop by up to 70 % when the similarity threshold allows cached responses to be served.
With Redis Flex tiering, storage cost can be reduced by up to 80 % per gigabyte compared with an all‑RAM deployment.
These numbers reinforce my decision to adopt semantic caching, TTL‑based agent memory, and Flex tiering as the core cost‑optimisation stack for LLM‑driven agents.
In mid‑October 2026 I posted a LinkedIn thread on how Redis can tackle the three biggest cost drivers in LLM‑driven systems – caching, agent memory, and storage tiering – and shared the concrete performance and cost numbers I observed.
- Semantic caching can make cache hits up to 15× faster.
I decided to adopt three Redis mechanisms: 1. Semantic caching – embed incoming prompts, search by meaning, and return the stored response when similarity exceeds a threshold. 2. Agent memory – keep live session state in RAM with a TTL and asynchronously extract facts into a long‑term vector store. 3. Flex tiering – configure the RAM‑to‑SSD ratio per workload (e.g., 10 % for low‑latency hot data, 50 % for heavier loads) so that cold keys migrate to NVMe automatically. These choices let me cut both latency and spend without rewriting application code.
Open questions remain around how much of our production traffic is truly repeat queries, what similarity threshold balances hit‑rate and answer freshness, and the optimal RAM‑SSD split for different traffic patterns.
For more background see Redis’s cost‑optimisation guide: https://fandf.co/47dMC3t.
This post expands the October 2026 series documented in the summary page.
- Token spend can drop up to 70 % when semantic caching is active.
- Flex storage can reduce per‑gigabyte cost up to 80 % versus an all‑RAM deployment.