I continued my September 2026 X thread series with a deeper dive into KV‑cache management, token‑cost reduction, positional encoding, and new model architectures. I highlight the open‑source september 2026 x posts part 1 LMCache service, the dramatic Claude‑Code token savings, the TwoTower diffusion‑LLM speedup, and practical embedding compression numbers.
LMCache uses CUDA IPC, a mechanism that lets separate processes access the same GPU memory.
The LMCache paper reports up to 15x higher throughput when combining it with vLLM across the evaluated workloads.
Claude Code used 3x fewer tokens with one change: Before: 10.4M tokens · 10 errors · $9.21 cost.
After: 3.7M tokens · 0 errors · $2.81 cost.
TwoTower fixes this by not forcing one network to do both. ↳ 2.42x higher generation throughput.
↳ Keeps 98.7% of the original model's quality.
Ten million 1,536-dimensional embeddings will occupy: 62 GB in float32, 15 GB in int8, 2 GB as packed bits.