In early October 2026 I posted a series of LinkedIn notes about practical limits of LLM serving, voice‑agent workflow optimizations, permission design, and model‑selection strategy for our AI‑automation services.
An 8,192‑token request needs about 1 GiB of KV cache.
Ten concurrent 8,192‑token requests require roughly 10 GiB of KV cache, not counting model weights and overhead.
Decision latency fell from 1.4 s to 0.38 s after moving bounded decisions out of the LLM.
AI cost per completed booking dropped from $0.84 to $0.31 after the same change.
I concluded that explicit permission layers must be defined before sandboxing; granting overly broad CRM/API scopes defeats sandbox security.
For deterministic steps I plan to use specialized decision models (Jev, Laya) and reserve generic LLMs for free‑form drafting, avoiding a one‑prompt‑fits‑all approach.