In October 2026 I documented how various LLM harnesses implement prompt‑caching using the prefix‑based mechanisms offered by model providers.
Every model provider gives us ways to control prompt caching through prefix-based caching.
You can pass a breakpoint or marker to indicate how far into the prompt you want to cache.
As long as the prefix remains the same, inference for that portion of the prompt can be skipped.
Claude Code puts the static system prompt and tools first, then project memory, followed by session context, and only then the messages.
OpenCode caches the first 2 system messages and the last 2 non-system messages.
Pi marks the system prompt, the last user message, and the last tool.
The pattern is simple - static content first, volatile content last, and a stable key per session.
If a harness breaks that order even once, say by editing the system prompt mid‑session, you end up paying (cost and energy both) for the entire prefix again.
These observations suggest that arranging prompts from the most stable components to the most volatile maximizes cache reuse and reduces compute cost.
For more background on AI tooling trends in October 2026 see linkedin october 2026 ai tooling.