I switched the yes/no answer pipeline to deepSeek 4.1 Flash together with Jev and a small reader model. The tweet I posted on 2026‑10‑21 calls this a “deadly combo” and notes it made the system >12× cheaper per answer while halving latency.
Backend LLM changed to deepSeek 4.1 Flash.
A small reader model is used for answer retrieval.
We switched our yes/no answer generation pipeline to the Jev model with a small reader, dramatically reducing cost and latency.
The new setup is 12x cheaper per answer.
Latency dropped from 13.4 s to 7.3 s.
$50 now reads 100K answers, not 8.3K.
I observed that most agent steps don't need the smartest model.
I concluded that they need the cheapest one that doesn't fumble tool calls.
I realized we have been overpaying for intelligence we never used.
To reduce costs while keeping reliability, I will adopt a model‑selection policy that picks the cheapest model whose tool‑call success rate meets a threshold (e.g., 99%). The orchestration layer will fallback to a higher‑tier model only when the success rate drops below that threshold.
if tool_call_success_rate(model) >= 99%:
use model
else:
fallback to next higher tier