I set up the Cost‑Aware Multi‑Model Router by cloning the repo, creating a Python virtual environment, and installing the backend dependencies. I then create a .env file containing the required API keys and default configuration values such as model names, per‑token pricing, and routing thresholds. The FastAPI service is launched with uvicorn api_server:app --host <redacted> --port 8000 --reload and the Next.js dashboard is started from the frontend‑next directory with npm install followed by npm run dev. This gives me a running OpenAPI UI at http://<redacted>:8000/docs and a UI at http://localhost:3000.
The default Gemini input price is 0.000001 per 1M tokens.
The default Gemini output price is 0.000005 per 1M tokens.
The default Mistral input price is 0.000003 per 1M tokens.
The default Mistral output price is 0.000015 per 1M tokens.
The confidence threshold is set to 0.7.
The complexity threshold is set to 0.7.
I store all application data in MongoDB and rely on Clerk for authentication. The FastAPI backend verifies Clerk JWTs with PyJWKClient, extracts a trusted clerk_user_id, and scopes every database query to that identifier.
The users collection has a unique index on clerk_user_id.
The connectors collection enforces a unique compound index on (clerk_user_id, provider).
The usage_events collection includes indexes on (clerk_user_id, created_at) and on request_id.
I built a modular Cost‑Aware Multi‑Model Router that intelligently routes LLM prompts to the cheapest capable model while preserving quality, and provides full observability, tool integration, and security.
The router defaults simple summarization and extraction tasks to Gemini 2.0 Flash to preserve budget without sacrificing output quality.
Baseline cost reference is "pure Mistral / high‑tier model execution" which provides a clear reference point to quantify savings.
Escalation is triggered when a low‑cost model response "fails validation or scores below configured thresholds".
I also integrated Dynamic MCP Tool Discovery, LangGraph multi‑step execution, Clerk‑based user‑scoped connectors, and LangSmith tracing to achieve end‑to‑end observability and a KPI dashboard showing request throughput, model usage breakdown, actual vs. baseline cost, net savings, confidence distribution, escalation frequency, and connector call counts.
I built the Cost‑Aware‑Multi‑Model‑Router repository on my GitHub (@Kathir18M). It is a Python project started and last updated in September 2026.
Cost‑Aware‑Multi‑Model‑Router is a repo on @Kathir18M's GitHub.
The primary language of the project is Python.
Intelligent Multi‑Model AI Routing Platform that dynamically selects the most cost‑efficient capable model, executes MCP‑powered tools, escalates when necessary, and provides real‑time cost, confidence, savings, and observability analytics.
Primary models used are Gemini 2.0 Flash and Mistral Small/Medium.
Dynamic routing was chosen to cut unnecessary spend on simple queries.
Savings are calculated as max(0, baseline - actual) ensuring non‑negative savings per request.
Up to 70 % of production LLM traffic consists of simple queries that do not need expensive frontier models.