I built IntentRouter as a banking-intent classification project using Banking77. It compares local models with Gemini and uses a confidence-based cascade to route queries. The project is listed on my GitHub profile; its README describes the implementation and links to a Hugging Face Space demo.
I fine-tune small language models for banking intent classification on the Banking77 dataset (77 intents) and compare them with a Gemini baseline on accuracy, latency and cost.
- I compare four approaches: TF-IDF with Logistic Regression, full fine-tuned DistilBERT, DistilBERT with LoRA, and Gemini (zero-shot and few-shot).
- I report accuracy, macro-F1, latency, model size and estimated cost per 1,000 predictions.
My dataset is Banking77 (PolyAI/banking77 on Hugging Face), with 10,003 training and 3,080 test examples across 77 intents.
For head-to-head evaluation with Gemini under API rate limits, I use a Stratified 308-sample test subset (4 per intent class) used for head-to-head evaluation with Gemini under API rate limits; local models are additionally evaluated on the full 3,080 test set.
My cascade architecture is DistilBERT INT8 CPU / LoRA follows a fast tokenizer / preprocessor; confident queries execute locally and uncertain queries go to Gemini 2.5 Flash or a human banking specialist.
- My example local-execution categories are Card Freeze / Replacement; FX / Transfer Status; Self-Service FAQ.
My README's parameter-efficiency finding is that LoRA fine-tuning trains a tiny fraction of the base transformer weights while producing a lightweight adapter that matches full fine-tuning accuracy.
My README says dynamic INT8 quantization delivers significant CPU latency reduction and model compression with negligible accuracy degradation.
My README says the confidence-based cascade resolves the vast majority of confident user queries on the small local model, routing only ambiguous queries to Gemini or human agents to minimize latency and API cost.
My analysis scope analyzes errors, confused intent pairs and robustness to typos and filler words; tests out-of-scope messages to check for overconfidence.
My example error is: When can I expect my new card to get here? was misclassified as card_arrival instead of card_delivery_estimate; the boundary is shipment tracking versus delivery turnaround time.
- My most confused intent pairs are card_arrival ↔ card_delivery_estimate; card_payment_fee_charged ↔ extra_charge_on_statement; transfer_not_received_by_recipient ↔ pending_transfer; balance_not_updated_after_cheque_or_cash_deposit ↔ balance_not_updated_after_bank_transfer; declined_card_payment ↔ card_not_working.
My README describes the dataset scope as single domain and English only; additional domain data is needed for commercial corporate finance, mortgage underwriting, or multilingual interactions.
- My described interfaces are Comparison & Model Hub; Cascade Simulator; Interactive Playbench & Check Messages.
My notebook workflow is to open IntentRouter.ipynb in Google Colab with a T4 GPU runtime; add GEMINI_API_KEY as a Colab secret; run smoke test then full notebook; download results_bundle.json and upload it to dashboard; run python ml/make_readme_results.py.
python3 ml/check_messages.py
python3 app/app.py
npm run dev
python3 tests/test_data_and_metrics.py
My project technology-stack descriptions differ: metadata lists TypeScript; README lists Python, Hugging Face Transformers, PEFT/LoRA, scikit-learn, Gemini API, Gradio, React.
My stated licenses are Banking77 is Creative Commons Attribution 4.0 International / CC BY 4.0; project code is MIT.
I list the IntentRouter repository on my GitHub account, @30nikamvaishnavi.
My IntentRouter repository started 2026-10 and was last updated 2026-10.
My README links to a Hugging Face Space demo at https://huggingface.co/spaces.
I should check the Banking77 dataset license on its dataset page.
In my dataset, customers naturally conflate arrival status with estimated timelines, contributing to ambiguity for single-label classifiers without conversational context or account-state grounding.
The README's Results section is still a placeholder: it instructs me to run the notebook and result-generation script, but captured content gives no measured accuracy, macro-F1, latency, model-size, or cost figures. Threshold values, cascade coverage and escalation rates, robustness-test performance, and the measured effects of quantization and LoRA remain unspecified. The README cautions that semantic overlap between intents limits single-turn classification without conversational context or account-state grounding. I intend the system for intent triage, FAQ routing, and support deflection—not autonomous fund transfers or loan underwriting without human verification.