I created the VahanAI_Assignment repository on my GitHub (@Eshaj20) as a reproducible benchmark kit for evaluating ASR systems on Bangalore locality names spoken in Hindi, Hinglish, or Kannada.
The repository name is VahanAI_Assignment.
It is written in Python.
Development started in 2026-05 and the last update was in 2026-05.
The final dataset comprises 40 total recordings.
It includes 20 quiet indoor clips, 10 traffic/outdoor clips, and 10 phone‑call/compressed clips.
The benchmark baseline model is Deepgram.
Comparator 1 is the faster‑whisper small model.
Comparator 2 is the faster‑whisper large‑v3 model.
Benchmark runs generate output directories results/deepgram_40/, results/fw_small_40/, and results/fw_large_40/.
I chose Deepgram as a production‑grade baseline and the two faster‑whisper variants to cover a quality‑speed spectrum, while fixing the dataset to 40 recordings to balance coverage and manageability.