Back to Gap Map Model components / Language-specific evaluation sets

Lahaja

AI4Bharat
gated / Overall score: 1.0

Lahaja is a Hindi speech recognition benchmark of 12.5 hours of read and extempore speech from 132 speakers across 83 districts of India, built to test models on regional accents. Each clip carries verbatim and normalized transcripts with the speaker's native language, district and occupation. AI4Bharat built it and released four NeMo baseline models with it.

Openness

3 medium confidence
3.0
license
cc-by-4.0 (card License section)+mit (card metadata tag; conflicts with the License section)
access
auto (Hugging Face gate asks for contact information and approves automatically)
dataset_card
present (card and GitHub README describe the recordings and metadata fields)

Both licenses the card names allow commercial reuse with attribution, and the Hugging Face form approves requests automatically. The card should say which one governs.

Adoption

1 high confidence
1.0

Hugging Face downloads of the benchmark repository through the automatic gate. An evaluation set is downloaded once per evaluation setup, so downloads say little about how often scores on it are reported.

Capability

not assessed

Lahaja is a speech test, and the map has no agreed way yet to turn hours of recorded speech into a measure of test depth, so it shows no capability score for now. Its overall score reflects adoption alone.

  • https://arxiv.org/abs/2408.11440 recorded 2026-11-08

    "a total of 12.5 hours of Hindi audio, sourced from 132 speakers spanning 83 districts of India"

  • https://arxiv.org/pdf/2408.11440 recorded 2026-11-08

    "a total of 12.5 hours of Hindi audio, sourced from 132 speakers"; read speech, digital interactions with voice assistants and "Extempore conversations" recorded by the participants; "All transcripts are generated in the native Devanagari script of Hindi"; "the initial transcript (maker) is verified by a senior transcriber (checker)"

  • https://huggingface.co/datasets/ai4bharat/Lahaja recorded 2026-11-08

    usage example loads the benchmark with split= "test"; "a total of 12.5 hours of Hindi audio, sourced from 132 speakers spanning 83 districts of India"

Verified 2026-09-24