IndicGenBench
Google ResearchIndicGenBench is a benchmark of generation tasks for large language models in 29 Indic languages across 13 scripts, covering cross-lingual summarization, machine translation, and multilingual and cross-lingual question answering. It extends CrossSum, FLORES, XQuAD and XorQA by having people translate their English examples into each target language, which makes the test data multi-way parallel. Google Research releases it as four task sets.
Openness
2 high confidence- license
- cc-by-sa-4.0 (Flores-IN and XQuAD-IN)+mit (XorQA-IN)+cc-by-nc-sa-4.0 (CrossSum-IN)
- access
- public (all four Hugging Face repositories gated: false; also in the GitHub repository)
- dataset_card
- present (GitHub README and per-task cards describe tasks, fields and splits)
Each task set keeps the license of the dataset it extends, so the summarization set is non-commercial while the translation and question-answering sets allow commercial use. All four download without a gate.
- https://huggingface.co/api/datasets/google/IndicGenBench_crosssum_in recorded 2026-09-24
"gated": false; license tag cc-by-nc-sa-4.0.
- https://huggingface.co/api/datasets/google/IndicGenBench_flores_in recorded 2026-09-24
"gated": false; license tag cc-by-sa-4.0.
- https://huggingface.co/api/datasets/google/IndicGenBench_xorqa_in recorded 2026-09-24
"gated": false; license tag mit.
- https://huggingface.co/api/datasets/google/IndicGenBench_xquad_in recorded 2026-09-24
"gated": false; license tag cc-by-sa-4.0.
- https://raw.githubusercontent.com/google-research-datasets/indic-gen-bench/main/LICENSE recorded 2026-09-24
"The different datasets distributed as part of this repository are subject to different licenses based on the license of the source dataset they were derived from."
- https://raw.githubusercontent.com/google-research-datasets/indic-gen-bench/main/README.md recorded 2026-09-24
"XQuAD-IN, Flores-IN: Under CC BY-SA 4.0 license ... XorQA-IN: Under MIT license ... CrossSum-IN: Under CC BY-NC-SA 4.0 license".
Adoption
2 high confidenceHugging Face downloads summed over the four task repositories. The same data is also cloned from GitHub, which publishes no count.
- https://huggingface.co/api/datasets/google/IndicGenBench_crosssum_in recorded 2026-09-24
711 downloads in the trailing 30 days for google/IndicGenBench_crosssum_in
- https://huggingface.co/api/datasets/google/IndicGenBench_flores_in recorded 2026-09-24
1093 downloads in the trailing 30 days for google/IndicGenBench_flores_in
- https://huggingface.co/api/datasets/google/IndicGenBench_xorqa_in recorded 2026-09-24
465 downloads in the trailing 30 days for google/IndicGenBench_xorqa_in
- https://huggingface.co/api/datasets/google/IndicGenBench_xquad_in recorded 2026-09-24
802 downloads in the trailing 30 days for google/IndicGenBench_xquad_in
Capability
3 medium confidenceIn the typical language IndicGenBench targets, its deepest test has four hundred to sixteen hundred items that count toward depth. In one of its 29 languages a test taken from another dataset counts for less, because nothing independent backs it there. An item count says how much a set can test, not how precisely it ranks models.
- https://arxiv.org/abs/2404.16816 recorded 2026-11-08
Abstract: "the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set 29 of Indic languages covering 13 scripts".
- https://arxiv.org/pdf/2404.16816 recorded 2026-11-08
CrossSum-IN: "We sample 700 English article-summary pairs (100 each from train/dev and 500 from test) and ask human translators to translate the English summary into the target Indic languages"; "CrossSum already contains data for 9 of our 29 target languages; for these languages we sample 100/100/500 examples from the original dataset". FLORES-IN: "where the same source English sentences are translated by humans into the target 200 languages"; "It contains data in 22 of our 29 target languages; we extend this by collecting human translations for the remaining 7 new languages"; "We collect translations for all 997 dev and 1012 devtest sentences, yielding 2009 sentences per language". XQuAD-IN: "we use the 240 passages and 1190 QA pairs from XQuAD as our test set"; "We additionally selected 20 passages and 100 QA pairs from the original SQuAD v1.1 training and development sets each to create our training and development set"; "For all the 280 passages and 1390 QA pairs we collect professional human translations in 12 Indic languages". XorQA-IN: "we select the 302 Bengali and 237 Telugu examples"; "dev set as our test data"; "to create our training (100) and development (500) set"; "We collect data for 28 Indic languages"; "We do not collect translations for Nepali." Table 1 gives 14.5k, 29.3k, 14.3k and 15.1k test instances; Table 9 gives each task's languages
- https://raw.githubusercontent.com/csebuetnlp/CrossSum/main/README.md recorded 2026-11-08
the language table's header row, "Language (ISO 639-1-Code) | am | ar | az | bn | my | zh-CN | zh-TW | en | fr | gu | ha | hi | ig | id | ja | rn | ko | ky | mr | np | om | ps | fa | pcm | pt | pa | ru | gd | sr | sr | si | so | es | sw | ta | te | th | ti | tr | uk | ur | uz | vi | cy | yo": of IndicGenBench's 29 languages, CrossSum has bn, gu, hi, mr, np, ps, pa, ta, te and ur
- https://raw.githubusercontent.com/google-research-datasets/indic-gen-bench/main/README.md recorded 2026-11-08
"the 'dev' and 'test' split of flores_in corresponds to the 'dev' and 'devtest' split of the original flores dataset"; "`split`: one of `{train, dev, test}`"
Verified 2026-09-24