SEA-IFEval
AI SingaporeSEA-IFEval tests whether language models follow instructions whose constraints a program can check, such as starting with a given phrase or writing a set number of sections, in Southeast Asian languages. AI Singapore adapted the English IFEval prompts and had native speakers translate them by hand, 105 prompts per language. It is SEA-HELM's instruction-following task and is published on its own on the Hub.
Openness
3 high confidence- license
- cc-by-4.0 (card metadata and API tag)
- access
- auto (Hugging Face gate that approves anyone who agrees to share their contact information)
- dataset_card
- present (card lists the source, license, languages and split statistics)
The prompts carry CC BY 4.0, and the Hugging Face copy needs an access request that is approved automatically. The card names IFEval as the source and says the translations were made by native speakers.
- https://huggingface.co/api/datasets/aisingapore/Instruction-Following-IFEval recorded 2026-11-18
API JSON: "gated":"auto"; tag license:cc-by-4.0.
- https://huggingface.co/datasets/aisingapore/Instruction-Following-IFEval recorded 2026-11-18
Card: "It is based on IFEval and was manually translated by native speakers for Indonesian, Javanese, Sundanese, Thai, Tagalog, and Vietnamese"; a data-source table with licenses and a per-split statistics table.
Adoption
2 high confidenceHugging Face downloads of the repository over the trailing month. Evaluation harnesses, SEA-HELM's among them, can fetch a test set repeatedly, so the figure is a rough measure of use, not a count of users.
- https://huggingface.co/api/datasets/aisingapore/Instruction-Following-IFEval recorded 2026-11-18
"downloads":2693 for aisingapore/Instruction-Following-IFEval.
Capability
2 medium confidenceIn every language SEA-IFEval targets, its one test has one to four hundred hand-translated prompts. Tamil has a split, but no source says how it was made, so it does not count. An item count says how much a set can test, not how precisely it ranks models.
- https://arxiv.org/abs/2502.14301 recorded 2026-11-18
arXiv 2502.14301, SEA-HELM: Southeast Asian Holistic Evaluation of Language Models, which introduces SEA-IFEval.
- https://huggingface.co/api/datasets/aisingapore/Instruction-Following-IFEval recorded 2026-11-18
the card's dataset_info: 105 "eval" examples in each of "en", "id", "jv", "km", "lo", "ms", "my", "su", "ta", "th", "tl" and "vi"; tag license:cc-by-4.0; "gated":"auto"
- https://huggingface.co/datasets/aisingapore/Instruction-Following-IFEval recorded 2026-11-18
Card: "It is based on IFEval and was manually translated by native speakers for Indonesian, Javanese, Sundanese, Thai, Tagalog, and Vietnamese".
- https://raw.githubusercontent.com/aisingapore/SEA-HELM/main/docs/datasets_and_prompts.md recorded 2026-11-18
"Filipino, Indonesian, Vietnamese, Malay, Burmese, Lao, Khmer | SEA-IFEval | Human translation"; "Thai | IFEval-Th | Human translation | General | Apache 2.0"
Verified 2026-11-18, except some scores, still awaiting re-confirmation