Back to Gap Map Model components / Language-specific evaluation sets

SEA-IFEval

AI Singapore
gated / Overall score: 2.0

SEA-IFEval tests whether language models follow instructions whose constraints a program can check, such as starting with a given phrase or writing a set number of sections, in Southeast Asian languages. AI Singapore adapted the English IFEval prompts and had native speakers translate them by hand, 105 prompts per language. It is SEA-HELM's instruction-following task and is published on its own on the Hub.

Openness

3 high confidence
3.0
license
cc-by-4.0 (card metadata and API tag)
access
auto (Hugging Face gate that approves anyone who agrees to share their contact information)
dataset_card
present (card lists the source, license, languages and split statistics)

The prompts carry CC BY 4.0, and the Hugging Face copy needs an access request that is approved automatically. The card names IFEval as the source and says the translations were made by native speakers.

Adoption

2 high confidence
2.0

Hugging Face downloads of the repository over the trailing month. Evaluation harnesses, SEA-HELM's among them, can fetch a test set repeatedly, so the figure is a rough measure of use, not a count of users.

Capability

2 medium confidence
2.0

In every language SEA-IFEval targets, its one test has one to four hundred hand-translated prompts. Tamil has a split, but no source says how it was made, so it does not count. An item count says how much a set can test, not how precisely it ranks models.

Verified 2026-11-18, except some scores, still awaiting re-confirmation