Kalahi
AI SingaporeKalahi tests whether language models answer the way a Filipino would in everyday cultural situations. It holds 150 prompts written by native Filipino speakers, each with responses a Filipino would find relevant or irrelevant, across topics such as family, food, beliefs and social customs. AI Singapore built it with native speakers, published it with its own paper, and runs a multiple-choice version inside SEA-HELM.
Openness
5 high confidence- license
- cc-by-4.0 (card metadata and API tag)
- access
- public (ungated on the Hub)
- dataset_card
- present (card describes the prompts, their topics and how they were written, and links the paper)
Kalahi downloads without a gate under CC BY 4.0, and its card and paper say who wrote the prompts and how. SEA-HELM's multiple-choice copy of the same prompts sits behind an automatic gate and is recorded on SEA-HELM.
- https://huggingface.co/api/datasets/aisingapore/kalahi recorded 2026-11-18
API JSON: "gated":false; tag license:cc-by-4.0.
- https://huggingface.co/datasets/aisingapore/kalahi/raw/main/README.md recorded 2026-11-18
Card metadata "license: cc-by-4.0", "language: - tl"; body: "Kalahi is composed of 150 situationally-enriched prompts and culturally relevant and irrelevant responses"; "It was collaboratively created by native Filipino speakers".
Adoption
1 high confidenceHugging Face downloads of Kalahi's own repository over the trailing month. SEA-HELM declares the same repository among its task repositories, so these downloads also count toward SEA-HELM's figure, and the runs that read SEA-HELM's multiple-choice copy count only there.
- https://huggingface.co/api/datasets/aisingapore/kalahi recorded 2026-11-18
"downloads":44 for aisingapore/kalahi.
Capability
2 medium confidenceIn its one target language, Kalahi's test has one to four hundred items that count toward depth, all written by native speakers. An item count says how much a set can test, not how precisely it ranks models.
- https://arxiv.org/abs/2409.15380 recorded 2026-11-18
Abstract: "We introduce Kalahi, a cultural LLM evaluation suite collaboratively created by native Filipino speakers. It is composed of 150 high-quality, handcrafted and nuanced prompts".
- https://huggingface.co/datasets/aisingapore/kalahi/raw/main/README.md recorded 2026-11-18
"Kalahi is composed of 150 situationally-enriched prompts"; the topic table sums to 150.
Verified 2026-11-18, except some scores, still awaiting re-confirmation