Back to Gap Map Model components / Language-specific evaluation sets

Kalahi

AI Singapore
open / Overall score: 1.7

Kalahi tests whether language models answer the way a Filipino would in everyday cultural situations. It holds 150 prompts written by native Filipino speakers, each with responses a Filipino would find relevant or irrelevant, across topics such as family, food, beliefs and social customs. AI Singapore built it with native speakers, published it with its own paper, and runs a multiple-choice version inside SEA-HELM.

Openness

5 high confidence
5.0
license
cc-by-4.0 (card metadata and API tag)
access
public (ungated on the Hub)
dataset_card
present (card describes the prompts, their topics and how they were written, and links the paper)

Kalahi downloads without a gate under CC BY 4.0, and its card and paper say who wrote the prompts and how. SEA-HELM's multiple-choice copy of the same prompts sits behind an automatic gate and is recorded on SEA-HELM.

Adoption

1 high confidence
1.0

Hugging Face downloads of Kalahi's own repository over the trailing month. SEA-HELM declares the same repository among its task repositories, so these downloads also count toward SEA-HELM's figure, and the runs that read SEA-HELM's multiple-choice copy count only there.

Capability

2 medium confidence
2.0

In its one target language, Kalahi's test has one to four hundred items that count toward depth, all written by native speakers. An item count says how much a set can test, not how precisely it ranks models.

Verified 2026-11-18, except some scores, still awaiting re-confirmation