FAIR Forward / Datenlabor BMZ (GIZ)

license

MIT(OSI)

source

public(GitHub)

core features withheld

no

Languagebench is MIT-licensed, with the stock license text and no appended terms. Its evaluation pipeline, FastAPI backend and React frontend all ship in one repository that the Hugging Face Space builds from, the result tables publish as open datasets, and no pricing, paid tier or license key appears anywhere in it.

• https://api.github.com/repos/datenlabor-bmz/evals-for-every-language/license recorded 2026-10-06

  GitHub's license endpoint reports spdx_id MIT and returns the stock MIT text (Copyright (c) 2025 Datenlabor BMZ & GIZ & DFKI) with no appended terms

• https://raw.githubusercontent.com/datenlabor-bmz/evals-for-every-language/main/README.md recorded 2026-10-06

  README documents the whole product from the one repo — `uv run evals/main.py` to evaluate, `uv run evals/backend.py` plus the npm frontend to explore, and results/ committed in the tree — with no pricing, paid tier or license key

There is no installable package for languagebench: it is a hosted leaderboard and the pipeline behind it, so the closest measurable artifacts are the two Hugging Face result datasets the README names as the programmatic access path. Downloads on those datasets are low, and the repository's GitHub stars, forks and the Space's likes all point the same direction.

• https://api.github.com/repos/datenlabor-bmz/evals-for-every-language recorded 2026-09-10

  21 stars, 2 forks, last push 2026-07-28

• https://huggingface.co/api/datasets/fair-forward/evals-for-every-language-results recorded 2026-09-10

  downloads = 57 in the trailing 30 days on the aggregated results dataset

• https://huggingface.co/api/datasets/fair-forward/evals-for-every-language-results-detailed recorded 2026-09-10

  downloads = 45 in the trailing 30 days on the per-sample results dataset

• https://huggingface.co/api/spaces/fair-forward/languagebench recorded 2026-09-10

  likes = 26, runtime stage RUNNING, and four resolved domains including datenlabor-bmz-ai-language-monitor.hf.space and fair-forward-languagebench.hf.space

Languagebench covers more languages than any harness in the category, up to 200 against EuroEval's 30-plus, but its task surface is narrow: seven borrowed benchmarks run only against hosted models through a single router. EuroEval trades that language breadth for support across encoders, decoders and encoder-decoders and its own dataset-creation scripts, leaving the two roughly comparable, and general-purpose harnesses still cover more ground than either.

• https://arxiv.org/abs/2507.08538 recorded 2026-09-10

  "a comprehensive multilingual benchmark that systematically assesses LLM performance across up to 200 languages, with a particular focus on low-resource languages"; aggregates translation, question answering, math and reasoning over FLORES+, MMLU, GSM8K, TruthfulQA and ARC; "an open-source, auto-updating leaderboard and dashboard"; a global proficiency map and trends over time

• https://raw.githubusercontent.com/datenlabor-bmz/evals-for-every-language/main/notes/system-architecture-diagram.md recorded 2026-09-10

  seven task functions (translation_from/to with BLEU+ChrF, classification, mmlu, arc, truthfulqa, mgsm), a static curated list of ~42 models filtered to a default top 40 through OpenRouter, languages sorted by speaker count with a 1000-language default ceiling, 10 samples per model x language x task, and bootstrap confidence intervals on the aggregate