Compar:IA
DINUM / beta.gouv.fr (French Government)Blind model-comparison arena created by the French government's betagouv and DINUM with the Ministry of Culture. A user types a prompt, two anonymous models reply, and the user votes; the identities and each model's estimated energy cost are revealed afterward. Its purpose is building large non-English human-preference datasets, and it deploys with Docker on a single server.
A second instance, AI-arenaen, runs in Denmark. Verified 2026-08-13 via the betagouv/ComparIA repository README and its LICENSE.
Openness
5 high confidence- license
- Apache-2.0(OSI
- source
- public
- core-gated
- ungated(publicly funded, README documents full Docker self-hosting)
Permissive OSI license with full public source code. The README bills the project as 'The open-source LLM arena. Deploy it for your organisation, sector, or language', with Docker self-hosting instructions for the whole platform and no paid tier to withhold anything for.
- https://raw.githubusercontent.com/betagouv/ComparIA/develop/LICENSE recorded 2026-06-22
LICENSE body 'Apache License, Version 2.0'
- https://github.com/betagouv/ComparIA recorded 2026-06-22
Apache-2.0 license badge, full source, betagouv ownership
- https://raw.githubusercontent.com/betagouv/ComparIA/develop/README.md recorded 2026-08-11
README states 'Self-host with Docker (single server, automatic HTTPS via Caddy)' and links devops/standalone_docker_install/DOCKER_INSTALL.md for the whole application. It also states 'compar:IA is funded by DINUM and the French Ministry of Culture, with European support from ALT-EDIC' and asks for funders - there is no commercial tier to withhold anything for.
- https://raw.githubusercontent.com/betagouv/ComparIA/develop/LICENSE recorded 2026-08-13
LICENSE body is 'Apache License, Version 2.0'.
- https://raw.githubusercontent.com/betagouv/ComparIA/develop/README.md recorded 2026-08-13
README leads 'The open-source LLM arena. Deploy it for your organisation, sector, or language', documents 'Self-host with Docker (single server, automatic HTTPS via Caddy)' for the whole platform, and says 'The whole platform is open source'. No commercial tier, license key or enterprise edition appears in it.
Adoption
3 high confidenceThe repository's 74 GitHub stars are beside the point here: compar:IA is a hosted French government LLM arena, and its traction is the data it has collected. The arXiv paper reports over 600,000 free-form prompts and over 250,000 preference votes as of 2026-02-07, about 89% of it in French. That collected data, rather than any repository or user count, is what the level rests on.
- https://arxiv.org/abs/2602.06669 recorded 2026-08-12
600,000+ free-form prompts and 250,000+ preference votes collected as of 2026-02-07, ~89% French
- https://github.com/betagouv/ComparIA recorded 2026-06-22
74 stars, 16 forks
Capability
4 high confidenceA broad arena feature set comparable to LMArena, plus environmental tracking and dataset generation: blind pairwise voting, a public leaderboard, open datasets published to Hugging Face, data.gouv.fr and Mozilla, Docker self-hosting, and EcoLogits maintained on the roadmap. It sits level with LMArena.
- https://github.com/betagouv/ComparIA recorded 2026-06-22
Feature list: model comparison, voting, EcoLogits, dataset publishing, self-hosting
- https://raw.githubusercontent.com/betagouv/ComparIA/develop/README.md recorded 2026-08-13
'compar:IA is a blind arena for large language models. You type a prompt, two anonymous models reply, and you vote for the answer you prefer'; teaching aims include 'the energy cost of generative AI'; the first public leaderboard shipped Nov 2025; datasets are published to Hugging Face, data.gouv.fr and the Mozilla Data Collective; a second instance, AI-arenaen, runs in Denmark; Docker self-hosting documented; an EcoLogits update is listed on the roadmap.
Verified 2026-08-12