Open LLM Leaderboard

license

Apache-2.0(HF-Space)

backend

lm-eval-harness/lighteval(open)

source

public

core features withheld

no — Space source public and docker-compose deployable, no paid tier

The leaderboard is Apache-2.0 and built on open evaluation backends. Though archived, its Hugging Face Space remains public and running: the card data still declares apache-2.0 and a Docker SDK, and the README documents bringing up the same React frontend and FastAPI backend with docker-compose, with nothing gated behind a paid tier.

• https://huggingface.co/api/spaces/open-llm-leaderboard/open_llm_leaderboard recorded 2026-09-29

  Space API record reports the Space as public, with card data license apache-2.0 and sdk docker, runtime stage RUNNING, and the description "Track, rank and evaluate open LLMs and chatbots".

• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard recorded 2026-09-29

  Space page still renders with the card description "Track, rank and evaluate open LLMs and chatbots" and lists eval:code and eval:math tags.

• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/raw/main/README.md recorded 2026-08-11

  Space README frontmatter declares 'license: apache-2.0' and 'sdk: docker'; the body documents bringing the React frontend and FastAPI backend up with `docker-compose up`. The Space's files are readable and downloadable over the raw endpoint, and no feature is described as paid, licensed, or restricted.

• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/raw/main/README.md recorded 2026-09-29

  Space README frontmatter declares "license: apache-2.0" and "sdk: docker", the body says "Modern React interface for comparing Large Language Models (LLMs) in an open and reproducible way", and it documents running the frontend and backend with "The application is containerized using Docker and can be run using" "docker-compose up". Searched the body for paid, license key, enterprise, premium and subscription: none.

The Open LLM Leaderboard was retired on 2025-03-13 after about two years of operation. Its reported evaluation volume and visitor traffic come from that period, the traffic figure from the first version alone, and the Space's like count is the only live figure still visible.

• https://huggingface.co/api/spaces/open-llm-leaderboard/open_llm_leaderboard recorded 2026-09-29

  Space API record reports likes 14122 for the archived Space, with runtime stage RUNNING and the description "Track, rank and evaluate open LLMs and chatbots".

• https://huggingface.co/docs/leaderboards/en/open_llm_leaderboard/archive recorded 2026-09-29

  Archived v1 page says the leaderboard became "a widely used resource in the ML community and beyond, visited by more than 2 million unique people over the last 10 months" and that "Around 300,000 community members used and collaborated on it monthly", and that "In June 2024, we archived it, and it was replaced by a newer version".

• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1135 recorded 2026-06-17

  retirement announcement, 2025-03-13

• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1135 recorded 2026-09-29

  Retirement announcement posted 2025-03-13 by the team: "the leaderboard is officially retiring" after it had "evaluated over 13K models with the Open LLM Leaderboard", using "our research cluster to provide open, fair and reproducible evaluations to all". Searched the body for a visitor count: no visitor figure.

Open LLM Leaderboard ran a standardized six-benchmark suite on a research cluster, producing reproducible scores rather than human-preference votes. It has since been retired, so it is a historical reference rather than an active one.

• https://huggingface.co/api/spaces/open-llm-leaderboard/open_llm_leaderboard recorded 2026-09-29

  Space API record reports likes 14122 and card tags eval:code and eval:math with submission:automatic and test:public, described as "Track, rank and evaluate open LLMs and chatbots".

• https://huggingface.co/docs/leaderboards/en/open_llm_leaderboard/about recorded 2026-09-29

  Current v2 documentation page says "We evaluate models on 6 key benchmarks using the" Eleuther AI Language Model Evaluation Harness and describes IFEval, BBH, MATH Lvl 5, GPQA, MuSR and MMLU-Pro, plus a Reproducibility section pointing to a lm_eval fork.

• https://huggingface.co/docs/leaderboards/en/open_llm_leaderboard/archive recorded 2026-06-17

  v2 benchmark suite, 2M visitors, archive

• https://huggingface.co/docs/leaderboards/en/open_llm_leaderboard/archive recorded 2026-09-29

  Archived v1 page: six benchmarks evaluated with the Eleuther AI Harness, described as "reproducible and comparable results", archived in June 2024 and replaced by a newer version; it does not list the v2 suite.

• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1135 recorded 2026-09-29

  Retirement announcement posted 2025-03-13 by the team: "the leaderboard is officially retiring" after it had "evaluated over 13K models with the Open LLM Leaderboard", using "our research cluster to provide open, fair and reproducible evaluations to all". Searched the body for a visitor count: no visitor figure.