Chatbot Arena

platform

closed(hosted arena.ai, proprietary live leaderboard + private vote data)

source

partial(FastChat is a real arena implementation but stale relative to the operating platform

reference-code

FastChat Apache-2.0 (powers Chatbot Arena but stale, last release Feb 2024, 39.5k stars)

aux-OSS

Arena-Hard-Auto Apache-2.0 (automatic LLM eval, highest correlation to LMArena)

Scored as the hosted platform: the live evaluation product, its vote stream and the current ranking pipeline are all closed, while a substantial open-source heritage (FastChat) and an open automatic-eval companion (Arena-Hard-Auto) sit alongside them. That open code is stale relative to the platform actually running - FastChat's newest published release is still v0.2.36 - so what is public is a real arena implementation, just not the one operating. This is not an open core with functionality withheld for a paid tier; it is an open periphery around a runtime that never ships, which makes the source only partial and the product source-available rather than closed. The platform's canonical domain is now arena.ai, with lmarena.ai redirecting to it, and the ranking pipeline behind it still does not ship.

• https://github.com/lm-sys/FastChat recorded 2026-06-04

  Apache-2.0; 'FastChat powers Chatbot Arena (lmarena.ai), serving over 10M chat requests for 70+ LLMs'; latest release v0.2.36 Feb 2024

• https://github.com/lmarena/arena-hard-auto recorded 2026-06-04

  Apache-2.0 automatic LLM eval tool, highest correlation to LMArena

• https://arena.ai/ recorded 2026-06-04

  hosted 'Official AI Ranking & LLM Leaderboard' with Battle Mode; conversations disclosed to providers (proprietary platform)

• https://raw.githubusercontent.com/lm-sys/FastChat/main/README.md recorded 2026-08-13

  README still opens 'FastChat powers Chatbot Arena (lmarena.ai), serving over 10 million chat requests for 70+ LLMs' and documents launching an arena instance locally. Apache-2.0. The repo's newest release is v0.2.36, published 2024-02-11, so the published arena implementation is more than two years behind the operating platform: a real implementation, not the running one.

• https://arena.ai/ recorded 2026-08-13

  Still the hosted ranking and battle platform, and now the canonical domain - lmarena.ai 301-redirects here. It offers no repository, no self-hostable ranking pipeline and no license to the evaluation code, and states that conversations are disclosed to the relevant AI providers.

The FastChat README, the primary source here, states that Chatbot Arena has served 'over 10 million chat requests for 70+ LLMs', and Arena Elo is the most-cited human-preference leaderboard in frontier model releases, appearing across vendor launch posts. That 10M chat-request figure is what the band rests on, placed conservatively in the 1M-10M range on request volume; cumulatively it could be higher. It and the 1.5M human-vote figure beside it in the README have not been updated since they were first published, so the headline is a frozen number - another reason to stay conservative - while the platform itself still operates at arena.ai.

• https://github.com/lm-sys/FastChat recorded 2026-06-04

  'serving over 10 million chat requests for 70+ LLMs'

• https://arena.ai/ recorded 2026-06-04

  hosted live leaderboard / battle platform in active operation June 2026

• https://raw.githubusercontent.com/lm-sys/FastChat/main/README.md recorded 2026-08-13

  'FastChat powers Chatbot Arena (lmarena.ai), serving over 10 million chat requests for 70+ LLMs' and 'Chatbot Arena has collected over 1.5M human votes from side-by-side LLM battles'. 39.5k stars on the repo.

• https://arena.ai/ recorded 2026-08-13

  hosted live leaderboard and battle platform in active operation; no standalone user or traffic figure is published on it.

The reference platform for human-preference evaluation and the de-facto Elo standard, cited everywhere. Not a 5, because as evaluation code it is largely closed and hosted, and narrower than execution-based harnesses on agentic and task coverage. Judged on the breadth of its evaluation methodology: the hosted arena runs blind pairwise battles, and Arena-Hard-Auto v2.0 ships 500 hard queries plus 250 creative-writing ones with GPT-4.1 and Gemini-2.5 judges.

• https://github.com/lmarena/arena-hard-auto recorded 2026-06-04

  500 SWE/math queries + 250 creative prompts, GPT-4.1/Gemini-2.5 judges, highest correlation to LMArena

• https://arena.ai/ recorded 2026-06-04

  battle-mode pairwise comparison across many models

• https://raw.githubusercontent.com/lmarena/arena-hard-auto/main/README.md recorded 2026-08-13

  'V2.0 contains 500 fresh, challenging real-world user queries ... and 250 creative writing queries sourced from Chatbot Arena. We employs automatic judges, GPT-4.1 and Gemini-2.5'; the README also claims the 'highest correlation and separability to LMArena ... among popular open-ended LLM benchmarks'.