Tatsu Lab (Stanford)

license

Apache-2.0(OSI)

source

public

core features withheld

no

Fully Apache-2.0. The README header carries a 'Data License CC By NC 4.0' badge, but both badges link to the sibling tatsu-lab/alpaca_farm repository rather than to anything in this one, so the NC term is not a license this product makes you accept.

• https://api.github.com/repos/tatsu-lab/alpaca_eval/license recorded 2026-09-22

  GitHub's license endpoint reports spdx_id Apache-2.0 for tatsu-lab/alpaca_eval at path LICENSE, and returns the Apache-2.0 body. The repo's root listing carries no second license file.

• https://github.com/tatsu-lab/alpaca_eval recorded 2026-06-17

  Apache-2.0, ~2.0k stars

• https://raw.githubusercontent.com/tatsu-lab/alpaca_eval/main/README.md recorded 2026-09-22

  README installs the evaluator with `pip install alpaca-eval` or straight from the git repo and runs it against the reader's own model outputs, printing a leaderboard locally; the annotator configs and reference outputs ship in the package. The only external cost named is the reader's own OpenAI credits, roughly $10 a run. No paid tier, hosted edition or license key appears in it.

AlpacaEval's only public adoption signal is its GitHub star count, since no download or install figure is published for the tool. Star counts alone cap how high a reading can go, because a star is not a use.

• https://api.github.com/repos/tatsu-lab/alpaca_eval recorded 2026-08-12

  stargazers_count = 2,012 for tatsu-lab/alpaca_eval

AlpacaEval reports a length-controlled win rate that tracks Chatbot Arena rankings closely, with a public leaderboard and a toolkit for building and analyzing new automatic evaluators. It measures only judge-based instruction-following comparison, which is narrower than the multi-task harnesses it sits beside.

• https://github.com/tatsu-lab/alpaca_eval recorded 2026-06-17

  AlpacaEval 2.0 LC win-rate

• https://pypi.org/pypi/alpaca-eval/json recorded 2026-09-29

  The PyPI listing of the alpaca-eval package carries the project README, which says length-controlled win rates "are out and used by default! This increases the correlation with ChatBot Arena from 0.93 to 0.98" and lists a "Toolkit for building automatic evaluators" for building and analyzing new evaluators, alongside a leaderboard of common models.

• https://raw.githubusercontent.com/tatsu-lab/alpaca_eval/main/README.md recorded 2026-08-13

  'Length-controlled Win Rates are out and used by default', raising correlation with Chatbot Arena from 0.93 to 0.98; LLM-as-judge auto-annotator against a reference baseline, plus a public leaderboard and tooling for building and analyzing new automatic evaluators.

• https://tatsu-lab.github.io/alpaca_eval/ recorded 2026-09-29

  The leaderboard site is titled "AlpacaEval Leaderboard" and its caution line says "(LC) win rates alleviate length biases of GPT-4, but it may favor models finetuned on its outputs", above a ranked table of models.