Terminal-Bench
Laude Institute'/%3e%3cdefs%3e%3cpattern%20id='pattern0_1240_293949'%20patternUnits='userSpaceOnUse'%20patternTransform='matrix(11.2%200%200%2011.2%203.93617%200)'%20preserveAspectRatio='none'%20viewBox='0%200%2016%2016'%20width='1'%20height='1'%3e%3cg%20id='pattern0_1240_293949_inner'%3e%3crect%20width='16'%20height='16'%20fill='%23f88376'%20mask='url(%23hatch_gaps)'/%3e%3cmask%20id='hatch_gaps'%20maskUnits='userSpaceOnUse'%20x='0'%20y='0'%20width='16'%20height='16'%3e%3crect%20width='16'%20height='16'%20fill='white'/%3e%3crect%20x='6'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='6'%20y='8'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='4'%20y='2'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='4'%20y='10'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='2'%20y='4'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='2'%20y='12'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20y='6'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20y='14'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='14'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='14'%20y='8'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='12'%20y='2'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='12'%20y='10'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='10'%20y='4'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='10'%20y='12'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='8'%20y='6'%20width='2'%20height='2'%20fill='black'/%3e%3crect%20x='8'%20y='14'%20width='2'%20height='2'%20fill='black'/%3e%3c/mask%3e%3c/g%3e%3c/pattern%3e%3c/defs%3e%3c/svg%3e)
Benchmark of difficult, hand-vetted tasks that agents must complete in a real terminal, from compiling and debugging code to system administration. It runs as a continuous benchmark with tagged releases (2.0, 2.1, 3.0) mirrored on Hugging Face and executed through the Harbor framework, with an audited leaderboard at tbench.ai. Frontier agent builders use it to track progress on long-horizon terminal work.
The org renamed from laude-institute to harbor-framework and the original repo is now terminal-bench-1; the product's numbers are the official harborframework Hub mirrors, whose 2.0 card states it is a read-only mirror of the GitHub source. PyPI terminal-bench is the legacy harness, superseded by harbor, and is left undeclared.
Openness
5 high confidence- license
- Apache-2.0(repository LICENSE body read (terminal-bench-1)
- every version repo and all three Hub mirrors carry license
- apache-2.0)
- access
- open(ungated Hub mirrors ("gated":false) and public GitHub task sources)
- answers
- released(tasks include oracle solutions and tests, run and graded locally by Harbor
- datasheet
- present(Hub dataset cards on each release mirror plus tbench.ai docs and repository CONTRIBUTING/task rubric)
Terminal-Bench's tasks and harness are Apache-2.0, the Hub release mirrors are ungated and carry dataset cards, and oracle solutions ship in the repository. The governing release is recorded as the newest published mirror, 3.0, though every tagged release carries the same license, so that choice does not change the reading.
- https://huggingface.co/api/datasets/harborframework/terminal-bench-2.0 recorded 2026-10-07
Dataset API for harborframework/terminal-bench-2.0: `gated: false`, `private: false`, cardData.license `apache-2.0`, with the task directories (instructions, environments, solutions and tests) listed.
- https://huggingface.co/datasets/harborframework/terminal-bench-2.0/raw/main/README.md recorded 2026-10-07
card front matter license: apache-2.0, pretty_name Terminal-Bench 2.0, citation crediting The Terminal-Bench Team via github.com/laude-institute/terminal-bench; body states the dataset is a read-only mirror whose primary source is github.com/harbor-framework/terminal-bench-2 and points at the official tbench.ai leaderboard
- https://raw.githubusercontent.com/harbor-framework/terminal-bench-1/main/LICENSE recorded 2026-10-07
Apache License 2.0 full body
- https://raw.githubusercontent.com/harbor-framework/terminal-bench/main/README.md recorded 2026-10-07
README: "Terminal-Bench is a [continuous benchmark]", with "[tagged releases]" published on the Harbor Hub; the latest dataset lives on hub.harborframework.com, the harness is Harbor, and the README covers oracle-solution runs and the contributing and task-review docs.
Adoption
4 high confidenceAdoption is measured by Hugging Face downloads summed across the three official harborframework release mirrors: terminal-bench-2.0, 2.1 and 3.0. Third-party mirrors of the dataset are excluded from the count, and so is the legacy PyPI harness, which measures installs of the old evaluation tool rather than use of the dataset.
- https://huggingface.co/api/datasets?author=harborframework recorded 2026-09-01
the org's dataset list with 30-day downloads — terminal-bench-2.0 37,797, terminal-bench-2.1 14,616, terminal-bench-3.0 13,259 (plus leaderboard/results and lfs auxiliaries not counted)
- https://huggingface.co/api/datasets/harborframework/terminal-bench-2.0 recorded 2026-10-07
76,755 downloads in the trailing 30 days for harborframework/terminal-bench-2.0
- https://huggingface.co/api/datasets/harborframework/terminal-bench-2.1 recorded 2026-10-07
177,621 downloads in the trailing 30 days for harborframework/terminal-bench-2.1
- https://huggingface.co/api/datasets/harborframework/terminal-bench-3.0 recorded 2026-10-07
157,559 downloads in the trailing 30 days for harborframework/terminal-bench-3.0
- https://pypistats.org/api/packages/terminal-bench/recent recorded 2026-09-01
last_month 81,412 for the legacy pypi harness — recorded, not banded
Capability
not assessedA dataset is not 'capable', so capability is left unscored, mirroring the category pattern (gsm8k).
- https://raw.githubusercontent.com/harbor-framework/terminal-bench/main/README.md recorded 2026-10-07
a task collection and how to run it; no performance, throughput or feature claim for the capability axis to read
Verified 2026-10-07, except some scores, still awaiting re-confirmation