Back to Gap Map Model components / Evaluation code

WebArena

WebArena (Carnegie Mellon University)
open source / Overall score: 3.0

WebArena is a self-hostable web environment and benchmark for autonomous agents. It runs working websites for online shopping, a discussion forum, collaborative software development and content management, with a map and reference manuals, and checks whether an agent actually completed each of 812 long-horizon tasks. Researchers at Carnegie Mellon University built it; the repository is the canonical implementation for reproducing the paper, while ServiceNow's AgentLab carries newer infrastructure for running it.

Openness

5 high confidence
5.0
license
Apache-2.0 (OSI; the LICENSE body is the Apache 2.0 text)
source
public (the browser environment, agents, evaluation harness and Docker setup for the websites)
core features withheld
no — a university research benchmark with nothing sold beside it, and the tree has no enterprise path

The environment, the evaluation harness and the setup for hosting the websites are Apache-2.0 in one public repository. It is an academic benchmark with no paid edition.

Adoption

2 low confidence
2.0

WebArena ships no package or dataset with a download count of its own, so GitHub stars on its repository are the only signal. A star is not a use, and the agent frameworks that run WebArena tasks through their own tooling are not counted.

Capability

4 medium confidence
4.0

WebArena grades agents by what they actually change on working, self-hosted websites rather than by multiple choice. It sits a step below OSWorld, which runs the same kind of execution-based test across whole desktops and several operating systems, because WebArena stays inside the browser.

  • https://arxiv.org/pdf/2307.13854 recorded 2026-11-12

    Paper (ICLR 2024), Carnegie Mellon University: "fully functional websites from four common domains: e-commerce, social forum discussions, collaborative software development, and content management"; "we release a ready-to-use benchmark with 812 long-horizon web-based tasks"; evaluation of "the functional correctness of task completions".

  • https://raw.githubusercontent.com/web-arena-x/webarena/main/README.md recorded 2026-11-12

    README: Docker resources and "Amazon Machine Image that pre-installed all websites"; "After evaluating the 812 examples, reset the environment"; site variables for SHOPPING, REDDIT and GITLAB.

Verified 2026-11-12