Back to Gap Map Organization

WebArena (Carnegie Mellon University)

lab · United States

Scores

1 product on the map — 1 open.

WebArena

Openness

5 high confidence
5.0
license
Apache-2.0 (OSI; the LICENSE body is the Apache 2.0 text)
source
public (the browser environment, agents, evaluation harness and Docker setup for the websites)
core features withheld
no — a university research benchmark with nothing sold beside it, and the tree has no enterprise path

The environment, the evaluation harness and the setup for hosting the websites are Apache-2.0 in one public repository. It is an academic benchmark with no paid edition.

Adoption

2 low confidence
2.0

WebArena ships no package or dataset with a download count of its own, so GitHub stars on its repository are the only signal. A star is not a use, and the agent frameworks that run WebArena tasks through their own tooling are not counted.

Capability

4 medium confidence
4.0

WebArena grades agents by what they actually change on working, self-hosted websites rather than by multiple choice. It sits a step below OSWorld, which runs the same kind of execution-based test across whole desktops and several operating systems, because WebArena stays inside the browser.

  • https://arxiv.org/pdf/2307.13854 recorded 2026-11-12

    Paper (ICLR 2024), Carnegie Mellon University: "fully functional websites from four common domains: e-commerce, social forum discussions, collaborative software development, and content management"; "we release a ready-to-use benchmark with 812 long-horizon web-based tasks"; evaluation of "the functional correctness of task completions".

  • https://raw.githubusercontent.com/web-arena-x/webarena/main/README.md recorded 2026-11-12

    README: Docker resources and "Amazon Machine Image that pre-installed all websites"; "After evaluating the 812 examples, reset the environment"; site variables for SHOPPING, REDDIT and GITLAB.