OSWorld

license

Apache-2.0(OSI)

source

public(github.com/xlang-ai/OSWorld)

core features withheld

no — full harness runs locally on VMware, VirtualBox or Docker, no paid tier

The harness and its virtual-machine environment are public under Apache-2.0, and the bundled 369-task set is openly redistributable as well.

• https://api.github.com/repos/xlang-ai/OSWorld/license recorded 2026-08-13

  GitHub's license endpoint reports spdx_id Apache-2.0 for xlang-ai/OSWorld and returns the Apache-2.0 text as the LICENSE body.

• https://github.com/xlang-ai/OSWorld recorded 2026-06-04

  Apache-2.0 license, public source, computer-use agent eval harness

• https://os-world.github.io/ recorded 2026-09-29

  The project site says "369 computer tasks involving real web and desktop apps in open domains" and notes 8 Google Drive tasks can be excluded ("361 tasks") due to network dependencies.

• https://pypi.org/pypi/desktop-env/json recorded 2026-09-29

  The PyPI record for desktop-env, the harness environment package, carries the classifier "License :: OSI Approved :: Apache Software License" and its README shows the Apache-2.0 badge.

• https://pypi.org/pypi/desktop-env/json recorded 2026-09-29

  The PyPI README announces "We released our [paper](https://arxiv.org/abs/2404.07972), [environment and benchmark](https://github.com/xlang-ai/OSWorld)" and lets users install the environment with "pip install desktop-env".

• https://pypi.org/pypi/desktop-env/json recorded 2026-09-29

  Searched the README for paid tier or license key: none. Local setup is described for VMware, with "features such as parallelism and macOS on Apple chips might not be well-supported" under VirtualBox and a Docker option "for hosting virtual machines".

• https://raw.githubusercontent.com/xlang-ai/OSWorld/main/README.md recorded 2026-08-11

  README carries the Apache-2.0 badge and gives full setup instructions for running the benchmark locally under VMware, VirtualBox, or Docker with KVM. AWS and Azure appear only as optional providers for parallel runs. No paid tier, commercial edition, or license key is mentioned.

• https://raw.githubusercontent.com/xlang-ai/OSWorld/main/README.md recorded 2026-08-13

  Apache-2.0 badge, full local setup under VMware, VirtualBox or Docker with KVM, AWS and Azure optional for parallel runs, and no paid tier, commercial edition or license key.

OSWorld scores appear across frontier computer-use launches through 2026, from OpenAI and Anthropic among others, which is the strongest adoption signal available for a harness like this. It is run by a concentrated set of frontier labs and agent teams rather than by a broad install base, so this reading stays directional, and GitHub stars corroborate only loosely alongside it.

• https://os-world.github.io/ recorded 2026-06-04

  best-model vs human baselines; ongoing updates with new LLMs/VLMs

• https://osworld-v1.xlang.ai/ recorded 2026-10-08

  the site os-world.github.io now redirects here; it still publishes the human-vs-model baselines - 'While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success' for the original v1 setting - and still lists the model families evaluated (Operator, GPT, Gemini, Claude, UI-TARS, Agent-S, Qwen, CogAgent).

• https://www.anthropic.com/news/claude-sonnet-4-6 recorded 2026-10-08

  The Sonnet 4.6 release post reports OSWorld results and charts Sonnet's progress on it across sixteen months. The benchmark is cited, which is the standard-citation signal; the numeric score is in the chart rather than the prose.

OSWorld scores computer-use agents by executing their full trajectories in a real desktop environment rather than by multiple choice, across 369 tasks and 134 evaluation functions on Ubuntu, Windows and macOS. It runs under VMware, VirtualBox, Docker or AWS, and frontier computer-use launches report results on it.

• https://github.com/xlang-ai/OSWorld recorded 2026-06-04

  harness + baselines for Operator/GPT/Gemini/Claude/UI-TARS/Agent-S

• https://os-world.github.io/ recorded 2026-09-29

  The site describes OSWorld as "supporting task setup, execution-based evaluation, and interactive learning across operating systems" with "369 computer tasks involving real web and desktop apps in open domains" and a 361-task option for excluding Google Drive tasks.

• https://osworld-v1.xlang.ai/ recorded 2026-09-29

  The site says it evaluates "LLM and VLM from open-source representatives such as UI-TARS, Agent-S, Qwen, Mixtral" alongside closed models, covering Ubuntu, Windows and macOS.

• https://pypi.org/pypi/desktop-env/json recorded 2026-09-29

  The PyPI-hosted README says "Alternatively, you can install the environment without any benchmark tasks" with "pip install desktop-env", and lists VMware, VirtualBox, Docker and AWS providers.

• https://raw.githubusercontent.com/xlang-ai/OSWorld/main/README.md recorded 2026-08-13

  harness runs the environment under VMware, VirtualBox, Docker/KVM, AWS or Azure and evaluates full agent trajectories against any VLM/LLM agent.