Comma v0.1

weights

open(Apache-2.0 safetensors, ungated (comma-v0.1-2t, -1t))

data

open(comma_v0.1_training_dataset released ungated on HF

code

open(data pipeline in r-three/common-pile (MIT)

license

Apache-2.0(OSI)

The full stack is public: Apache-2.0 weights, the exact training dataset released ungated on the Hub, the data collection and processing pipeline in an MIT repository, and the training run reproducible from the published lingua config (open BSD-3-Clause framework) — the Apertus shape, where a third-party open framework plus published configs and reconstruction/processing code satisfies code:open. That is the {data: open, code: open, license_tier: osi} rung: 5, open_source, the fully open bucket. Checkpoints are not claimed: the checkpoints repo publishes configs and training metrics rather than intermediate weights, and the dimension is optional. One honest caveat carried from the card: the team cannot guarantee zero unlicensed text due to license laundering upstream, which is a provenance caveat about the corpus, not a restriction on what ships.

• https://huggingface.co/api/models/common-pile/comma-v0.1-2t recorded 2026-09-01

  `gated: false`, cardData.license `apache-2.0`, safetensors shards listed, datasets: [common-pile/comma_v0.1_training_dataset]; 381 downloads in the trailing 30 days.

• https://huggingface.co/common-pile/comma-v0.1-2t/raw/main/README.md recorded 2026-09-01

  Card: "trained on 2 trillion tokens from the Comma v0.1 dataset, comprising of openly licensed text from the Common Pile"; "Training was performed using lingua on 512 AMD MI300A GPUs. Hyperparameters can be found in our lingua config file" linking comma-v0.1-2t-checkpoints/blob/main/config.yaml; benchmark table vs OLMo Twin / Llama 2 / DeepSeekLLM; the license-laundering caveat.

• https://huggingface.co/api/datasets/common-pile/comma_v0.1_training_dataset recorded 2026-09-01

  `gated: false`, `private: false`, 12,338 downloads in the trailing 30 days — the exact training mixture is published and ungated.

• https://api.github.com/repos/r-three/common-pile recorded 2026-09-01

  `license: MIT`, description "Code for collecting, processing, and preparing datasets for the Common Pile" — the data pipeline end of code:open.

• https://api.github.com/repos/facebookresearch/lingua recorded 2026-09-01

  `license: BSD-3-Clause`, description "Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs" — the training framework the card names is itself open.

• https://huggingface.co/api/models/common-pile/comma-v0.1-2t-checkpoints recorded 2026-09-01

  Ungated repo whose files are config.yaml, config.hq_cd.yaml, metrics.jsonl, metrics.hq_cd.jsonl — the published training configs and run metrics; configs and logs rather than intermediate weights, which is why checkpoints is not claimed.

• https://arxiv.org/abs/2506.05209 recorded 2026-09-01

  The Common Pile v0.1 paper: "An 8TB Dataset of Public Domain and Openly Licensed Text" (Kandpal, Lester, Raffel et al., with EleutherAI authors), documenting corpus construction and the Comma training runs.

511 downloads in the trailing 30 days across the two declared checkpoints (comma-v0.1-2t 381; comma-v0.1-1t 130), which bands at level 1 (<10K). A research artifact, not a production model — the training dataset itself sees more pulls (12,338) than the weights.

• https://huggingface.co/api/models/common-pile/comma-v0.1-2t recorded 2026-09-01

  381 downloads in the trailing 30 days for common-pile/comma-v0.1-2t

• https://huggingface.co/api/models/common-pile/comma-v0.1-1t recorded 2026-09-01

  130 downloads in the trailing 30 days for common-pile/comma-v0.1-1t

The finding, stated plainly: openness is not the blocker here, capability is. A 7B model at Llama 2 parity sits two bands below olmo's current fully-open frontier and level with the category's other small fully-open research models (lucie-7b, luciole, above pythia). The result the model exists to demonstrate — that openly licensed data alone reaches budget-matched parity — is real, and it is a 2023-era bar.

• https://huggingface.co/common-pile/comma-v0.1-2t/raw/main/README.md recorded 2026-09-01

  The card's benchmark table: Comma v0.1 2T avg 57.4 vs OLMo Twin 51.6, Llama 2 55.8, DeepSeekLLM 58.8; "It performs comparably to budget-matched models (7 billion parameters, 2 trillion tokens) trained on unlicensed data."