AI Potluck
Contribute
Back to Gap Map Model components / Base / pretrained models

Comma v0.1

EleutherAI
open source / Overall score: 1.7

A 7B base model from the EleutherAI-led Common Pile collaboration, trained on 2 trillion tokens drawn entirely from the Common Pile v0.1, an 8TB corpus of public domain and openly licensed text. It exists to test whether competitive models can be trained without unlicensed data, and performs comparably to budget-matched models like Llama 2 7B.

Openness

5 high confidence
5.0
weights
open(Apache-2.0 safetensors, ungated (comma-v0.1-2t, -1t))
data
open(comma_v0.1_training_dataset released ungated on HF
code
open(data pipeline in r-three/common-pile (MIT)
license
Apache-2.0(OSI)

Comma's Apache-2.0 weights ship alongside the exact training dataset, released ungated on the Hub, an MIT-licensed data pipeline, and a training run reproducible from a published config on the open BSD-3-Clause Lingua framework. Only intermediate checkpoints are missing, and the team notes it cannot guarantee the corpus is entirely free of unlicensed text because of license laundering upstream.

  • https://api.github.com/repos/facebookresearch/lingua recorded 2026-09-01

    `license: BSD-3-Clause`, description "Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs" — the training framework the card names is itself open.

  • https://api.github.com/repos/r-three/common-pile recorded 2026-09-01

    `license: MIT`, description "Code for collecting, processing, and preparing datasets for the Common Pile" — the data pipeline end of code:open.

  • https://arxiv.org/abs/2506.05209 recorded 2026-10-07

    The Common Pile v0.1 paper: "An 8TB Dataset of Public Domain and Openly Licensed Text" (Kandpal, Lester, Raffel et al., with EleutherAI authors), documenting corpus construction and the Comma training runs.

  • https://huggingface.co/api/datasets/common-pile/comma_v0.1_training_dataset recorded 2026-10-07

    Dataset API for common-pile/comma_v0.1_training_dataset: `gated: false`, `private: false`, with the training mixture published as per-source jsonl.gz chunks.

  • https://huggingface.co/api/models/common-pile/comma-v0.1-2t recorded 2026-10-07

    Model API for common-pile/comma-v0.1-2t: `gated: false`, `private: false`, cardData.license `apache-2.0`, safetensors weights listed, and datasets `common-pile/comma_v0.1_training_dataset`.

  • https://huggingface.co/api/models/common-pile/comma-v0.1-2t-checkpoints recorded 2026-10-07

    Ungated repo (`gated: false`) whose files are config.yaml, config.hq_cd.yaml, metrics.jsonl and metrics.hq_cd.jsonl: the published training configs and run metrics, not intermediate weights.

  • https://huggingface.co/common-pile/comma-v0.1-2t/raw/main/README.md recorded 2026-10-07

    Card: Comma v0.1-2T is a 7 billion parameter language model "trained on 2 trillion tokens from [the Comma v0.1 dataset]", "comprising of openly licensed text from [the Common Pile]". "Training was performed using [lingua]", and "Hyperparameters can be found in our [lingua config file]", which links comma-v0.1-2t-checkpoints. The card carries a benchmark table against OLMo Twin, Llama 2 and DeepSeekLLM, and the caveat that "license laundering and inaccurate metadata can result in erroneous license information".

Adoption

1 high confidence
1.0

Adoption is measured as Hugging Face downloads summed across the two declared checkpoints. Comma is a research artifact rather than a production model, and its training dataset draws more downloads than the weights themselves.

Capability

2 high confidence
2.0

Comma reaches Llama 2-era parity, showing that openly licensed data alone can match budget-trained peers. That is a 2023-era capability bar, well below OLMo's current frontier and level with the category's other small fully-open research models, Lucie-7B and Luciole, ahead of Pythia.

Verified 2026-09-27