RWKV

weights

open(Apache-2.0)

data

documented-not-released(itemized machine-readable component index with resolvable links to public datasets, but no released corpus and no reconstruction pipeline)

code

open(github.com/BlinkDL/RWKV-LM)

license

Apache-2.0(OSI)

This is a close call, and RWKV has published more than a first look suggests. What is released: an itemized, machine-readable index (index/train.jsonl and train.csv) naming every component dataset with a resolvable URL, a category and a citation - almost all of them public - plus 100k and 1M preview subsamples. That is real openness work, well beyond a prose description of the mixture. What is not released is the corpus itself, or any script that reconstructs it. The exact subsampling amounts are referred to as supplementary material (wiki.txt, oscar.txt) that appears neither in the dataset repository, nor among the arXiv ancillary files, nor in RWKV-LM, which carries model and training code but no data-preparation directory. No mixing proportions or per-component token counts are given, one component (OSCAR23.01) is itself gated, and the maintainers' own notes still ask which of the three versions of peS2o was used, so the mixture is not reproducible in detail even in principle. Open data means a released dataset or scripts that reconstruct the corpus end to end, and neither holds here - which narrows one claim rather than downgrading the project, since the weights, the training code and the license are all fully open under Apache-2.0. This lands in a genuine gap in the openness scale: every component named and publicly resolvable, but the mixture not reproducible, is meaningfully more open than prose documentation and less than a released corpus, and the scale has no value for it. Nemotron 3's partial release falls into the same gap. The evidence behind all of this: the Goose-World repository holds only the component index, the two preview subsamples, two READMEs and a figure, with no corpus shards; RWKV-LM is public and Apache-2.0 at 14,661 stars with no data-preparation directory; and rwkv-7-world is ungated apache-2.0.

• https://huggingface.co/datasets/RWKV/RWKV-World-Listing/raw/main/train.csv recorded 2026-08-13

  Header `Dataset;Category;Categories;Citation;Version;URL;Notes`; every source named with a resolvable link. Notes defer exact sampling to unpublished supplementary material ("see wiki.txt in the supplementary material for exact" amounts) and include the unresolved question "peS2o - Which version out of the 3?".

• https://huggingface.co/api/datasets/Goose-World/RWKV-World-v3 recorded 2026-08-13

  `gated: false`, `private: false`, 217 downloads. Files are `index/train.jsonl`, `train.csv`, `100k/subsample_100k.jsonl`, `1m/subsample_1m.jsonl`, two READMEs and one figure. No corpus shards, which is what holds `data` at documented-not-released.

• https://api.github.com/repos/BlinkDL/RWKV-LM recorded 2026-08-13

  `private: false`, license `Apache-2.0`, 14,642 stars, pushed 2026-07-23. Model and training code by version (RWKV-v1..v8). No data-prep or corpus-reconstruction directory.

• https://huggingface.co/api/models/BlinkDL/rwkv-7-world recorded 2026-08-13

  `gated: false`, `private: false`, card license `apache-2.0`, 4,537 downloads. Weight files downloadable as `RWKV-x070-World-{0.1B,0.4B,1.5B}-*.pth` - the only source on file that establishes the open weights this axis records.

• https://arxiv.org/abs/2503.14456 recorded 2026-08-13

  RWKV-7 Goose paper abstract: "we also present an extended open source 3.1 trillion token multilingual corpus". No arXiv ancillary files are attached, so the supplementary material the dataset notes point to is not obtainable from the paper either.

4,652 downloads in the trailing 30 days across the two repositories the product declares (BlinkDL/rwkv-7-world 4,395; RWKV/RWKV7-Goose-World3-1.5B-HF 257), which bands at level 1 (<10K) on the model adoption scale. A real download signal exists, so the band rests on that rather than on a GitHub star count, which had been flattering the family's reach. Widening the count to every repository under the BlinkDL and RWKV organizations would give about 71,000 downloads and level 2, but only the artifacts a product declares count toward its reach.

• https://huggingface.co/api/models/BlinkDL/rwkv-7-world recorded 2026-08-12

  4,395 downloads in the trailing 30 days for BlinkDL/rwkv-7-world

• https://huggingface.co/api/models/RWKV/RWKV7-Goose-World3-1.5B-HF recorded 2026-08-12

  257 downloads in the trailing 30 days for RWKV/RWKV7-Goose-World3-1.5B-HF

Strong intelligence-per-token for its size via a linear-time RNN architecture; not at frontier absolute capability. The paper makes a 3B-scale multilingual claim and an efficiency argument and makes no frontier-scale claim at all, which is the shape of a 4 rather than a 5 here.

• https://arxiv.org/abs/2503.14456 recorded 2026-08-13

  RWKV-7 Goose paper - 3B SoTA-multilingual claims on far fewer training tokens, and the extended 3.1-trillion-token multilingual corpus