RWKV

weights

open(Apache-2.0)

data

documented-not-released(itemized machine-readable component index with resolvable links to public datasets, but no released corpus and no reconstruction pipeline)

code

open(github.com/BlinkDL/RWKV-LM)

license

Apache-2.0(OSI)

Corrected from 5/open_source on 2026-07-28, then re-examined the same day after challenge. The correction stands but it is a close call, and RWKV published more than a first look suggests. What IS released: an itemized, machine-readable index (index/train.jsonl, train.csv) naming every component dataset with a resolvable URL, category and citation, essentially all of them public, plus 100k and 1M preview subsamples. That is real openness work and well beyond a prose description. What is NOT released, and why this is not data:open: no corpus and no reconstruction scripts. The exact subsampling amounts are referenced as "supplementary material (see wiki.txt, oscar.txt)" which is not published in the dataset repo, not an arXiv ancillary file, and not in RWKV-LM, which carries model and training code but no data-prep directory. No mixing proportions or per-component token counts are given, one component (OSCAR23.01) is itself gated, and the maintainers' own notes record "peS2o - Which version out of the 3?", so the mixture is not reproducible in detail even in principle. The rubric defines data:open as a released dataset OR scripts that reconstruct the corpus end to end; neither holds. Weights, training code and license are unaffected and remain fully open under Apache-2.0, so this narrows one claim rather than downgrading the project. NOTE: this lands in a genuine rubric gap. "Every component named and publicly resolvable, but the mixture not reproducible" is meaningfully more open than prose documentation and less than a released corpus, and the data enum has no value for it. Nemotron 3's partial release hits the same gap, so two of the nine products checked fall between the available values.

• https://huggingface.co/datasets/RWKV/RWKV-World-Listing/raw/main/train.csv recorded 2026-07-30

  Header `Dataset;Category;Categories;Citation;Version;URL;Notes`; every source named with a resolvable link. Notes defer exact sampling to unpublished supplementary material ("see wiki.txt in the supplementary material for exact" amounts) and include the unresolved question "peS2o - Which version out of the 3?".

• https://huggingface.co/api/datasets/Goose-World/RWKV-World-v3 recorded 2026-07-30

  `gated: false`, `private: false`, 217 downloads. Files are `index/train.jsonl`, `train.csv`, `100k/subsample_100k.jsonl`, `1m/subsample_1m.jsonl`, two READMEs and one figure. No corpus shards, which is what holds `data` at documented-not-released.

• https://api.github.com/repos/BlinkDL/RWKV-LM recorded 2026-07-30

  `private: false`, license `Apache-2.0`, 14,642 stars, pushed 2026-07-23. Model and training code by version (RWKV-v1..v8). No data-prep or corpus-reconstruction directory.

• https://huggingface.co/api/models/BlinkDL/rwkv-7-world recorded 2026-07-30

  `gated: false`, `private: false`, card license `apache-2.0`, 4,537 downloads. Weight files downloadable as `RWKV-x070-World-{0.1B,0.4B,1.5B}-*.pth`. Added on the 2026-07-30 re-read: the axis recorded `weights:open(Apache-2.0)` while citing only the code repo and the dataset, so nothing on file actually established the weights.

• https://arxiv.org/abs/2503.14456 recorded 2026-07-30

  RWKV-7 Goose paper abstract: "we also present an extended open source 3.1 trillion token multilingual corpus". No arXiv ancillary files are attached, so the supplementary material the dataset notes point to is not obtainable from the paper either.

~14.6k stars on RWKV-LM; HF download volume low/noisy - stars_fallback (capped at 3).

• https://github.com/BlinkDL/RWKV-LM recorded 2026-06-18

  ~14.6k stars, RWKV-7 Goose

Strong intelligence-per-token for its size via a linear-time RNN architecture; not at frontier absolute capability.

• https://arxiv.org/abs/2503.14456 recorded 2026-06-18

  RWKV-7 3B SoTA-multilingual claims, Pareto vs transformers