OLMoE Instruct

weights

open(Apache-2.0)

data

open(OLMoE-mix-0924 pretraining + Tulu/UltraFeedback post-training)

code

open(OLMoE training code, SFT/DPO/KTO recipes)

checkpoints

open(244 intermediate)

license

Apache-2.0(OSI)

A fully open mixture-of-experts model: Apache-2.0 weights, open pretraining data, open post-training data, open training code and 244 intermediate checkpoints.

• https://allenai.org/blog/olmoe-an-open-small-and-state-of-the-art-mixture-of-experts-model-c258432d0514 recorded 2026-09-29

  Blog says OLMoE is "released with open data, code, evaluations, logs, and intermediate training checkpoints" and lists "244 checkpoints for the pretrained model, one every 5000 steps".

• https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct recorded 2026-09-29

  Model card states "OLMoE is 100% open-source" and links pretraining, SFT and DPO/KTO checkpoints, code, data and logs.

• https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct/raw/main/README.md recorded 2026-09-29

  Card front matter reads "license: apache-2.0"; summary says "OLMoE is 100% open-source", with pretraining "Data" and "Logs" links, SFT data "tulu-v3.1-mix-preview-4096-OLMoE", and preference data "allenai/ultrafeedback_binarized_cleaned"; code links "allenai/open-instruct/tree/olmoe-sft".

• https://huggingface.co/api/datasets/allenai/OLMoE-mix-0924 recorded 2026-09-29

  Dataset API lists downloadable pretraining shards such as "data/dclm/dclm-0242.json.zst" with tag "license:odc-by" and "gated":false.

• https://huggingface.co/api/datasets/allenai/tulu-v3.1-mix-preview-4096-OLMoE recorded 2026-09-29

  Dataset API for the OLMoE SFT mix (post-training data) carries the tag "license:odc-by", "gated":false, and states "We do not introduce any new data with this dataset." for the "OLMoE SFT Mix".

• https://huggingface.co/api/datasets/allenai/ultrafeedback_binarized_cleaned recorded 2026-09-29

  Dataset API for the preference (DPO) post-training data carries the tag "license:mit", "gated":false, and says "This is a version of the UltraFeedback binarized dataset but with TruthfulQA prompts removed".

• https://raw.githubusercontent.com/allenai/open-instruct/main/LICENSE recorded 2026-09-29

  open-instruct LICENSE file is the full Apache text, headed "Apache License" and "Version 2.0, January 2004", so the post-training code is under an OSI license.

• https://raw.githubusercontent.com/allenai/open-instruct/main/README.md recorded 2026-09-29

  open-instruct README describes the post-training code repo: "Code for finetuning language models with latest techniques and instruction datasets in a unified format." and "Code for DPO, preference finetuning and reinforcement learning with verifiable rewards (RLVR)."

Adoption is measured as Hugging Face downloads summed across the two declared OLMoE artifacts, the instruct model and its pretraining mixture dataset. Nearly all of that volume goes to the model itself rather than to the dataset download.

• https://huggingface.co/api/datasets/allenai/OLMoE-mix-0924 recorded 2026-08-13

  1,428 downloads in the trailing 30 days for allenai/OLMoE-mix-0924

• https://huggingface.co/api/models/allenai/OLMoE-1B-7B-0924-Instruct recorded 2026-08-13

  26,721 downloads in the trailing 30 days for allenai/OLMoE-1B-7B-0924-Instruct

The model card calls it state-of-the-art among models of similar cost, about 1B active parameters, and its own comparison table puts it ahead of OLMo-7B, DeepSeek-3B-16B and Qwen1.5-3B-14B chat models on average score. Its small active-parameter budget keeps it well below full-size current chat models on absolute capability.

• https://allenai.org/blog/olmoe-an-open-small-and-state-of-the-art-mixture-of-experts-model-c258432d0514 recorded 2026-09-29

  Blog title "OLMoE: An open, small, and state-of-the-art mixture-of-experts model" and text on being "the first model to be on the Pareto frontier of performance and size".

• https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct/raw/main/README.md recorded 2026-09-29

  Evaluation Snapshot, OLMoE +DPO row: MMLU 51.9, GSM8k 45.5, Human-Eval 54.8, Alpaca-Eval 1.0 84.0, IFEval 48.1, Avg 57.7; the summary says "state-of-the-art performance among models with a similar cost (1B)" and "competitive with much larger models like Llama2-13B-Chat".