OLMoE Instruct

weights

open(Apache-2.0)

data

open(OLMoE-mix-0924 pretraining + Tulu/UltraFeedback post-training)

code

open(OLMoE training code, SFT/DPO/KTO recipes)

checkpoints

open(244 intermediate)

license

Apache-2.0(OSI)

A fully open mixture-of-experts model: Apache-2.0 weights, open pretraining data (OLMoE-mix-0924, built on Dolma and DataComp), open post-training data, open training code and 244 intermediate checkpoints. Open source at the top of the scale.

• https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct recorded 2026-08-13

  `license:apache-2.0` in the repo tags; "OLMoE is 100% open-source"; the card links Checkpoints, Code, Data and Logs for pretraining, for SFT, and for DPO/KTO separately, and names allenai/ultrafeedback_binarized_cleaned as the preference data

• https://allenai.org/blog/olmoe-an-open-small-and-state-of-the-art-mixture-of-experts-model-c258432d0514 recorded 2026-08-13

  released with open data, code, evaluations, logs, and intermediate training checkpoints

The two declared artifacts sum to 28,149 downloads in the trailing 30 days (allenai/OLMoE-1B-7B-0924-Instruct 26,721; allenai/OLMoE-mix-0924 1,428), which bands at level 2 (10K-100K) on the software and model adoption scale.

• https://huggingface.co/api/models/allenai/OLMoE-1B-7B-0924-Instruct recorded 2026-08-13

  26,721 downloads in the trailing 30 days for allenai/OLMoE-1B-7B-0924-Instruct

• https://huggingface.co/api/datasets/allenai/OLMoE-mix-0924 recorded 2026-08-13

  1,428 downloads in the trailing 30 days for allenai/OLMoE-mix-0924

State-of-the-art among ~1B-active-parameter models at release (beats Gemma2, Llama2-13B-Chat, OLMo-7B, DeepSeekMoE-16B per the blog), but absolute capability is low-tier vs full-size 2026 chat models given the 1.3B active budget. Scored 2 on absolute capability. Exact per-competitor numbers live in arXiv 2409.02060 (blog charts are images), so the comparative claims are directional -- hence medium confidence.

• https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct recorded 2026-08-13

  Evaluation Snapshot, OLMoE +DPO row - MMLU 51.9, GSM8k 45.5, HumanEval 54.8, Alpaca-Eval 1.0 84.0, IFEval 48.1, Avg 57.7; card claims "state-of-the-art performance among models with a similar cost (1B)" and competitiveness with Llama2-13B-Chat

• https://allenai.org/blog/olmoe-an-open-small-and-state-of-the-art-mixture-of-experts-model-c258432d0514 recorded 2026-08-13

  positioned state-of-the-art at the ~1B-active cost class