Back to Gap Map Model components / Fine-tuned / chat models

Devstral

Mistral AI
open weights / Overall score: 3.7

Devstral is an open-weight family of agentic coding models from Mistral, built for software-engineering tasks such as exploring codebases and editing multiple files. Devstral Small 2 is a 24B instruct model with a 256K context window that runs on a single RTX 4090 or a Mac with 32GB RAM, and the 123B Devstral 2 is the larger model in the release.

Scored against the Devstral 2 release; the earlier Small 1.1 checkpoints are superseded.

Openness

3 high confidence
3.0
weights
open(Apache-2.0, on HF)
data
closed
code
partial(inference/serving
license
Mistral-Modified-MIT(non-OSI

Devstral 2's smaller 24B model ships under Apache-2.0, but the larger 123B model carries a non-OSI Modified MIT license; since the two are sizes of one release, the record follows the more restrictive license. Training and post-training data and the pipeline behind the checkpoints are undisclosed, which keeps the family at open weights regardless of which license the SKU uses.

Adoption

3 high confidence
3.0

Devstral is a specialist coding-agent model, adopted mainly through OpenHands and similar coding-agent tooling rather than general chat use. Adoption is measured as Hugging Face downloads summed across the two declared Devstral 2 checkpoints, most of it on the smaller 24B model rather than the 123B flagship.

Capability

4 high confidence
4.0

Devstral 2's 123B model scores 72.2% on SWE-bench Verified on its model card, ahead of Qwen3 Coder Plus, Kimi K2 Thinking and GLM 4.6, while the smaller 24B model scores 68.0%. DeepSeek v3.2 and the proprietary models on the same table score higher, and Mistral's newer Medium 3.5 model has replaced Devstral 2 as its coding model, which keeps Devstral a step below the frontier.

  • https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512/raw/main/README.md recorded 2026-09-29

    Benchmark Results table in the model card: the "Devstral 2" row reads "72.2%" on SWE Bench Verified, "61.3%" on SWE Bench Multilingual and "32.6%" on Terminal Bench 2; "Devstral Small 2" reads "68.0%", "55.7%" and "22.5%"; competitor rows include "Kimi K2 Thinking", "GLM 4.6" and "DeepSeek v3.2", with "Claude Sonnet 4.5" at "77.2%"; the footnote reads "Benchmark results presented are based on publicly reported values for competitor models".

  • https://huggingface.co/mistralai/Mistral-Medium-3.5-128B/raw/main/README.md recorded 2026-09-29

    Vendor card for a newer model: "Mistral Medium 3.5 supersedes all our previous coding models, namely Devstral, across all benchmarks" and "It scores **91.4%** on τ³-Telecom and **77.6%** on SWE-Bench Verified."

Verified 2026-09-27