AI Potluck
Contribute
Back to Gap Map Model components / Embeddings & retrieval models

ColPali

ILLUIN Technology
open weights / Overall score: 4.0(strong)

ColPali applies the ColBERT late-interaction recipe to page screenshots: a vision-language model embeds each image patch of a rendered document page, a query is embedded token by token, and MaxSim scores the two multi-vector sets. That removes the OCR-and-chunk pipeline from document retrieval entirely. The line has grown past the original PaliGemma-based ColPali into ColQwen2, ColQwen2.5, the sub-1B ColSmol checkpoints and ColQwen-Omni, all released from the vidore Hub org with the shared colpali-engine training and inference code.

Published by ILLUIN Technology - code at github.com/illuin-tech/colpali, weights and the ViDoRe benchmark under the `vidore` Hugging Face org - not by a company called ViDoRe.

Openness

3 high confidence
3.0
weights
open(45 ungated checkpoints on the vidore org, covering ColPali, ColQwen2/2.5, ColSmol and ColQwen-Omni)
data
open(vidore/colpali_train_set, the 118,195-pair training set, published ungated on the Hub)
code
open(colpali-engine[train] ships scripts/train/train_colbert.py and the per-model training configs used for the released checkpoints)
license
Gemma-License(non-OSI, resolved most-restrictive across the distributed SKUs - the PaliGemma-backed colpali-v1.2/v1.3-hf and -merged weights carry the Gemma terms)+MIT(OSI, the ColPali adapters and, with Apache-2.0 Qwen2-VL backbones, the ColQwen SKUs)

ColPali publishes its training set and training pipeline in full, which is as open as data and code get in this category. But some checkpoints, the PaliGemma-backed ColPali releases, merge in Gemma-licensed weights, and a family's license reading follows its most restrictive distributed checkpoint; the ColQwen line alone, released under MIT and Apache-2.0, would score higher.

Adoption

4 high confidence
4.0

Adoption is measured as Hugging Face downloads summed across the 45 vidore repositories carrying the ColPali and ColQwen lines, wider than the two checkpoints this product declares. Downloads run mostly to the ColQwen successors, with the original ColPali checkpoints trailing well behind.

Capability

4 high confidence
4.0

ColPali's family, especially ColQwen2.5, sits near the top of the ViDoRe leaderboard it defined, a few points behind the current leader. It carries the same MaxSim late-interaction mechanism as ColBERT, extended from text retrieval into the document-image setting.

Verified 2026-09-11