AI Potluck
Back to Gap Map Model components / Embeddings & retrieval models

CLIP

OpenAI
open weights / Overall score: 2.9

CLIP is OpenAI's 2021 contrastive image-text model and the origin of the whole cross-modal embedding line. An image encoder and a text encoder are trained to put matching pairs close together in one shared space, which makes image-text retrieval and zero-shot classification the same operation. Five years on it remains the single most downloaded vision model on the Hub, mostly as the frozen text or image tower inside diffusion and VLM stacks rather than as a retriever in its own right.

None of the four openai/clip-* Hub repos declares a license field; the MIT grant comes from the openai/CLIP repository LICENSE that distributes the checkpoints. The Xenova and timm re-uploads of these weights are mirrors, not products, and are excluded from the download sum. Verified 2026-09-11 via the model card, the CLIP repository and the Hugging Face API.

Openness

3 high confidence
3.0
weights
open(four ungated openai/clip-vit-* repos on the Hub, plus the original checkpoints served by the CLIP package)
data
closed(WIT-400M
code
partial(the openai/CLIP repository ships the model definition, inference and zero-shot evaluation notebooks, with no training pipeline)
license
MIT(OSI, from the openai/CLIP repository LICENSE

The category's centre of gravity, and the reason OpenCLIP exists. Weights under an OSI licence that anyone can run, a corpus the publisher says explicitly it will never release, and a repository that lets you use the model but not rebuild it.

Adoption

5 high confidence
5.0

32,961,856 downloads in the trailing 30 days across the four openai/clip-vit-* repos (base-patch32 21,008,990; large-patch14 7,730,667; large-patch14-336 2,586,045; base-patch16 1,636,154). The highest figure anywhere in this set, and the base-patch32 repo alone clears the top band.

Capability

2 high confidence
2.0

Rung 2 on the definition rather than on reputation: English-only, a 77-token text context, and beaten on its own instrument by both the open reimplementation trained on open data and by SigLIP 2. Still enormously useful in its niche, which is exactly what rung 2 describes. Placed against OpenCLIP because the two are measured in the same table.

Verified 2026-09-11