Back to Gap Map Model components / Embeddings & retrieval models

Perception Encoder

Meta
open weights / Overall score: 3.7

Meta FAIR's family of contrastively trained vision encoders. The PE Core models are CLIP-style image and video encoders with a matching text encoder, for zero-shot classification and retrieval, in sizes from T/16 to G/14, the smaller ones distilled from the largest. Meta describes them as outperforming SigLIP 2 on image benchmarks and InternVideo2 on video.

Only the PE Core checkpoints are declared. PE Lang and PE Spatial, tuned for language models and dense prediction, and the audio-visual PE-AV models come from the same repository but are not retrieval models; the Perception LM checkpoints are a separate entry.

Openness

3 high confidence
3.0
weights
open (five PE Core checkpoints on the Hub, ungated)
data
closed (the image-text pretraining corpus is not named or released; the PE Video Dataset used for video fine-tuning is released)
code
partial (model code with zero-shot classification and retrieval evaluation; no pretraining pipeline)
license
Apache-2.0 (OSI)

The PE Core weights download freely under Apache-2.0. Meta releases the video dataset used to fine-tune them, but not the image-text corpus they were pretrained on or the pipeline that trained them.

Adoption

3 high confidence
3.0

Hugging Face downloads over the trailing month, summed across the five PE Core checkpoints and led by the L/14 model. The timm conversions are excluded as format copies.

Capability

4 medium confidence
4.0

Meta's figures put PE Core slightly ahead of SigLIP 2 on the same zero-shot ImageNet and COCO retrieval tests. The margin is narrow and comes from Meta's own table, so it sits level with Google's encoder rather than above it.

Verified 2026-11-12