Back to Gap Map Infrastructure / Classic ML & computer vision

EoMT

tue-mps
open weights / Overall score: 2.4

EoMT (Encoder-only Mask Transformer) is an image segmentation model that turns a plain Vision Transformer into a segmenter without adapters or decoders, by passing segmentation queries through the encoder alongside the image patches. Built on DINOv2 or DINOv3 backbones, it handles semantic, instance and panoptic segmentation and is available in Hugging Face Transformers. It comes from the Mobile Perception Systems Lab at Eindhoven University of Technology.

Openness

4 medium confidence
4.0
weights
open (DINOv2- and DINOv3-based COCO, ADE20K and Cityscapes checkpoints on the Hub, ungated; the DINOv3 ones are deltas over Meta's DINOv3 weights)
data
documented-not-released (fine-tuned on COCO, ADE20K and Cityscapes from DINOv2 and DINOv3 backbones pretrained on Meta's unreleased LVD-142M and LVD-1689M corpora)
code
open (training and evaluation code and configs)
license
MIT (code and checkpoints)+DINOv3-License (the DINOv3 backbone weights the DINOv3-based checkpoints are applied to)

EoMT's code and checkpoints are MIT, and the training code and configs for COCO, ADE20K and Cityscapes are public. The segmenters are fine-tuned from Meta's DINOv2 and DINOv3 backbones, whose pretraining images Meta has not released, and the DINOv3-based checkpoints are published as differences that need Meta's DINOv3 weights, which come under Meta's own license.

Adoption

2 medium confidence
2.0

Hugging Face downloads over the trailing 30 days for the four most-used EoMT checkpoints, led by the COCO panoptic large model. Weights fetched through the repository's own download links count the same way, since they are served from the Hub.

Capability

3 high confidence
3.0

Each EoMT checkpoint labels pixels with the classes of the benchmark it was trained on, so a new set of classes needs a training run, as with RT-DETR's COCO detectors.

  • https://raw.githubusercontent.com/tue-mps/eomt/master/README.md recorded 2026-11-12

    README: DINOv3-based EoMT "Panoptic Segmentation: Up to 58.9 PQ on COCO", "Instance Segmentation: Up to 49.9 mAP on COCO", "Semantic Segmentation: Up to 59.5 mIoU on ADE20K"; fine-tuning with --model.load_ckpt_class_head False on a dataset with different classes.

Verified 2026-11-12