Back to Gap Map Infrastructure / Classic ML & computer vision

CoTracker

Meta
restricted / Overall score: 2.8

CoTracker is Meta AI's transformer model for tracking points through video: given any pixels in a frame, picked by hand or sampled on a grid, it follows them jointly across the clip. CoTracker3, the current version, comes in online and offline variants and was trained on synthetic Kubric video, then fine-tuned on real videos with pseudo-labels. It loads through torch.hub.

Openness

2 high confidence
2.0
weights
open (CoTracker3 baseline and scaled, online and offline checkpoints on the Hub, ungated)
data
documented-not-released (the synthetic Kubric training set is on the Hub; the real videos used for pseudo-label fine-tuning are not released)
code
open (training scripts for Kubric and for pseudo-label fine-tuning)
license
CC-BY-NC-4.0 (most of the code and the checkpoints; a few third-party parts are MIT or Apache-2.0)

CoTracker's checkpoints and most of its code are under CC BY-NC 4.0, which forbids commercial use. Meta publishes the training scripts and its synthetic Kubric training set, but not the real videos the released models were fine-tuned on.

Adoption

2 high confidence
2.0

Hugging Face downloads over the trailing 30 days for the CoTracker3 and earlier CoTracker repositories, almost all of them for CoTracker3. Loading through torch.hub fetches the same Hub files.

Capability

4 medium confidence
4.0

CoTracker follows whichever points the user picks in their own video, with no training step, much as Segment Anything segments whatever the user points at. It does point tracking only, where Ultralytics spans detection, segmentation and depth.

Verified 2026-11-12