Scale SEAL Leaderboard

data

closed(proprietary held-out prompt sets, ~1,000 examples/domain, deliberately unpublished to prevent contamination/training-set leakage)

datasheet

no(methodology described but examples not redistributable)

license

none(not distributed)

access

held-out(results visible on leaderboard, dataset not downloadable)

The evaluation sets behind SEAL are deliberately private and held out, so only the leaderboard results are published and never the underlying data, which is why the corpus reads as closed rather than gated or open. Two details this record carries — the approximate size per domain and the mixing in of some open-source material — are not confirmed by the page cited as the source.

• https://scale.com/blog/leaderboard recorded 2026-09-29

  Announces the SEAL Leaderboards and states "proprietary datasets will remain private and unpublished, ensuring they cannot be exploited or incorporated into model training data". Searched the body for download, license and dataset card: no download link, license statement or dataset card.

SEAL is Scale's private leaderboard suite, built on held-out evaluation data, and Scale reports that model developers cite its benchmarks in major model releases and system cards. No download or usage count is published, because the data is held out by design, so the only adoption evidence is Scale's own account of that citation.

• https://scale.com/blog/leaderboard recorded 2026-09-29

  SEAL is positioned as a frontier-model leaderboard answering "Inconsistent Reporting: There is a lack of consistency in model comparisons and reliability in how eval results are reported" in model comparison; searched the body for downloads, users, submissions and usage counts and found none.

• https://scale.com/blog/scale-labs recorded 2026-09-29

  Scale states that SEAL, "launched in 2023, cemented Scale’s authority in benchmarking frontier AI systems, with leading labs citing SEAL benchmarks across major model releases and system cards", a vendor-reported uptake claim with no usage count.

• https://scale.com/leaderboard recorded 2026-09-29

  The leaderboard hub reports "20+" benchmarks and "100+" models evaluated "From leading AI labs including OpenAI, Anthropic, Google, Meta, and open-source contributors", a participation count and not a usage count.

Capability is not a meaningful axis for an eval dataset, so it is left unscored; openness and adoption carry this category.

• https://scale.com/blog/leaderboard recorded 2026-09-29

  The post says the SEAL Leaderboards "rank frontier LLMs using curated private datasets" and lists initial domains Coding, Instruction Following, Math and Multilinguality; the page ranks models and describes no capability of the prompt sets. Searched the body for accuracy, SOTA and benchmark scores of the corpus: none.