Back to Gap Map Model components / Base / pretrained models

T5

Google Research
open source / Overall score: 2.2

T5 (Text-to-Text Transfer Transformer) is Google's encoder-decoder language model that treats every task, from translation and summarization to classification, as turning input text into output text. It was released in 2019 with the C4 web corpus it was pretrained on, in five sizes from Small (60M parameters) to 11B, and Google later published experimental T5 1.1 checkpoints trained on C4 alone. It is used mainly as a base for fine-tuning.

mT5, ByT5 and Flan-T5 were released separately as their own models and are not counted here.

Openness

5 medium confidence
5.0
weights
open (Small, Base, Large, 3B and 11B checkpoints and the T5 1.1 set, ungated on Hugging Face)
data
open (C4 is public and its TensorFlow Datasets builder recreates it from Common Crawl, and the supervised tasks mixed into the original checkpoints are public benchmarks defined in the t5 library. C4 is filtered from Common Crawl's April 2019 web extracted text, and Ai2's copy is released under ODC-BY with the reuser bound by Common Crawl's terms of use for the page content, which leave the pages under their owners' terms. CCBot documents a robots.txt check before fetching. No removal route for C4 was found)
code
open (the t5 library pretrains, fine-tunes and evaluates from the command line)
license
Apache-2.0 (OSI)

T5's checkpoints and training code are published under Apache-2.0, and its C4 pretraining corpus is public, with a TensorFlow Datasets build that recreates it from Common Crawl. The supervised tasks mixed into the original checkpoints are public benchmarks, so the whole training run can be reproduced.

  • https://arxiv.org/pdf/1910.10683 recorded 2026-11-14

    T5 paper, section 2.2: the heuristics for cleaning Common Crawl's web extracted text, then 'To assemble our base data set, we downloaded the web extracted text from April 2019 and applied the aforementioned filtering', released 'as part of TensorFlow Datasets'.

  • https://commoncrawl.org/ccbot recorded 2026-11-14

    'CCBot identifies itself in its UserAgent string as: CCBot/2.0'; 'To prevent Common Crawl from crawling your website, include the following in your robots.txt : User-agent: CCBot Disallow: /'; links an Opt-out Registry.

  • https://commoncrawl.org/faq recorded 2026-11-14

    'CCBot is an automated crawler, checking first the robots.txt'; to block it, 'Add these lines to your robots.txt file and our crawler will stop crawling your website: User-agent: CCBot Disallow: /'; 'Our older bot identified itself with the UserAgent string' CCBot/1.0. The crawler's own documentation of its robots.txt check, not proof that reuse of any page is lawful.

  • https://commoncrawl.org/terms-of-use recorded 2026-11-14

    Common Crawl Terms of Use, 'LAST UPDATED: March 7, 2024'. Crawled Content 'may be subject to separate terms of use or terms of service from the owners of such Crawled Content'; 'BY USING THE CRAWLED CONTENT, YOU AGREE TO RESPECT THE COPYRIGHTS AND OTHER APPLICABLE RIGHTS OF THIRD PARTIES IN AND TO THE MATERIAL CONTAINED THEREIN.' The grant is 'a limited, non-assignable, non-transferable, non-sublicensable, non-exclusive, limited license to access and use the Service', not a license over the crawled pages; the prohibited uses are conduct terms with no commercial or field-of-use bar, and 'CC strongly recommends that you obtain the advice of legal counsel before making any use, including commercial use'. The allenai/c4 card binds the reuser to these terms for the page content.

  • https://huggingface.co/api/datasets/allenai/c4 recorded 2026-11-12

    Dataset API for allenai/c4: `gated: false` and the tag `license:odc-by`.

  • https://huggingface.co/api/models/google-t5/t5-11b recorded 2026-11-12

    Model API for google-t5/t5-11b: `gated: false`, cardData.license `apache-2.0`, with pytorch_model.bin among the files.

  • https://huggingface.co/api/models/google-t5/t5-base recorded 2026-11-12

    Model API for google-t5/t5-base: `gated: false`, cardData.license `apache-2.0`, cardData.datasets `c4`, with model.safetensors among the files.

  • https://huggingface.co/datasets/allenai/c4/raw/main/README.md recorded 2026-11-14

    allenai/c4 card: 'A colossal, cleaned version of Common Crawl's web crawl corpus'; 'This is the processed version of Google's C4 dataset'; 'The C4 and mC4 datasets are collections text sourced from the public Common Crawl web scrape', with the TensorFlow Datasets c4.py build code linked; 'We are releasing this dataset under the terms of ODC-BY. By using this, you are also bound by the Common Crawl terms of use in respect of the content contained in the dataset.' No opt-out, PII or removal route on the card.

  • https://huggingface.co/google-t5/t5-base/raw/main/README.md recorded 2026-11-12

    Model card: pretrained on C4 "on a multi-task mixture of unsupervised (1.) and supervised tasks (2.)", listing the GLUE and SuperGLUE tasks used, and "License: Apache 2.0".

  • https://raw.githubusercontent.com/google-research/text-to-text-transfer-transformer/main/LICENSE recorded 2026-11-12

    LICENSE body is the Apache License, Version 2.0.

  • https://raw.githubusercontent.com/google-research/text-to-text-transfer-transformer/main/README.md recorded 2026-11-12

    README: "The C4 dataset we created for unsupervised pre-training is available in TensorFlow Datasets", with the Beam command that builds it from Common Crawl; command-line instructions to "pre-train a model from scratch"; and the released T5-Small to T5-11B checkpoints.

  • https://raw.githubusercontent.com/google-research/text-to-text-transfer-transformer/main/released_checkpoints.md recorded 2026-11-12

    Experimental t5.1.1 checkpoints, small to xxl, "Pre-trained on C4 only without mixing in the downstream tasks".

Adoption

5 high confidence
5.0

Hugging Face downloads summed across the five original checkpoints and the five T5 1.1 checkpoints, most of them for T5-Small.

Capability

1 medium confidence
1.0

T5 set the state of the art on many language benchmarks in 2019 and, unlike an encoder, it generates text, but it has no results on the benchmarks used for today's language models and is mostly run in its smaller sizes. It sits a step below Comma, a 7B model that only matches 2023-era peers.

Verified 2026-11-12, except some scores, still awaiting re-confirmation