BERT
Google ResearchBERT (Bidirectional Encoder Representations from Transformers) is Google's encoder-only language model, pretrained by masking words and predicting the next sentence on English Wikipedia and BookCorpus. Released from 2018 in Base (110M parameters) and Large (340M) sizes, cased and uncased, with later multilingual and Chinese checkpoints, it is built to be fine-tuned for classification, tagging and question answering rather than to generate text.
Openness
4 high confidence- weights
- open (Base, Large, multilingual and Chinese checkpoints, ungated on Hugging Face)
- data
- documented-not-released (English Wikipedia and BookCorpus are named, but the preprocessed corpus was never released and BookCorpus is no longer distributed)
- code
- open (create_pretraining_data.py and run_pretraining.py in google-research/bert)
- license
- Apache-2.0 (OSI)
BERT's checkpoints and its pretraining code, from data generation to the training loop, are published under Apache-2.0. The pretraining corpus is described but was never released, and BookCorpus is no longer offered by its collectors, so the original training run cannot be rebuilt exactly.
- https://huggingface.co/api/models/google-bert/bert-base-uncased recorded 2026-11-12
Model API for google-bert/bert-base-uncased: `gated: false`, cardData.license `apache-2.0`, cardData.datasets `bookcorpus` and `wikipedia`, with model.safetensors among the files.
- https://raw.githubusercontent.com/google-research/bert/master/LICENSE recorded 2026-11-12
LICENSE body is the Apache License, Version 2.0.
- https://raw.githubusercontent.com/google-research/bert/master/README.md recorded 2026-11-12
README: "All code *and* models are released under the Apache 2.0 license"; it documents pre-training with create_pretraining_data.py and run_pretraining.py, and states "We will **not** be able to release the pre-processed datasets used in the paper" and that the BookCorpus collectors "no longer have it available for public download".
Adoption
5 high confidenceHugging Face downloads summed across Google's original BERT checkpoints, most of them for bert-base-uncased.
- https://huggingface.co/api/models/google-bert/bert-base-cased recorded 2026-11-12
"downloads":3293999 for google-bert/bert-base-cased, the trailing 30-day count the Hub API reports.
- https://huggingface.co/api/models/google-bert/bert-base-chinese recorded 2026-11-12
"downloads":345615 for google-bert/bert-base-chinese, the trailing 30-day count the Hub API reports.
- https://huggingface.co/api/models/google-bert/bert-base-multilingual-cased recorded 2026-11-12
"downloads":1526145 for google-bert/bert-base-multilingual-cased, the trailing 30-day count the Hub API reports.
- https://huggingface.co/api/models/google-bert/bert-base-multilingual-uncased recorded 2026-11-12
"downloads":1957689 for google-bert/bert-base-multilingual-uncased, the trailing 30-day count the Hub API reports.
- https://huggingface.co/api/models/google-bert/bert-base-uncased recorded 2026-11-12
"downloads":31365205 for google-bert/bert-base-uncased, the trailing 30-day count the Hub API reports.
- https://huggingface.co/api/models/google-bert/bert-large-cased recorded 2026-11-12
"downloads":343944 for google-bert/bert-large-cased, the trailing 30-day count the Hub API reports.
- https://huggingface.co/api/models/google-bert/bert-large-uncased recorded 2026-11-12
"downloads":278884 for google-bert/bert-large-uncased, the trailing 30-day count the Hub API reports.
Capability
1 high confidenceBERT set the state of the art on eleven language-understanding tasks in 2018, but it is an encoder of at most 340M parameters that cannot generate text, and the benchmarks used for today's language models do not apply to it. It sits a step below Comma, a 7B model that only matches 2023-era peers.
- https://arxiv.org/abs/1810.04805 recorded 2026-11-12
Abstract: BERT "obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5% ..., MultiNLI accuracy to 86.7% ..., SQuAD v1.1 question answering Test F1 to 93.2 ... and SQuAD v2.0 Test F1 to 83.1".
- https://raw.githubusercontent.com/google-research/bert/master/README.md recorded 2026-11-12
README lists BERT-Large as "24-layer, 1024-hidden, 16-heads, 340M parameters" and BERT-Base as "12-layer, 768-hidden, 12-heads, 110M parameters".
Verified 2026-11-12