RoBERTa-base is a large language model released by Meta, released under the MIT license, with a claimed knowledge cutoff of 2019-07-26. Its weights, configuration, and tokenizer files have been cryptographically hashed and are queued for on-chain attestation by Weight Registry.
Description
Re-tuned BERT pretraining recipe -- 10x more training data (160GB vs BERT's 16GB), larger batches, longer training, dropped the next-sentence-prediction objective, dynamic masking -- at the same 125M parameter count, demonstrating BERT had been substantially undertrained and pushing GLUE/SQuAD state-of-the-art at release.
Metadata
Hugging Face stats
8,183,402 downloads on Hugging Face · 666 likes · pipeline: fill-mask · library: transformers
Weight Hashes
Weights file
SHA-2565bde1d28afb363d0103324efeb5afc8b2b397fe5e04beabb9b1ef355255ade81
SHA3-25620383ef540d32207825f8eaf197a04f79ef84412f605d567f84f44805e3b5935
BLAKE3a6d9c33ed669ea4e72516b1e1b55efe990ac537a02d1ec88748052b89418d087
Config file
SHA-256ef0185e2aae6e06c5f105a285006952c340e20c7dbf43c86ec82601b13fc45e9
SHA3-2562036a6c2403c862e2458d8bb455e9171f5270e211f521a7326c6a83efcd138c5
BLAKE33e6f582c574a43752aeb966d8e376c9bdbdcd0cd0d34e3e947c4abad72c56d53
Tokenizer file
SHA-256847bbeab6174d66a88898f729d52fa8d355fafe1bea101cf960dd404581df70e
SHA3-2560c697da2703e8ea995954b4a85cff8cbe531d8b9429cf74df59db238cb0f96ff
BLAKE34c2a3681edff22cad38aaaf3fcce153102d1e5146d55a39f0ac6632b8faf554e
Verify it yourself
curl -L -o "model.safetensors" "https://huggingface.co/FacebookAI/roberta-base/resolve/e2da8e2f811d1448a5b465c236feacd80ffbac7b/model.safetensors" sha256sum "model.safetensors"
Expected SHA-256: 5bde1d28afb363d0103324efeb5afc8b2b397fe5e04beabb9b1ef355255ade81