RoBERTa-base is a large language model released by Meta, released under the mit license, with a claimed knowledge cutoff of 2019-07-26. Its weights, configuration, and tokenizer files have been cryptographically hashed and permanently attested on Ethereum (Sepolia) by Weight Registry.
Description
Re-tuned BERT pretraining recipe -- 10x more training data (160GB vs BERT's 16GB), larger batches, longer training, dropped the next-sentence-prediction objective, dynamic masking -- at the same 125M parameter count, demonstrating BERT had been substantially undertrained and pushing GLUE/SQuAD state-of-the-art at release.
Metadata
Hugging Face stats
8,183,402 downloads on Hugging Face · 666 likes · pipeline: fill-mask · library: transformers
Weight Hashes
Combined commitment: a Merkle root over the weights, config, and tokenizer hashes together (not a raw weights-only hash) -- see this project's v2 on-chain schema.
Combined SHA-2560x38a06929f7a80862d2dc79cb9e6f32f9797765b8db678e168d3dea337d7f47ed
Combined SHA3-2560xd7f7ffef6934f9fa352e06225da554d66fe5459e4b73b34dd58d062cb4dba05b
Combined BLAKE30x6ed50439f0daf5fb7058b1c19c48981a26d56194b36f7deba9f37890930928c2
Per-file breakdown
Computed locally prior to attestation; the combined hash above is the canonical on-chain commitment.
Weights file
SHA-2565bde1d28afb363d0103324efeb5afc8b2b397fe5e04beabb9b1ef355255ade81
SHA3-25620383ef540d32207825f8eaf197a04f79ef84412f605d567f84f44805e3b5935
BLAKE3a6d9c33ed669ea4e72516b1e1b55efe990ac537a02d1ec88748052b89418d087
Config file
SHA-256ef0185e2aae6e06c5f105a285006952c340e20c7dbf43c86ec82601b13fc45e9
SHA3-2562036a6c2403c862e2458d8bb455e9171f5270e211f521a7326c6a83efcd138c5
BLAKE33e6f582c574a43752aeb966d8e376c9bdbdcd0cd0d34e3e947c4abad72c56d53
Tokenizer file
SHA-256847bbeab6174d66a88898f729d52fa8d355fafe1bea101cf960dd404581df70e
SHA3-2560c697da2703e8ea995954b4a85cff8cbe531d8b9429cf74df59db238cb0f96ff
BLAKE34c2a3681edff22cad38aaaf3fcce153102d1e5146d55a39f0ac6632b8faf554e
Verify it yourself
curl -L -o "model.safetensors" "https://huggingface.co/FacebookAI/roberta-base/resolve/e2da8e2f811d1448a5b465c236feacd80ffbac7b/model.safetensors" sha256sum "model.safetensors"
Expected SHA-256: 5bde1d28afb363d0103324efeb5afc8b2b397fe5e04beabb9b1ef355255ade81