DistilBERT-base-uncased is a large language model released by Hugging Face, released under the Apache-2.0 license, with a claimed knowledge cutoff of 2019-10-02. Its weights, configuration, and tokenizer files have been cryptographically hashed and are queued for on-chain attestation by Weight Registry.
Description
Knowledge-distilled compression of BERT (66M params vs BERT-base's 110M, ~40% smaller, ~60% faster at inference) retaining ~97% of its GLUE benchmark performance via a triple loss combining distillation, masked-language-modeling, and cosine-embedding objectives -- an early landmark for practical model distillation.
Metadata
Hugging Face stats
8,071,364 downloads on Hugging Face · 1,473 likes · pipeline: fill-mask · library: transformers
Weight Hashes
Weights file
SHA-2565e3f1108e3cb34ee048634875d8482665b65ac713291a7e32396fb18f6ff0063
SHA3-256c3be58cb344e645d9610df852f6931f9e594abaa3156f178e00ae5954c4c4a7f
BLAKE3e620f44be96fcb0c90459dd67f5e3a2c588e86c9fec7e6db8ff60c0068bd4397
Config file
SHA-25669c94b0222d5d1f4b0ad027ca7416cdafb98378cbbb8305d0bf47c9365c60c83
SHA3-256b46c69554638f95a863a2f9d4e1c41aba9a5f05bc7e7ef6a5123569a17357ee6
BLAKE3b147365f03a293dd2516d4605ab76ab35140e1d19e5ce30baa40584e50234878
Tokenizer file
SHA-256ce64fce797c24f68df90b40a3f74f579b336a493db14bd583fd520ea0d8c9a98
SHA3-25648e0eb031593b2d46432e97a97073752b4d62db14a58838b8381b1446916b986
BLAKE33d5a598086b8e94c303d07454f3451cdb37cecf75c207fc37958e7c06ff07878
Verify it yourself
curl -L -o "model.safetensors" "https://huggingface.co/distilbert/distilbert-base-uncased/resolve/12040accade4e8a0f71eabdb258fecc2e7e948be/model.safetensors" sha256sum "model.safetensors"
Expected SHA-256: 5e3f1108e3cb34ee048634875d8482665b65ac713291a7e32396fb18f6ff0063