Mixtral-8x22B and the Rise of Open Mixture-of-Experts Models

By early 2024, Mistral AI had already popularized the sparse Mixture-of-Experts (MoE) architecture in the open-weight world with the smaller Mixtral 8x7B. Mixtral-8x22B-Instruct-v0.1, released in January 2024, scaled that same idea up: 141 billion total parameters, with only a fraction active per token, released fully under Apache 2.0.

Why it mattered

Sparse MoE architectures let a model have a much larger total parameter count -- and the capacity that implies -- without paying the full inference cost of a dense model that size. Mixtral-8x22B helped prove this approach could be released fully open and still be commercially usable without restriction, a pattern several other issuers, including DeepSeek, would build on at even larger scale.

On the record

• Issuer: Mistral AI
• License: Apache-2.0
• Claimed cutoff: 2024-01-31
• Status: hashed, queued for on-chain attestation

See the full hash breakdown on Mixtral-8x22B-Instruct-v0.1's Explorer page.