Mixtral 8×7BOpen Weight
Released
Mixtral 8×7B is an AI model released by Mistral on Friday, Dec 8 2023, 72 days after Mistral 7B. It is an open-weight model — the trained weights are available to download and run.
About
Mixtral 8×7B, released December 8, 2023 — again as an unceremonious magnet link — brought sparse mixture-of-experts architecture to the open ecosystem. With eight expert networks of which two activate per token (~13B active of 47B total), it matched or beat GPT-3.5 and Llama 2 70B while inferring at the speed of a much smaller model, all under Apache 2.0.
It was many developers' first hands-on proof that MoE was the future of efficient scaling, months before it became known that GPT-4 itself used the technique. Mixtral powered a generation of open chat products and was followed by the larger Mixtral 8×22B in April 2024.
Compare Mixtral 8×7B with
Suggested comparisons
Frequently asked questions
Mixtral 8×7B was released by Mistral on Friday, Dec 8 2023.
Mixtral 8×7B was built by Mistral. French AI company building open and commercial models. Founded in 2023 by former Meta and DeepMind researchers.
Partly. Mixtral 8×7B is an open-weight model: the trained weights are free to download and run locally or on your own infrastructure, but the training data and code are not fully released and the license may restrict some commercial uses. It is not open source in the strict sense.
Mistral's previous tracked release was Mistral 7B on Sep 27 2023, 72 days earlier. It was followed by Mistral Medium on Dec 11 2023.