Mixtral 8×7B, released December 8, 2023 — again as an unceremonious magnet link — brought sparse mixture-of-experts architecture to the open ecosystem. With eight expert networks of which two activate per token (~13B active of 47B total), it matched or beat GPT-3.5 and Llama 2 70B while inferring at the speed of a much smaller model, all under Apache 2.0.

It was many developers' first hands-on proof that MoE was the future of efficient scaling, months before it became known that GPT-4 itself used the technique. Mixtral powered a generation of open chat products and was followed by the larger Mixtral 8×22B in April 2024.