Mixture of Experts

Mistral Large 4 Preview: What a One-Trillion-Parameter Sparse Model Costs to Run, and What Mistral Has Not Said Yet

Mistral Large 4 is a 1T-parameter mixture-of-experts preview with weights promised by month end. The memory and compute math, the self-reported benchmarks,…