Mistral models — VRAM fit & GPU picks

Mistral's catalog spans the classic 7B v0.3, the Devstral 2 24B coding model, and the 119B/128B Small 4 / Medium 3.5 flagships at 256K context. Mix of Apache-2.0 and Modified MIT licenses.

Mistral AI is the French lab that turned "small but mighty" into a brand. Its open-weight catalog now stretches from the classic Mistral 7B — still one of the most fine-tuned models in the community — up to the 119B Mistral Small 4 and 128B Mistral Medium 3.5 flagships, both pushing 256K context windows. Along the way it picked up Devstral 2, a 24B model aimed squarely at coding agents.

For local inference the family is friendly to plan around: everything ships as GGUF, most of the line is Apache-2.0, and the two big models split neatly into "one big MoE that idles cheap" and "one dense heavyweight that wants serious VRAM." Below we size each model for Q4 at 8K context using our standard formula: Q4 VRAM ≈ q4_gb + 0.025 × active_b × 8 + 1 GB of overhead.

Size ladder

Mistral-7B-Instruct-v0.3 (7.2B dense) — Q4 ≈ 4.0 GB weights, ~6.5 GB total at 8K → comfortable on any 12GB card. The veteran: 32K context, Apache 2.0, and years of community fine-tunes and tooling. Still a fine default for lightweight chat, embeddings-adjacent tasks, and low-power boxes.
Devstral-Small-2-24B-Instruct-2512 (24B dense) — Q4 ≈ 13.2 GB weights, ~19 GB total at 8K → needs a 24GB card (RTX 3090/4090 class). Mistral's coding-focused model with a 256K context, built for agentic coding loops where long files and tool traces eat context fast. Apache-2.0.
Mistral-Small-4-119B-2603 (119B MoE, 6.5B active, 128 experts / 4 per token) — Q4 ≈ 65.5 GB weights, ~68 GB total at 8K → fits a 96GB setup (dual 48GB or a workstation card). Don't let "Small" fool you: it stores 119B parameters but only activates 6.5B per token, so speed feels like a ~7B model while quality sits far above it. Apache-2.0, 256K context.
Mistral-Medium-3.5-128B (128B dense) — Q4 ≈ 70.4 GB weights, ~97 GB total at 8K → just over a single 96GB card; plan for 192GB+ (multi-GPU) or accept heavy offloading. The dense flagship: every token touches all 128B parameters, so it is dramatically slower per second than Small 4 but appeals to users who want dense-model behavior at 256K context. Note the Modified MIT license — check the terms before commercial deployment.

What makes the family interesting

MoE where it matters. Mistral Small 4 is the standout for homelab builders: MoE inference cost is driven by the 6.5B active parameters, not the 119B total, so tok/s lands in single-GPU-friendly territory — you pay in VRAM, not in speed. The KV cache term also keys off active parameters, which keeps long-context costs modest relative to a dense model of the same total size.

256K context on the 2026 models. Devstral 2, Small 4, and Medium 3.5 all ship 256K context. Be aware that our 8K estimates above are the shopping baseline; at much longer contexts the KV cache grows linearly and can add tens of GB on the dense Medium 3.5. Budget headroom if long documents are your use case.

Licensing is mostly permissive, with one exception. 7B v0.3, Devstral 2, and Small 4 are Apache-2.0 — do what you want. Medium 3.5 uses a Modified MIT license; read it before building a product on top.

Which one should you pick

12GB VRAM (RTX 3060 12GB, etc.): Mistral-7B-Instruct-v0.3, with room to spare even at its full 32K context.
16GB VRAM: still 7B territory for Q4 comfort — Devstral 2 at Q4 is ~19 GB, just out of reach unless you drop to Q2 or offload layers.
24GB VRAM (RTX 3090/4090): Devstral-Small-2-24B is the pick, especially if you write code. It is the largest Mistral that runs purely on a single consumer card.
48GB VRAM (dual 24GB or one A6000): a middle ground — you can run Devstral 2 at higher quants and longer context, but the 100B+ models still won't fit.
96GB VRAM: Mistral-Small-4-119B at Q4 becomes practical, and its MoE speed makes it the sweet spot of the whole family.
192GB+ VRAM: Mistral-Medium-3.5-128B at Q4 (~97 GB) fits with context headroom to spare.

If you can only buy one GPU tier for Mistral in 2026, 96GB is the value king — it unlocks Small 4, the model where Mistral's MoE engineering pays off most at home.

Links

Models in this family

4 models in the Mistral family, grouped by series. Q4 (GB) is weights-only; total VRAM adds KV cache and overhead — the fit check links compute it for a representative retail GPU at 8K context.