Question 1

What GPU do I need for MoE models?

Accepted Answer

At Q4 / 8K context, the 14 MoE models in our catalog need about 13.55–134.71 GB of VRAM (weights + KV cache + 1 GB overhead). 14 of 14 fully fit at least one retail GPU we track; the most common best-value pick on this page is the Mac Studio M3 Ultra 96GB (MSRP $3,999 — list price, not live retail).

Question 2

Can a 24 GB card like the RTX 4090 24GB run MoE models?

Accepted Answer

Yes for 5 of 14 at Q4/8K: gpt-oss-20b, Qwen3-30B-A3B, Qwen3-30B-A3B-Instruct-2507, Qwen3-Coder-30B-A3B-Instruct and 1 more. The rest need more than 24 GB at 8K context — the per-model minimum is in the table.

Question 3

Should I run MoE models at Q4 or FP16?

Accepted Answer

Q4. FP16 weights are about 3.6x larger than Q4 (2.0 vs 0.55 bytes per parameter), so FP16 needs a much bigger card for the same model. This page ranks GPUs at Q4 / 8K context, the typical local setup; each linked fit check also shows the FP16 verdict.

Question 4

Why do MoE models run fast for their size?

Accepted Answer

A Mixture-of-Experts model stores all of its parameters (which sets the VRAM requirement) but activates only a fraction per token — and decode speed tracks the active fraction. That is why a 30B MoE with 3B active parameters decodes far faster than a dense 30B on the same GPU. Our fit math uses total parameters; our tok/s math uses active parameters.

#	Model	Params	Q4/8K needs (GB)	Best value GPU (fits)	Fastest retail GPU	Buy
1	gpt-oss-20b	21.5B MoE 3.6B act.	13.55	RX 7600 XT 16GB ≈50.9 tok/s est.	RTX 5090 32GB ≈282.5 tok/s measured	Search Amazon (affiliate)
2	Qwen3-30B-A3B	30.5B MoE 3B act.	18.38	RX 7900 XT 20GB ≈169.7 tok/s est.	RTX 5090 32GB ≈380.1 tok/s est.	Search Amazon (affiliate)
3	Qwen3-30B-A3B-Instruct-2507	30.5B MoE 3B act.	18.38	RX 7900 XT 20GB ≈169.7 tok/s est.	RTX 5090 32GB ≈380.1 tok/s est.	Search Amazon (affiliate)
4	Qwen3-Coder-30B-A3B-Instruct	30.5B MoE 3B act.	18.38	RX 7900 XT 20GB ≈169.7 tok/s est.	RTX 5090 32GB ≈380.1 tok/s est.	Search Amazon (affiliate)
5	GLM-4.7-Flash	31.2B MoE 3B act.	18.76	RX 7900 XT 20GB ≈169.7 tok/s est.	RTX 5090 32GB ≈380.1 tok/s est.	Search Amazon (affiliate)
6	Mixtral-8x7B-Instruct-v0.1	46.7B MoE 12.9B act.	29.27	Mac mini M4 32GB ≈5.9 tok/s est.	RTX 5090 32GB ≈88.4 tok/s est.	Search Amazon (affiliate)
7	Qwen3-Coder-Next	79.7B MoE 3B act.	45.44	Mac mini M4 Pro 64GB ≈57.9 tok/s est.	RTX 6000 Ada 48GB ≈203.6 tok/s est.	Search Amazon (affiliate)
8	Qwen3-Next-80B-A3B-Instruct	81.3B MoE 3B act.	46.32	Mac mini M4 Pro 64GB ≈57.9 tok/s est.	Mac Studio M3 Ultra 96GB ≈173.7 tok/s est.	Search Amazon (affiliate)
9	GLM-4.5-Air	110.5B MoE 12B act.	64.18	Mac Studio M3 Ultra 96GB ≈43.4 tok/s est.	Mac Studio M3 Ultra 96GB ≈43.4 tok/s est.	Search Amazon (affiliate)
10	gpt-oss-120b	120.4B MoE 5.1B act.	68.24	Mac Studio M3 Ultra 96GB ≈102.2 tok/s est.	Mac Studio M3 Ultra 96GB ≈102.2 tok/s est.	Search Amazon (affiliate)
11	DeepSeek-V4-Flash	158.1B MoE 13B act.	90.55	Mac Studio M3 Ultra 96GB ≈40.1 tok/s est.	Mac Studio M3 Ultra 96GB ≈40.1 tok/s est.	Search Amazon (affiliate)
12	MiniMax-M2.7	228.7B MoE 10B act.	128.79	Mac Studio M2 Ultra 192GB ≈50.9 tok/s est.	Mac Studio M2 Ultra 192GB ≈50.9 tok/s est.	Search Amazon (affiliate)
13	Qwen3-235B-A22B	235.1B MoE 22B act.	134.71	Mac Studio M2 Ultra 192GB ≈23.1 tok/s est.	Mac Studio M2 Ultra 192GB ≈23.1 tok/s est.	Search Amazon (affiliate)
14	Qwen3-235B-A22B-Instruct-2507	235.1B MoE 22B act.	134.71	Mac Studio M2 Ultra 192GB ≈23.1 tok/s est.	Mac Studio M2 Ultra 192GB ≈23.1 tok/s est.	Search Amazon (affiliate)

Best GPUs for MoE Models You Can Run Locally

Popular fit checks

Compare the picks