DGX Spark (GH200 480GB) — local LLM fit & speed

NVIDIA · 480 GB LPDDR5X · 500 GB/s · 170 W · MSRP $3,000

CUDA

The DGX Spark (GH200 480GB) is a NVIDIA accelerator with 480 GB LPDDR5X and about 500 GB/s peak memory bandwidth. AI Hashrate estimates local LLM decode (batch 1) at 8K context: fit counts weights + KV cache + 1 GB runtime overhead — not weights alone. At Q4, about 66 curated models fit fully on this card; at FP16, about 55 fit. Top Q4 speeds: MiniCPM5-1B ≈ 294.6 tok/s (estimated, fits); Qwen3.5-2B ≈ 159.1 tok/s (estimated, fits); DeepSeek-R1-0528-Qwen3-8B-layer-mix-bpw-3.8-mlx ≈ 138.3 tok/s (estimated, fits); Qwen3.6-35B-A3B ≈ 106.1 tok/s (estimated, fits); Qwen3.5-35B-A3B ≈ 106.1 tok/s (estimated, fits). Relative ranking is more reliable than absolute tok/s. See methodology for the bandwidth formula and measured-anchor policy. Methodology.

🛒 Where to Buy

Buy on Amazon ↗

affiliate link · we earn from qualifying purchases

Table: decode speed estimates (Q4 / FP16). Measured rows override estimates. MSRP is list price, not live retail. Context default 8K.

Model	Params (B)	Quant	tok/s	Fits?
MiniCPM5-1B	1.08	Q4	294.6 est.	Yes
Qwen3.5-2B	2.0	Q4	159.1 est.	Yes
DeepSeek-R1-0528-Qwen3-8B-layer-mix-bpw-3.8-mlx	2.3	Q4	138.3 est.	Yes
Qwen3.6-35B-A3B	36.0	Q4	106.1 est.	Yes
Qwen3.5-35B-A3B	34.7	Q4	106.1 est.	Yes
GLM-4.7-Flash	31.2	Q4	106.1 est.	Yes
Qwen3-30B-A3B-Instruct-2507	30.5	Q4	106.1 est.	Yes
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16	30.0	Q4	106.1 est.	Yes
Qwen3-Coder-Next	79.7	Q4	106.1 est.	Yes
Ornith-1.0-35B	35.0	Q4	106.1 est.	Yes
Qwen3-Next-80B-A3B-Instruct	81.3	Q4	106.1 est.	Yes
qwen35b-a3b-fable-sft-abliterated	34.7	Q4	106.1 est.	Yes
Llama-3.2-3B-Instruct	3.2	Q4	99.4 est.	Yes
gpt-oss-20b	21.5	Q4	88.4 est.	Yes
gemma-4-26B-A4B-it	25.2	Q4	83.7 est.	Yes
MiniCPM5-1B	1.08	FP16	81.0 est.	Yes
Qwen3-4B-Instruct-2507	4.0	Q4	79.5 est.	Yes
NVIDIA-Nemotron-3-Nano-4B-BF16	4.0	Q4	79.5 est.	Yes
Qwen3-4B-Thinking-2507	4.0	Q4	79.5 est.	Yes
gemma-4-E2B-it	5.1	Q4	62.4 est.	Yes
gpt-oss-120b	120.4	Q4	62.4 est.	Yes
Mistral-Small-4-119B-2603	119.0	Q4	49.0 est.	Yes
Mistral-7B-Instruct-v0.3	7.2	Q4	44.2 est.	Yes
Qwen3.5-2B	2.0	FP16	43.8 est.	Yes
Qwen2.5-7B-Instruct	7.6	Q4	41.9 est.	Yes
Llama-3.1-8B-Instruct	8.0	Q4	39.8 est.	Yes
gemma-4-E4B-it	8.0	Q4	39.8 est.	Yes
Qwen3-8B	8.2	Q4	38.8 est.	Yes
DeepSeek-R1-0528-Qwen3-8B	8.2	Q4	38.8 est.	Yes
DeepSeek-R1-0528-Qwen3-8B-layer-mix-bpw-3.8-mlx	2.3	FP16	38.0 est.	Yes
Qwen3.5-9B	8.95	Q4	35.6 est.	Yes
Ornith-1.0-9B	9.0	Q4	35.4 est.	Yes
Qwen3.5-122B-A10B	122.0	Q4	31.8 est.	Yes
MiniMax-M2.7	228.7	Q4	31.8 est.	Yes
Qwen3.6-35B-A3B	36.0	FP16	29.2 est.	Yes
Qwen3.5-35B-A3B	34.7	FP16	29.2 est.	Yes
GLM-4.7-Flash	31.2	FP16	29.2 est.	Yes
Qwen3-30B-A3B-Instruct-2507	30.5	FP16	29.2 est.	Yes
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16	30.0	FP16	29.2 est.	Yes
Qwen3-Coder-Next	79.7	FP16	29.2 est.	Yes
Ornith-1.0-35B	35.0	FP16	29.2 est.	Yes
Qwen3-Next-80B-A3B-Instruct	81.3	FP16	29.2 est.	Yes
qwen35b-a3b-fable-sft-abliterated	34.7	FP16	29.2 est.	Yes
Step-3.5-Flash	196.8	Q4	28.9 est.	Yes
Llama-3.2-3B-Instruct	3.2	FP16	27.3 est.	Yes
gemma-4-12b-it	11.95	Q4	26.6 est.	Yes
NVIDIA-Nemotron-3-Super-120B-A12B-BF16	120.0	Q4	26.5 est.	Yes
GLM-4.5-Air	110.5	Q4	26.5 est.	Yes
DeepSeek-V4-Flash	158.1	Q4	24.5 est.	Yes
gpt-oss-20b	21.5	FP16	24.3 est.	Yes
OLMo-2-1124-13B-Instruct	13.7	Q4	23.2 est.	Yes
gemma-4-26B-A4B-it	25.2	FP16	23.0 est.	Yes
Qwen3-4B-Instruct-2507	4.0	FP16	21.9 est.	Yes
NVIDIA-Nemotron-3-Nano-4B-BF16	4.0	FP16	21.9 est.	Yes
Qwen3-4B-Thinking-2507	4.0	FP16	21.9 est.	Yes
Qwen3-14B	14.8	Q4	21.5 est.	Yes
Qwen2.5-14B-Instruct	14.8	Q4	21.5 est.	Yes
DeepSeek-R1-Distill-Qwen-14B	14.8	Q4	21.5 est.	Yes
MiMo-V2.5	310.0	Q4	21.2 est.	Yes
Kimi-K2.6	1030.0	Q4	19.9 est.	No
Kimi-K2.7-Code	1030.0	Q4	19.9 est.	No
Llama-4-Scout-17B-16E-Instruct	109.0	Q4	18.7 est.	Yes
Qwen3.5-397B-A17B	397.0	Q4	18.7 est.	Yes
gemma-4-E2B-it	5.1	FP16	17.2 est.	Yes
gpt-oss-120b	120.4	FP16	17.2 est.	Yes
Inkling	975.0	Q4	17.2 est.	No
Qwen3.5-122B-A10B-Heretic-v2-MLX-mixed-3.8bit	19.8	Q4	16.1 est.	Yes
Hy3	295.0	Q4	15.2 est.	Yes
Qwen3-235B-A22B-Instruct-2507	235.1	Q4	14.5 est.	Yes
MiniMax-M3	428.0	Q4	13.8 est.	Yes
Mistral-Small-4-119B-2603	119.0	FP16	13.5 est.	Yes
Devstral-Small-2-24B-Instruct-2512	24.0	Q4	13.3 est.	Yes
Mistral-7B-Instruct-v0.3	7.2	FP16	12.2 est.	Yes
Qwen3.5-27B	27.0	Q4	11.8 est.	Yes
gemma-3-27b-it	27.4	Q4	11.6 est.	Yes
Qwen2.5-7B-Instruct	7.6	FP16	11.5 est.	Yes
Qwen3.6-27B	27.8	Q4	11.4 est.	Yes
Llama-3.1-8B-Instruct	8.0	FP16	10.9 est.	Yes
gemma-4-E4B-it	8.0	FP16	10.9 est.	Yes
Qwen3-8B	8.2	FP16	10.7 est.	Yes
DeepSeek-R1-0528-Qwen3-8B	8.2	FP16	10.7 est.	Yes
gemma-4-31B-it	30.7	Q4	10.4 est.	Yes
MiMo-V2.5	310.0	FP16	9.9 est.	No
Qwen3.5-9B	8.95	FP16	9.8 est.	Yes
Qwen2.5-32B-Instruct	32.8	Q4	9.7 est.	Yes
Qwen2.5-Coder-32B-Instruct	32.8	Q4	9.7 est.	Yes
DeepSeek-R1-Distill-Qwen-32B	32.8	Q4	9.7 est.	Yes
Ornith-1.0-9B	9.0	FP16	9.7 est.	Yes
Qwen3-Coder-480B-A35B-Instruct	480.2	Q4	9.1 est.	Yes
Qwen3.5-122B-A10B	122.0	FP16	8.8 est.	Yes
MiniMax-M2.7	228.7	FP16	8.8 est.	Yes
DeepSeek-R1	684.5	Q4	8.6 est.	Yes
Step-3.5-Flash	196.8	FP16	8.0 est.	Yes
GLM-5.2	753.3	Q4	8.0 est.	Yes
GLM-5.1	753.9	Q4	8.0 est.	Yes
Hy3	295.0	FP16	7.7 est.	No
gemma-4-12b-it	11.95	FP16	7.3 est.	Yes
NVIDIA-Nemotron-3-Super-120B-A12B-BF16	120.0	FP16	7.3 est.	Yes
GLM-4.5-Air	110.5	FP16	7.3 est.	Yes
DeepSeek-V4-Flash	158.1	FP16	6.7 est.	Yes
DeepSeek-V4-Pro	861.6	Q4	6.5 est.	Yes
OLMo-2-1124-13B-Instruct	13.7	FP16	6.4 est.	Yes
Qwen3-14B	14.8	FP16	5.9 est.	Yes
Qwen2.5-14B-Instruct	14.8	FP16	5.9 est.	Yes
DeepSeek-R1-Distill-Qwen-14B	14.8	FP16	5.9 est.	Yes
NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4	550.0	Q4	5.8 est.	Yes
Qwen3.5-397B-A17B	397.0	FP16	5.4 est.	No
Llama-4-Scout-17B-16E-Instruct	109.0	FP16	5.1 est.	Yes
Llama-3.1-70B-Instruct	70.6	Q4	4.5 est.	Yes
Llama-3.3-70B-Instruct	70.6	Q4	4.5 est.	Yes
Qwen2.5-72B-Instruct	72.7	Q4	4.4 est.	Yes
Qwen3.5-122B-A10B-Heretic-v2-MLX-mixed-3.8bit	19.8	FP16	4.4 est.	Yes
Qwen3-235B-A22B-Instruct-2507	235.1	FP16	4.0 est.	Yes
Devstral-Small-2-24B-Instruct-2512	24.0	FP16	3.6 est.	Yes
MiniMax-M3	428.0	FP16	3.4 est.	No
Qwen3.5-27B	27.0	FP16	3.2 est.	Yes
gemma-3-27b-it	27.4	FP16	3.2 est.	Yes
Qwen3.6-27B	27.8	FP16	3.1 est.	Yes
gemma-4-31B-it	30.7	FP16	2.9 est.	Yes
Qwen2.5-32B-Instruct	32.8	FP16	2.7 est.	Yes
Qwen2.5-Coder-32B-Instruct	32.8	FP16	2.7 est.	Yes
DeepSeek-R1-Distill-Qwen-32B	32.8	FP16	2.7 est.	Yes
Mistral-Medium-3.5-128B	128.0	Q4	2.5 est.	Yes
Qwen3-Coder-480B-A35B-Instruct	480.2	FP16	1.8 est.	No
Llama-3.1-70B-Instruct	70.6	FP16	1.2 est.	Yes
Llama-3.3-70B-Instruct	70.6	FP16	1.2 est.	Yes
Qwen2.5-72B-Instruct	72.7	FP16	1.2 est.	Yes
NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4	550.0	FP16	0.9 est.	No
DeepSeek-R1	684.5	FP16	0.8 est.	No
AI21-Jamba-Large-1.5	398.6	Q4	0.8 est.	Yes
Mistral-Medium-3.5-128B	128.0	FP16	0.7 est.	Yes
GLM-5.2	753.3	FP16	0.6 est.	No
GLM-5.1	753.9	FP16	0.6 est.	No
DeepSeek-V4-Pro	861.6	FP16	0.4 est.	No
AI21-Jamba-Large-1.5	398.6	FP16	0.2 est.	No

Fit checks

Qwen3-8B — fits · ≈38.8 tok/s Q4 est.
gemma-4-26B-A4B-it — fits · ≈83.7 tok/s Q4 est.
Qwen2.5-7B-Instruct — fits · ≈41.9 tok/s Q4 est.
Qwen3.5-9B — fits · ≈35.6 tok/s Q4 est.
DeepSeek-R1 — fits · ≈8.6 tok/s Q4 est.
Llama-3.1-8B-Instruct — fits · ≈39.8 tok/s Q4 est.
gpt-oss-20b — fits · ≈88.4 tok/s Q4 est.
Qwen3.6-35B-A3B — fits · ≈106.1 tok/s Q4 est.
Qwen3.5-35B-A3B — fits · ≈106.1 tok/s Q4 est.
gemma-4-E2B-it — fits · ≈62.4 tok/s Q4 est.
Qwen3.6-27B — fits · ≈11.4 tok/s Q4 est.
Mistral-7B-Instruct-v0.3 — fits · ≈44.2 tok/s Q4 est.

All 69 model fit checks — see the linked Fits? column in the table above