AI Hashrate Which GPU for local LLMs? EN

更新于 2026-07-17 17:28:14 · 63 个模型 · 50 款 GPU · 100 条实测锚点

Tell us your budget — we’ll suggest GPUs that can run local models.

Use feel labels to shop — tok/s is a rough estimate. List $ is MSRP only; Amazon price is whatever you see after click. We prefer picks with a real product-page Buy link. MoE speed estimates can look optimistically high.

Default view: consumer GPUs only · Q4 quant · 8K context. How estimates work

I already know my GPU or model (advanced)

I have this GPU — what can I run?

I want this model — which GPU?

Quant Context Aim for smooth chat (~30+ tok/s est.) · fit includes context memory
Show Hides datacenter cards (H100, etc.)

Full comparison table

Hidden by default so you can decide from the picks above. Open only if you want every model × GPU cell.

Loading data…

FAQ

How the numbers work, in plain terms. Full math lives on the methodology page.

How are tok/s numbers estimated?
Decode (generation) at batch size 1 is mostly memory-bandwidth bound, so each estimate is: tok/s ≈ (memory bandwidth GB/s × utilization) ÷ (active params in billions × bytes per param). Utilization is 35% of theoretical peak when the model fits in VRAM, 8% for CPU-only, and falls as (VRAM ÷ model size)² when it does not fit. The relative ranking between GPUs is more reliable than the absolute number.
What does "fit" mean?
A model fits when its estimated memory need — weights + KV cache at the selected context length + ~1 GB runtime overhead — stays within 95% of the card's VRAM. Fit is not weights-only: an 8 GB card can hold ~8B Q4 weights alone but still fail at 8K context, which we mark as No. Default context is 8K; you can switch 4K / 8K / 32K.
Q4 vs FP16 — which one should I look at?
Q4 (~0.55 bytes per param) is the typical local quantization: much smaller and faster per token, with a modest quality trade-off. FP16 (2.0 bytes per param) is full precision — roughly 4× the memory and bandwidth cost. The default view is Q4; switch any time in the Quant row.
Why does a measured number differ from the estimate?
Cells labeled measured are real published or curated results that override the estimate for that model × GPU × quant. Real inference engines (llama.cpp, vLLM, MLX, …), batch size, and kernels can move results by tens of percent, so treat estimates as a shopping guide, not a purchase certificate.
Is there an API?
Yes. The same JSON the site renders is available at /api/data (models, hardware, and measured anchors). Estimate math runs in the browser with the same constants; the formula is documented on the methodology page. The API is read-only and needs no key.
Are the prices live?
No. The dollar figures are MSRP (manufacturer list price), not live street prices. Buy links open a specific Amazon product page when we have an ASIN, otherwise an Amazon search — the actual price is whatever Amazon shows after you click. Links are affiliate (Amazon Associates) and may earn us a commission at no extra cost to you.
Numbers off for your setup? Send your tok/s · Missing model/GPU? Request · Methodology
Guides: VRAM by model size · Q4 vs FP16 · Best GPUs 2026 · Offload & MoE
MSRP is manufacturer list price, not live street price. Buy opens a specific Amazon product page when we have an ASIN, otherwise Amazon search. May earn us a commission at no extra cost to you (Amazon Associates).
Copied