V100 32GB SXM2

NVIDIA · 32 GB HBM2 · 900 GB/s · 300 W · MSRP $10,000

The V100 32GB SXM2 is a NVIDIA accelerator with 32 GB HBM2 and about 900 GB/s peak memory bandwidth. AI Hashrate estimates local LLM decode (batch 1) at 8K context: fit counts weights + KV cache + 1 GB runtime overhead — not weights alone. At Q4, about 43 curated models fit fully on this card; at FP16, about 18 fit. Top Q4 speeds: MiniCPM5-1B ≈ 530.3 tok/s (estimated, fits); Qwen3.5-2B ≈ 286.4 tok/s (estimated, fits); DeepSeek-R1-0528-Qwen3-8B-layer-mix-bpw-3.8-mlx ≈ 249.0 tok/s (estimated, fits); Qwen3.6-35B-A3B ≈ 190.9 tok/s (estimated, fits); Qwen3.5-35B-A3B ≈ 190.9 tok/s (estimated, fits). Relative ranking is more reliable than absolute tok/s. See methodology for the bandwidth formula and measured-anchor policy. Methodology.

Table: decode speed estimates (Q4 / FP16). Measured rows override estimates. MSRP is list price, not live retail. Context default 8K.

Fit checks

• Qwen3-8B — fits · ≈69.8 tok/s Q4 est.
• gemma-4-26B-A4B-it — fits · ≈150.7 tok/s Q4 est.
• Qwen2.5-7B-Instruct — fits · ≈75.4 tok/s Q4 est.
• Qwen3.5-9B — fits · ≈64.0 tok/s Q4 est.
• Qwen3.6-35B-A3B-NVFP4 — fits · ≈26.5 tok/s Q4 est.
• Llama-3.1-8B-Instruct — fits · ≈71.6 tok/s Q4 est.
• Qwen3.6-35B-A3B-FP8 — fits · ≈15.9 tok/s Q4 est.
• gpt-oss-20b — fits · ≈159.1 tok/s Q4 est.
• Qwen3.6-35B-A3B — fits · ≈190.9 tok/s Q4 est.
• Qwen3.5-35B-A3B — fits · ≈190.9 tok/s Q4 est.
• gemma-4-E2B-it — fits · ≈112.3 tok/s Q4 est.

All 73 model fit checks — see the linked Fits? column in the table above