We treat '70B-class' as open-weight models with over 45B parameters — 24 in our catalog. The table ranks every catalog GPU that fully fits at least one of them at Q4 / 8K context, by average estimated decode tok/s across the models it fits. Fit means weights + KV cache + 1 GB runtime overhead within 95% of VRAM — not weights alone. Methodology.
| # | GPU | VRAM (GB) | Bandwidth (GB/s) | MSRP | Fits | Avg Q4 tok/s | Buy |
|---|---|---|---|---|---|---|---|
| 1 | B200 192GB SXM | 192 | 8,000 | $40,000 | 13/24 | 562.3 incl. measured | — |
| 2 | MI300X 192GB | 192 | 5,300 | $15,000 | 13/24 | 349.7 est. | — |
| 3 | H200 141GB SXM5 | 141 | 4,800 | $35,000 | 11/24 | 349.1 est. | — |
| 4 | H100 80GB SXM5 | 80 | 3,350 | $30,000 | 9/24 | 255.9 est. | — |
| 5 | Gaudi 3 128GB | 128 | 3,276 | $12,500 | 10/24 | 241.2 est. | — |
| 6 | A100 80GB SXM4 | 80 | 2,039 | $15,000 | 9/24 | 154.5 incl. measured | — |
| 7 | RTX 6000 Ada 48GB | 48 | 960 | $6,800 | 2/24 | 125.5 est. | Search Amazon (affiliate) |
| 8 | L40S 48GB | 48 | 864 | $8,000 | 2/24 | 113.0 est. | Search Amazon (affiliate) |
| 9 | RTX A6000 48GB | 48 | 768 | $4,500 | 2/24 | 100.4 est. | Search Amazon (affiliate) |
| 10 | RTX 5090 32GB | 32 | 1,792 | $1,999 | 1/24 | 88.4 est. | Search Amazon (affiliate) |
| 11 | Mac Studio M3 Ultra 96GB | 96 | 819 | $3,999 | 10/24 | 61.0 incl. measured | Search Amazon (affiliate) |
| 12 | Mac Studio M2 Ultra 192GB | 192 | 800 | $6,999 | 13/24 | 51.3 incl. measured | Search Amazon (affiliate) |
| 13 | MacBook Pro M4 Max 128GB | 128 | 546 | $4,399 | 10/24 | 42.0 incl. measured | Search Amazon (affiliate) |
| 14 | MacBook Pro M3 Max 128GB | 128 | 400 | $4,199 | 10/24 | 30.0 incl. measured | Search Amazon (affiliate) |
| 15 | Mac mini M4 Pro 64GB | 64 | 273 | $2,199 | 7/24 | 20.2 incl. measured | Search Amazon (affiliate) |
| 16 | Mac mini M4 32GB | 32 | 120 | $999 | 1/24 | 5.9 est. | Search Amazon (affiliate) |