Estimates from memory-bandwidth formula; rows marked "measured" override. Methodology.
Fit & speed by quant and context
Quant
Context
VRAM needed
Fits?
tok/s (decode)
Q4
4K
20.37 GB
No
0.3±est.
Q4
8K default
21.37 GB
No
0.2±est.
Q4
32K
27.37 GB
No
—
FP16
4K
67.6 GB
No
—
FP16
8K default
68.6 GB
No
—
FP16
32K
74.6 GB
No
—
Fits = weights + KV(ctx) + 0.5 GB overhead ≤ 95% of VRAM. Measured anchors are context-agnostic; the fit verdict is recomputed per context. A missing tok/s means the model is far beyond this card (offload-only territory).