LOCAL LLM VRAM GUIDE
How much VRAM do you need for a local LLM?
VRAM is the first constraint in local inference. Start with the memory available to your machine, then let the calculator test model size, quantization, context, and workload together.
Weights are only part of the memory budget
A model needs memory for weights, the KV cache that grows with context, and runtime overhead. Quantization reduces weight memory, but a longer context can still push a model over the limit.
Use context as a real input
A 4K chat session and a 32K coding session do not have the same memory requirement. Change the context slider instead of relying on a single VRAM number.
Bandwidth affects the speed after it fits
Two cards can have the same capacity but different bandwidth. Once a model fits, the calculator estimates decode tok/s from bandwidth and measured anchors where available.
Memory brackets in the catalog
| Memory | Representative hardware | Planning note |
|---|---|---|
| 8 GB | RTX 4060 8GB · RX 7600 8GB | Start with smaller models and shorter context; verify the fit result. |
| 16 GB | RTX 5060 Ti 16GB · RX 7800 XT 16GB | A practical capacity band; test quantization and context together. |
| 24 GB | RTX 4090 24GB · RTX 3090 24GB | More room for larger quantized models or a longer context. |
| 48 GB | L40S 48GB · RTX 6000 Ada 48GB | A wider memory envelope for larger models and long context. |
| 128 GB | MacBook Pro M4 Max 128GB · Gaudi 3 128GB | For very large local models; bandwidth still affects decode speed. |
These are representative catalog inputs, not a promise that every model in the bracket will fit. Open a hardware link to test its exact configuration.
Ready to test your machine?
Enter a GPU or usable VRAM and the calculator will rank models for your workload, quantization, and context.
Open the calculator