Which LLMs run on 24 GB of VRAM?

33 of our 61 catalog models fit a 24 GB card at Q4/8K. Fastest on the RTX 4090 24GB: Qwen3-0.6B, gemma-3-1b-it, Llama-3.2-1B-Instruct.

What is the largest model that fits in 24 GB VRAM?

GLM-4.7-Flash at 31.2B — it needs about 18.76 GB at Q4/8K and runs at roughly 213.8 tok/s on the RTX 4090 24GB (estimated).

Best LLMs for 24 GB VRAM GPUs (Q4/8K)

Our catalog lists 5 GPUs with 24 GB VRAM (RTX 3090 Ti 24GB, RTX 4090 24GB, RTX 4090D 24GB (China), RX 7900 XTX 24GB, RTX 3090 24GB). The ranking uses the RTX 4090 24GB — the highest-bandwidth 24 GB card at 1,008 GB/s — and shows the 33 of 61 catalog models that fully fit it at Q4 / 8K context (weights + KV cache + 1 GB overhead within 95% of VRAM), fastest first. Methodology.

#	Model	Params (B)	Q4 tok/s	VRAM needed (GB)	Fit check
1	Qwen3-0.6B	0.8	801.8 est.	1.6	fit check
2	gemma-3-1b-it	1.0	641.5 est.	1.75	fit check
3	Llama-3.2-1B-Instruct	1.2	534.5 est.	1.9	fit check
4	Qwen3-1.7B	2.0	320.7 est.	2.5	fit check
5	gpt-oss-20b	21.5	225.2 measured	13.55	fit check
6	Qwen3-30B-A3B	30.5	213.8 est.	18.38	fit check
7	Qwen3-Coder-30B-A3B-Instruct	30.5	213.8 est.	18.38	fit check
8	GLM-4.7-Flash	31.2	213.8 est.	18.76	fit check
9	Qwen3-30B-A3B-Instruct-2507	30.5	213.8 est.	18.38	fit check
10	Llama-3.2-3B-Instruct	3.2	200.5 est.	3.4	fit check
11	Mistral-7B-Instruct-v0.3	7.2	183.3 measured	6.4	fit check
12	Phi-4-mini-instruct	3.8	168.8 est.	3.85	fit check
13	Phi-4-mini-reasoning	3.8	168.8 est.	3.85	fit check
14	gemma-3-4b-it	4.3	164.5 est.	4.15	fit check
15	Qwen3-4B-Instruct-2507	4.0	160.4 est.	4.0	fit check
16	Qwen3-4B-Thinking-2507	4.0	160.4 est.	4.0	fit check
17	Llama-3.1-8B-Instruct	8.0	145.0 measured	7.0	fit check
18	Qwen3-8B	8.2	104.3 measured	7.15	fit check
19	DeepSeek-R1-Distill-Qwen-14B	14.8	92.5 measured	12.1	fit check
20	Qwen2.5-14B-Instruct	14.8	92.0 measured	12.1	fit check
21	Qwen2.5-7B-Instruct	7.6	84.4 est.	6.7	fit check
22	Qwen2.5-Coder-7B-Instruct	7.6	84.4 est.	6.7	fit check
23	DeepSeek-R1-Distill-Qwen-7B	7.6	84.4 est.	6.7	fit check
24	DeepSeek-R1-0528-Qwen3-8B	8.2	78.2 est.	7.15	fit check
25	Qwen3-14B	14.8	69.1 measured	12.1	fit check
26	gemma-3-12b-it	12.2	53.5 est.	10.11	fit check
27	Phi-4	14.7	43.6 est.	12.03	fit check
28	Phi-4-reasoning	14.7	43.6 est.	12.03	fit check
29	Qwen2.5-Coder-14B-Instruct	14.8	43.3 est.	12.1	fit check
30	Devstral-Small-2507	23.6	27.2 est.	18.7	fit check
31	Mistral-Small-3.2-24B-Instruct-2506	24.0	26.7 est.	19.0	fit check
32	Magistral-Small-2509	24.0	26.7 est.	19.0	fit check
33	gemma-3-27b-it	27.4	23.6 est.	21.51	fit check

tok/s are bandwidth-based estimates unless tagged measured. Every 24 GB card fits the same set of models and ranks them identically; absolute tok/s scales with each card's bandwidth.

Buy the RTX 4090 24GB: GIGABYTE GeForce RTX 4090 Gaming OC 24GB (affiliate) · More listings — Amazon prices change; the price after you click is the live price.

More lists: 7B-class models · 13B-class models · 30B-class models · 70B+ models · Under $500 · Under $1,000 · Under $2,000 · 8 GB VRAM · 12 GB VRAM · 16 GB VRAM · For Qwen3 · For DeepSeek · For Llama · For Gemma · For GLM · For coding · For reasoning · MoE picks

Best models for 24 GB VRAM