gpt-oss

gpt-oss is OpenAI's open-weight model family, released in 2025 — the company's first open-weight language models in years, and an instant community favorite for local inference. The family has exactly two members, gpt-oss-20b and gpt-oss-120b, and both are built around the same idea: large total parameter counts with a very small active footprint per token, thanks to a Mixture-of-Experts (MoE) architecture.

Both models are reasoning-oriented, ship with a 128K context window, and are released under the permissive Apache 2.0 license, so you can run, fine-tune, and redistribute them commercially. GGUF quants are widely available, which makes them a natural fit for llama.cpp, Ollama, and similar local runtimes.

Size ladder

• gpt-oss-20b (21.5B params, MoE with 3.6B active per token, 32 experts / 4 active) — Q4 weights are about 11.8 GB; budget roughly 13.5 GB with KV cache at 8K context, so it lands squarely on a 16GB card. On a 12GB card you can still run it at Q2 (about 6 GB of weights). See gpt-oss-20b estimates
• gpt-oss-120b (120.4B params, MoE with 5.1B active per token, 128 experts / 4 active) — Q4 weights are about 66.2 GB, roughly 68 GB at 8K context, which means a 96GB tier setup (a single 96GB workstation card, or two 48GB cards). A single 48GB card can still run it at Q2, with weights around 33.7 GB. See gpt-oss-120b estimates

What makes gpt-oss different

The headline trait is the tiny active footprint. Although the 120b model holds 120.4B parameters in VRAM, only about 5.1B are used per generated token — so decode speed feels far closer to a small dense model than to a dense 120B. The 20b behaves the same way with 3.6B active. You pay for capacity in VRAM, not in speed.

The second trait is the 128K context window on both sizes. Long context is not free, though: stretching from 8K to the full 128K adds roughly 11.5 GB of KV cache on the 20b and 16 GB on the 120b. If you plan to actually use long documents, size your GPU for that — a 24GB card runs the 20b at Q4 with comfortable long-context headroom, while the same model at 8K fits in 16GB.

Finally, the Apache 2.0 license is about as permissive as it gets: commercial use, modification, and redistribution are all fine, which is one reason GGUF quants and fine-tunes of both models are everywhere.

Which one should you pick

• 12GB VRAM — gpt-oss-20b at Q2 or Q3. Usable, but Q4 is where the model shines; consider this the budget entry point.
• 16GB VRAM — the sweet spot for gpt-oss-20b at Q4 with 8K context. This is the cheapest tier that runs the family at full quality.
• 24GB VRAM — gpt-oss-20b at Q4 with real long-context headroom, or higher quants. The 120b is out of reach here.
• 48GB VRAM — gpt-oss-120b becomes possible at Q2 on a single card, or the 20b at FP16-class weights (about 43 GB) if you want maximum quality from the small model.
• 96GB VRAM — gpt-oss-120b at Q4, either on one workstation card or a dual-48GB rig. This is the practical home of the big model.
• 192GB+ VRAM — headroom for the 120b at long context, higher quants, or FP16 experiments; FP16 weights alone are about 241 GB, so that remains multi-GPU server territory.

For most single-GPU buyers, gpt-oss-20b on a 16GB or 24GB card is the rational choice: near-flagship reasoning behavior at a consumer-friendly footprint. The 120b is for builders who specifically want its extra capability and are willing to pay the 96GB-tier hardware bill.

Links

• gpt-oss-20b — fit and speed estimates
• gpt-oss-120b — fit and speed estimates
• Best MoE models to run locally
• Best models for 16GB GPUs
• Guide: how much VRAM do local LLMs need?
• Guide: LLM quantization explained
• gpt-oss-20b on Hugging Face
• gpt-oss-120b on Hugging Face