Nemotron

Nemotron is NVIDIA's own open-weights LLM line, and the current Nemotron 3 generation is built around one idea: a hybrid Mamba-Transformer architecture paired with MoE routing, so you get very long context and fast decoding without paying full dense-model compute per token. The family ships in four sizes — Nano 4B, Nano-Omni 30B-A3B, Super 120B-A12B, and Ultra 550B-A55B — and every one of them runs a 256K context window. All weights are published under NVIDIA's open model licenses (NVIDIA Open Model License for most of the lineup, OpenMDW-1.1 for the Ultra).

Because three of the four models are MoE, the number that matters for speed is the active parameter count per token, while the number that matters for VRAM is the total parameter count — you still have to hold all the experts in memory. Our estimates below use the site formula: Q4 VRAM at 8K context ≈ Q4 weights + 0.025 GB per active billion × 8 + 1 GB overhead. Longer contexts grow the KV cache line on top of that.

Size ladder

• NVIDIA-Nemotron-3-Nano-4B-BF16 (4B dense) — Q4 ≈ 4 GB total at 8K ctx → any 12GB GPU is massive overkill; even 8GB cards have headroom. This is the tiny always-on assistant tier. Model page
• Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 (30B MoE, 3B active) — Q4 ≈ 18 GB at 8K ctx → a clean fit on a 24GB GPU (RTX 3090/4090 class). Only 3B active per token means it decodes fast for its size, and it's the reasoning-focused omni variant of the family. Model page
• NVIDIA-Nemotron-3-Super-120B-A12B-BF16 (120B MoE, 12B active) — Q4 ≈ 69 GB at 8K ctx → you're in 96GB territory: a single RTX 6000-class workstation card or dual 48GB GPUs. Speeds feel closer to a 12B dense model, which is remarkable for this capability tier. Model page
• NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 (550B MoE, 55B active) — Q4 ≈ 315 GB at 8K ctx → no single consumer GPU comes close; this is 192GB+ multi-GPU territory, realistically a 4×96GB+ rig or a small datacenter node. The published checkpoint is NVFP4, signaling it's meant for serious inference stacks, not a gaming PC. Model page

What makes the family stand out

Three things come straight out of the spec sheet. First, MoE everywhere that matters: Super runs 512 experts with 22 active per token, Nano-Omni 128 experts with 6 active — quality of a big model, decode cost of a small one. Second, uniform 256K context across all four sizes, which is where the hybrid Mamba layers pay off: KV cache growth at long context is far gentler than a pure Transformer. Third, permissive open licensing directly from NVIDIA, which makes these safe picks for commercial products. FP16/BF16 full-precision fans should note the budget: 8 GB (Nano 4B), 60 GB (Nano-Omni), 240 GB (Super), and roughly 1.1 TB (Ultra) just for weights.

Which one should you pick

• 12GB VRAM or less: Nano 4B at Q4 with room to stretch context. Simple choice.
• 24GB (RTX 3090/4090): Nano-Omni 30B-A3B is the sweet spot of the whole family — reasoning-tuned, fast, and it fits with headroom for longer contexts. This is the card-to-model pairing we'd recommend most people build around.
• 48GB (dual 24GB or one 48GB card): stuck between tiers; Super 120B still won't fit at Q4. Either accept tighter quants or plan for 96GB.
• 96GB: Super 120B-A12B at Q4 is a genuinely flagship-class local model, and 12B active keeps it interactive.
• 192GB+: only here does Ultra 550B enter the conversation — and at ~315 GB you'll need several big-memory GPUs working together. If you have to ask, the Super is your ceiling.

Links

• How much VRAM do local LLMs need?
• Best GPUs for MoE models
• Best GPUs for reasoning LLMs
• NVIDIA Nemotron on Hugging Face