Running open-weight large language models locally appeals to both developers and privacy-conscious Swiss SMBs determined to keep sensitive documents strictly on-premises. Yet the gap between social-media hype and PCIe memory physics catches many buyers off guard: a "compact" model suddenly runs out of 16 GB of VRAM as soon as you feed it a twenty-page contract, while a Mixture-of-Experts (MoE) model advertised with "only 3.3 billion active parameters" refuses to load on a laptop GPU.
Here is our verified late-2026 engineering breakdown: how much VRAM frontier open-source models actually require, what genuinely runs smoothly on a mid-range vs. high-end gaming PC, smart home hardware options beyond 32 GB, and real-world electricity costs in Swiss francs (CHF) using official ElCom tariffs.
1. The real VRAM formula — and the Mixture-of-Experts memory trap
To achieve responsive token streaming without spilling onto slow CPU RAM, three components must fit inside video memory simultaneously:
Total VRAM (GB) ≈ (Total parameters in billions × (Bits / 8) × 1.12) + KV Cache (context) + Engine overhead (0.8 to 1.5 GB)
The 1.12 multiplier accounts for quantization block scales (GGUF Q4_K_M or MXFP4) and higher-precision attention tensors. Above all, never confuse total parameters with active parameters on a Mixture-of-Experts (MoE) model:
- Total parameters determine required memory (VRAM): every expert's weights must stay resident in memory because each generated token can route to different experts.
- Active parameters determine compute speed (tokens/s) and watts per token:
Qwen3-30B-A3Bneeds ~21 GB of memory just like a 30B model, yet generates each token as fast and efficiently as a tiny 3.3B model!
Equally important is the KV cache (Key-Value cache), which grows linearly with context length: expanding context from 4,096 tokens to 32,768 or 65,536 tokens easily adds 2 to 6 GB of extra VRAM (quantizing the KV cache to q8_0 in Ollama or llama.cpp cuts that overhead nearly in half with negligible quality loss).
2. VRAM requirements for flagship open-weight models in 2026
| Open-weight model | Architecture (total / active) | FP16 weights | Real 4-bit VRAM (Q4_K_M / MXFP4 + 16K KV) | Minimum smooth hardware |
|---|---|---|---|---|
| Qwen3-8B / R1-Distill-8B | Dense (8.2B) | ~16.5 GB | ~6.7 GB | 8 GB to 12 GB GPU |
| Qwen3-14B / R1-Distill-14B | Dense (14.8B) | ~29.6 GB | ~10.8 GB | RTX 3060 / 4070 / 5070 (12 GB) |
| openai/gpt-oss-20b | MoE (21B / 3.6B active) | Native MXFP4 | ~14.5 GB | 16 GB GPU (RTX 4060 Ti / 5060 Ti / 5080) |
| Gemma 3 27B / Mistral Small 24B | Dense (24B – 27B) | ~48 – 54 GB | ~18.0 – 19.5 GB | RTX 3090 / 4090 / RX 7900 XTX (24 GB) |
| Qwen3-30B-A3B | MoE (30.5B / 3.3B active) | ~61 GB | ~21.0 GB | 24 GB GPU (>110 tok/s) or 32 GB |
| Qwen3-32B | Dense (32.8B) | ~65.6 GB | ~23.5 GB (16K) / ~26 GB (64K) | Tight on 24 GB · Ideal on RTX 5090 (32 GB) |
| Llama 3.3 70B Instruct | Dense (70.6B) | ~141 GB | ~46.0 GB | Dual 24 GB GPUs or 64 GB unified memory |
| openai/gpt-oss-120b | MoE (117B / 5.1B active) | Native MXFP4 | ~70.0 GB | 1× H100 80 GB or 96–128 GB unified RAM |
| Qwen3-235B-A22B | MoE (235B / 22B active) | ~470 GB | ~148 GB | 192 GB+ Mac Studio or multi-GPU server |
| DeepSeek-R1 / V3 (full) | MoE (671B / 37B active) | ~1,340 GB | ~420 GB | Datacenter cluster only (or cloud API) |
3. Mid-range vs. high-end gaming PCs: what can you realistically run at home?
Mid-range gaming PCs (12 GB to 16 GB VRAM)
- 12 GB tier (RTX 4070, RTX 4070 Super, RTX 5070, RTX 3060 12 GB): runs Qwen3-8B at near-lossless
Q8_0precision or Qwen3-14B / DeepSeek-R1-Distill-14B inQ4_K_Mwith an 8K–16K context window at 45–65 tokens/s. Great for private drafting, local summarization, and IDE autocomplete, though constrained for multi-step agentic tool loops. - 16 GB tier (RTX 5060 Ti 16 GB, RTX 4070 Ti Super, RTX 5080, RX 7800 XT): the best budget sweet spot in 2026 thanks to
openai/gpt-oss-20b(Apache 2.0, post-trained natively in MXFP4), which fits comfortably inside 16 GB while only activating 3.6B parameters per token.
High-end and enthusiast gaming PCs (24 GB to 32 GB VRAM)
- 24 GB tier (used RTX 3090, RTX 4090, Radeon RX 7900 XTX): still the value champion for local inference.
Qwen3-30B-A3Bscreams past 110 tokens/s with on-demand/thinkand/no_thinkswitching, while Gemma 3 27B handles local multimodal document vision. Dense Qwen3-32B fits inQ4_K_Mas long as context stays around 8K–16K. - 32 GB tier (NVIDIA GeForce RTX 5090): backed by 32 GB of GDDR7 (~1.79 TB/s bandwidth), the RTX 5090 unlocks Qwen3-32B with a generous 64K context window or higher
Q6_Kquantization. However, let's be crystal clear: even an RTX 5090 cannot fit a 70B dense model orgpt-oss-120bpurely in VRAM without spilling layers onto system DDR5 RAM (dragging dense generation down to 5–8 tokens/s).
Going beyond 32 GB at home: unified memory and dual-GPU builds
If you want to host gpt-oss-120b (~70 GB) or Qwen3-235B-A22B (~148 GB) quietly at home or in a small office, the smartest machine is not a gaming rig — it is a high-bandwidth unified memory workstation such as an Apple Mac Studio (M2/M3/M4 Max or Ultra with 96 to 192 GB) or a compact AMD Ryzen AI Max+ 395 ("Strix Halo", 128 GB LPDDR5X) mini-PC. Because sparse MoE models only read 5B to 22B active weights per token, unified memory runs them at 20 to 45 tokens/s under 110 watts! On the PC side, the proven enthusiast alternative is pairing two used 24 GB RTX 3090 cards (48 GB total VRAM) on a dual-x8 motherboard with a 1,200 W power supply.
4. Model energy efficiency and real electricity costs in Switzerland (ElCom 2026)
According to the Swiss Federal Electricity Commission (ElCom), the median Swiss household electricity tariff for 2026 (standard H4 profile, 4,500 kWh/year) is 27.7 centimes per kWh (CHF 0.277/kWh) — down roughly 4% from 29.0 ct./kWh in 2025, with municipal tariffs spanning ~9.6 ct./kWh to ~43.6 ct./kWh across cantons.
Here is what running local AI actually adds to a Swiss power bill at the wall:
| Local hardware setup | Wall power draw (active / idle) | Home / dev use (2 h/day active + 6 h idle) | Office server (8 h/day active + 16 h idle) | Electricity per 1M generated tokens |
|---|---|---|---|---|
| Mac Studio / 128 GB unified mini-PC (e.g. gpt-oss-120b or Qwen3-30B-A3B) | ~95 W / ~12 W | ~8.0 kWh/mo ~CHF 2.20/mo (~CHF 26/yr) | ~31.7 kWh/mo ~CHF 8.80/mo (~CHF 105/yr) | ~0.65 to 1.20 kWh (CHF 0.18 – 0.33) |
| Mid-range gaming PC (RTX 5070 12 GB / 5060 Ti 16 GB) | ~290 W / ~55 W | ~27.7 kWh/mo ~CHF 7.65/mo (~CHF 92/yr) | ~97.3 kWh/mo ~CHF 27.00/mo (~CHF 324/yr) | ~1.20 to 1.60 kWh (CHF 0.33 – 0.44) |
| High-end gaming PC (RTX 4090 24 GB) | ~480 W / ~65 W | ~41.1 kWh/mo ~CHF 11.40/mo (~CHF 136/yr) | ~148.4 kWh/mo ~CHF 41.10/mo (~CHF 493/yr) | MoE: ~0.90 kWh (CHF 0.25) Dense 32B: ~3.5 kWh (CHF 0.97) |
| Enthusiast flagship PC (RTX 5090 32 GB, 575 W GPU TDP) | ~650 W / ~70 W | ~52.3 kWh/mo ~CHF 14.50/mo (~CHF 174/yr) | ~192.2 kWh/mo ~CHF 53.25/mo (~CHF 639/yr) | MoE: ~0.95 kWh (CHF 0.26) Dense 32B: ~3.8 kWh (CHF 1.05) |
Two practical conclusions stand out for Swiss users and CTOs:
- MoE architectures cut electricity per task by nearly 4×: on an RTX 4090, generating one million tokens with
Qwen3-30B-A3B(~130 tok/s) costs roughly CHF 0.25 in Swiss electricity, compared with nearly CHF 1.00 on a dense 32B model running at a third of the speed. - Hardware depreciation, not Swiss electricity, dominates TCO: CHF 14.50/month on an RTX 5090 is minor next to a CHF 3,500–4,500 workstation. Below 15–20 million tokens per month, a governed cloud endpoint in Zurich (
europe-west6) is cheaper overall; above that volume — or under strict air-gapped confidentiality rules — local inference pays back its hardware within 12 to 18 months.
Technical sources
- ElCom (Swiss Federal Electricity Commission) — 2026 Swiss Electricity Tariffs (27.7 ct./kWh median)
- OpenAI — Official
gpt-oss-120b&gpt-oss-20bModel Card (MXFP4, Apache 2.0) - Qwen Team — Official
Qwen3-30B-A3B& Qwen3 Technical Documentation - llama.cpp / GGML — GGUF Quantization & KV Cache Memory Reference
Weighing an on-premises GPU/unified-memory server in Switzerland against a sovereign Google Cloud Zurich deployment or hybrid routing in 21brain? Book a sizing review with our AI engineers or contact 21datas.
in Data