Your GPU’s VRAM tier—not brand loyalty or the price tag alone—sets a practical limit on which local language models can fit. This guide uses Q4_K_M as its default quantization, but there’s a catch: context length can shift the memory estimate by several gigabytes. The figures below are calculator-based estimates, not results from testing hardware.
The One Formula That Explains Every VRAM Number
VRAM calculators use different assumptions, so the same model can get different estimates. Here are the two formulas side by side:
| Calculator | Formula and assumptions |
|---|---|
| llmhardware.io | VRAM in GB = parameters in billions × bytes per parameter × 1.2 + 2. It uses 0.5 bytes per parameter for Q4_K_M. |
| willitrunllm.com | Parameters × bits per weight ÷ 8, then add 10% and 1 GB for roughly 4K context. It uses 4.85 bits per weight for Q4_K_M and 8.5 for Q8_0. |
For a 7B model at Q4_K_M, the llmhardware.io formula gives 7 × 0.5 × 1.2 + 2 = 6.2 GB. Applying the willitrunllm.com formula gives approximately 5.7 GB after adding its stated overhead and context allowance. Those examples are estimates, not measured peaks.
Across published estimates, the 7B Q4_K_M result spans 5.0–6.2 GB: quantized.uk reports 5.1 GB at 4K context, llamaperf.com lists 5 GB, fitmyllm.com estimates 5.2 GB, and llmhardware.io gives 6.2 GB. This is not a disagreement to average away: context and overhead assumptions differ. The calculators’ estimates can disagree by up to about 1.2 GB at 7B, so check the assumptions before relying on a result.
The Context-Length Caveat (Read This Before Any Table)
A model’s weight file is only part of its memory footprint. When a model handles a longer prompt or conversation, it needs more memory for its key-value (KV) cache. For the same 7B Q4_K_M model, quantized.uk estimates about 5.1 GB at 4K context and about 9.0 GB at 32K context; the KV cache grows from 0.5 GB to 4.0 GB. That difference can turn an apparent fit into an out-of-memory error.
Parameter count alone also does not predict KV-cache size. At 32K context, modelvram.com estimates 8.0 GB of KV cache for Granite 4.2 30B, compared with 0.19 GB for Nemotron 3 Nano 30B-A3B. The architectures and cache requirements differ even though the parameter counts are similar.
ModelVRAM says it checks its predictions against public llama.cpp and vLLM measurements on its accuracy page. Keep this caveat in mind when reading the tier table. Its largest-model estimates use 32K context under modelvram.com’s stated setup, while many other published figures assume shorter context and different headroom.
The 2026 VRAM Tier Table: 8 GB to 128 GB
Q4_K_M is the default reference point below. “Largest model that fits” follows modelvram.com’s estimates at 32K context; the neighboring footprint estimates come from llmhardware.io. GPU examples are omitted because this guide does not have sourced official vendor specifications to list.
| VRAM tier | Largest model that fits (32K context) | Typical Q4_K_M footprint |
|---|---|---|
| 8 GB | LensVLM 9B, 7.43 GB (modelvram.com) | 7–8B: 6.2–6.8 GB (llmhardware.io) |
| 12 GB | Gemma 4 12B, 8.98 GB (modelvram.com) | 14B: 10.4 GB (llmhardware.io) |
| 16 GB | gpt-oss-20b MXFP4, 15.44 GB (modelvram.com) | 14B fits; 27B: 18.2 GB, so it does not fit (llmhardware.io) |
| 24 GB | Ornith 1.5 35B-A3B MoE, 23.47 GB (modelvram.com) | 32B: 21.2 GB; 27B: 18.2 GB (llmhardware.io) |
| 32 GB | K2-Horizon MoVA 36B-A4B, 30.31 GB (modelvram.com) | 32B Q4_K_M: 21.2 GB; Q5: 26.0 GB (llmhardware.io) |
| 48 GB | K2-Horizon MoVA 36B-A4B, 30.31 GB (modelvram.com) | 70B Q4_K_M: about 44 GB (llmhardware.io) |
| 64 GB | No matching modelvram.com tier; see the footnote (modelvram.com) | 70B Q5: 54.5 GB (llmhardware.io) |
| 128 GB | Mistral Medium 3.5 128B, 91.75 GB, on a Mac with 96 GB of unified memory usable by the GPU (modelvram.com) | 70B Q8: 86 GB; FP16: 170 GB (llmhardware.io) |
Table notes: modelvram.com assumes Q4_K_M weights, 32K context, one request, FP16 KV cache, 0.5 GB free and 10% overhead; its figures use GB to mean GiB. Its tier ladder includes 80 GB, not 64 GB: its 80 GB example is Qwen3.5 122B-A10B MoE at 78.85 GB, so there is no directly matching 64 GB tier. llmhardware.io provides the typical footprints in the table; llamaperf.com and quantized.uk assume short contexts with modest headroom, which is why the same model can appear smaller in those sources. All figures are estimates, not measured peaks.
Tier Walkthrough — Entry Level
At 8 GB, a 7–8B Q4_K_M model is a reasonable starting point: llmhardware.io estimates 6.2–6.8 GB. A 3B model at Q8 is estimated at about 3.5 GB by llmhardware.io, leaving more room for context. For model picks in this tier, see ModelVRAM’s 8 GB guide. In modelvram.com’s 32K estimate, the largest listed fit at this tier is LensVLM 9B at 7.43 GB, leaving little spare capacity under its assumptions.
At 12 GB, llmhardware.io estimates 10.4 GB for a 14B Q4_K_M model. The largest listed fit in modelvram.com’s report is Gemma 4 12B at 8.98 GB at 32K context. Those examples use different assumptions, so don’t treat them as interchangeable guarantees. For more models at this tier, see ModelVRAM’s 12 GB guide.
At 16 GB, a 14B Q4 model fits according to llmhardware.io, but the same source estimates 18.2 GB for a 27B Q4 model—too much for that tier. modelvram.com instead identifies gpt-oss-20b MXFP4 at 15.44 GB as its largest listed fit at 32K context. This is a near-miss tier: small differences in model format, context or overhead can matter.
Tier Walkthrough — The 24 GB Tier
At 24 GB, the llmhardware.io estimate for a 32B Q4_K_M model is 21.2 GB. Under modelvram.com’s assumptions, Ornith 1.5 35B-A3B MoE is the largest listed fit at 23.47 GB. At this estimate, a 32B model leaves about 2.8 GB of a 24 GB card unused. For more models at this tier, see ModelVRAM’s 24 GB guide.
At 32 GB, llmhardware.io estimates 26.0 GB for a 32B Q5 model, while a 32B Q8 estimate of 40.4 GB misses the tier. For its 32K estimates, modelvram.com lists K2-Horizon MoVA 36B-A4B at 30.31 GB as the largest fit.
The notable change at 48 GB is not necessarily a larger model. modelvram.com lists the same largest-fitting model at both 32 GB and 48 GB, and reports that the extra memory can extend context from 37K to 115K. Separately, llmhardware.io estimates about 44 GB for a 70B Q4_K_M model, putting that class within reach at this tier.
Tier Walkthrough — Workstation Territory
At 64 GB, llmhardware.io estimates 54.5 GB for a 70B Q5 model. Treat the table’s “largest model” cell as a tier-mismatch disclosure, not a claim that modelvram.com tested a 64 GB tier: that report uses 80 GB for its comparable high-memory entry.
At 128 GB, llmhardware.io estimates 86 GB for a 70B Q8 model and 170 GB for FP16. modelvram.com reports that Mistral Medium 3.5 128B at 91.75 GB fits on a 128 GB Mac with 96 GB usable, under its stated assumptions. These estimates illustrate why “model size” alone is not enough: quantization, cache and usable memory all matter.
Near-Misses and the Offloading Trap
A model that is close to your VRAM limit may still fail to load. For example, llmhardware.io estimates 17.6 GB for a 13B Q8 model, which misses 16 GB, and 40.4 GB for a 32B Q8 model, which misses 32 GB. Leave room for context and runtime overhead rather than treating the card’s full capacity as available for model weights.
There is also a calculation-versus-file-size conflict worth keeping visible: llmhardware.io gives 18.2 GB for Qwen3.8-27B Q4_K_M using its formula, while the same source cites a 17.6 GB Unsloth UD-Q4_K_XL file. These are not the same kind of figure: a formula-based footprint estimate and a specific quantized file size can differ.
Another often-mentioned workaround is CPU offloading, where some model layers use system memory instead of GPU memory. llmhardware.io gives a rule of thumb that offloading 20–30% of layers to the CPU typically drops generation below 5 tok/s. That is the source’s rule of thumb, not our measurement or a promise about every setup.
The Bottom Line
Match your VRAM tier to a model class, not a model name. At 8–12 GB, 7–14B Q4_K_M models are the practical starting range in llmhardware.io’s estimates. At 16 GB, a 27B Q4 estimate of 18.2 GB just misses (llmhardware.io). At 24 GB, a 32B Q4_K_M estimate of 21.2 GB fits with about 2.8 GB to spare (llmhardware.io). At 48 GB, a 70B Q4_K_M estimate of about 44 GB fits, and the extra VRAM can buy more context rather than a larger listed model (llmhardware.io; modelvram.com).
For any other model or quantization, the llmhardware.io VRAM calculator does the arithmetic. Before buying, check the model’s numbers with both formulas from llmhardware.io and willitrunllm.com, then check what context length each estimate assumes. The 7B Q4_K_M figures range from 5.0 to 6.2 GB across published sources, and quantized.uk estimates that the same model grows from about 5.1 GB at 4K to about 9.0 GB at 32K. Use the estimates to narrow your options, not as a guarantee that a model will fit.
This review is based on published VRAM calculators, model file sizes and model file sizes cited by those sources — we did not benchmark these on hardware.
How This Guide Was Built
We did not run any models on hardware, and we performed no benchmarks. The figures come from six published sources: llmhardware.io, updated 2026-10-15; modelvram.com, data version 1.2026-10-20; willitrunllm.com, checked 2026-10-20; llamaperf.com; quantized.uk; and fitmyllm.com.
The numbers are calculator estimates under stated assumptions about context length, KV-cache format and overhead—not measured peaks. modelvram.com specifies one request, labels its results estimates rather than measured peaks, and notes its GB figures use GiB. Where sources conflict, we show the difference rather than averaging it; the 7B range and the Qwen3.8-27B file-size comparison above are examples. This data snapshot is dated 2026-10-21; model files and quantizations can change. Model cards and the Hugging Face API can provide further model-specific information, but they do not remove the need to check calculator assumptions.
FAQ
How much VRAM do I need for a 7B or 8B model at Q4_K_M?
The published estimates span 5.0–6.2 GB. quantized.uk gives 5.1 GB for a 7B model at 4K context, llamaperf.com lists 5 GB, fitmyllm.com estimates 5.2 GB, and llmhardware.io estimates 6.2 GB. The difference reflects context and overhead assumptions, not a single definitive footprint. At 32K context, quantized.uk estimates about 9.0 GB for the same 7B Q4_K_M model.
Does more VRAM always mean a bigger model?
No. modelvram.com lists K2-Horizon MoVA 36B-A4B at 30.31 GB as the largest fit at both 32 GB and 48 GB, while the extra capacity extends its context estimate from 37K to 115K. More VRAM may improve the room available for context instead of changing the largest model that fits.
What does quantization level actually cost in VRAM?
For a 70B model, llmhardware.io estimates about 44 GB at Q4_K_M, 54.5 GB at Q5, 86 GB at Q8 and 170 GB at FP16. For a smaller comparison, llamaperf.com lists 30B Q4 at 18 GB and Q8 at 32 GB. These are estimates under each source’s assumptions, not universal file sizes.
Why do different VRAM calculators give me different answers?
The formulas differ: llmhardware.io uses bytes per parameter plus fixed overhead, while willitrunllm.com uses bits per weight plus percentage overhead and a context allowance. Context assumptions also differ—for example, 4K versus 32K. The calculators can disagree by up to about 1.2 GB at 7B; check each source’s assumptions and treat every output as an estimate, not a measured peak.
📊 See how these tools compare →
📖 Related Reads
- CodeIntel Log — code quality, debugging, and software engineering benchmarks
- ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
Cross-links automatically generated from None.
Back to all posts