What Can You Actually Run on 8 GB of VRAM? A 2026 Reality Check
· 13 min read

What Can You Actually Run on 8 GB of VRAM? A 2026 Reality Check

A practical, source-checked walkthrough of the 8 GB VRAM tier for local LLMs: how quantization math sets file sizes, what fits at 4K, 8K and 16K context, what measured throughput looks like on a modern 8 GB card, and when the extra VRAM is worth paying for.

Eight gigabytes of VRAM is the floor for local LLM work in 2026, not a comfortable middle. It can run one small model well, and it fails cleanly once you ask for more. This guide is built from official documentation and published third-party benchmarks — we did not run these tests ourselves.

The short answer

An 8 GB card comfortably runs one 7B–9B model at Q4_K_M, with context up to about 8K, at roughly 40–80 tok/s on a 2026-era 8 GB card; no dense model larger than ~9B stays fully in VRAM. The measured 7B-class results on an RTX 5060 8 GB land at 58–73 tok/s, while the file for a 14B model is 9.0 GB before KV cache and a 70B file is 40.20 GiB. Those two facts are the whole story: 8 GB is a single-small-model tier.

How the 8 GB budget adds up

Usable VRAM is about 90% of 8 GB, or 7.2 GB, because the driver, framework, and allocator all take a slice. The budget equation is total = model file + KV cache + overhead. The file-size formula is File size = parameters × bits per weight ÷ 8. Llama 3.1 8B at Q4_K_M works out to 4.58 GiB, matching the published file size, and the practical rule of thumb is about 0.6 GB of VRAM per billion parameters at Q4_K_M. A file quant such as Q4_K_M mixes its base type with larger types for some tensors, so its average is slightly above the base; on Llama 3.1 8B, Q4_K_M measures about 4.89 effective bits per weight. Published builds vary by a few percent between sources — one table gives 4.52 GB for the same model where another gives 4.58 GiB — so treat any single figure as the build it was measured on.

One quantization table, no jargon wall

QuantFile sizePerplexity increase
Q2_K2.96 GB+3.5199
Q3_K_S3.41 GB+1.6321
Q3_K_M3.74 GB+0.6569
Q3_K_L4.03 GB+0.5562
Q4_04.34 GB+0.4685
Q4_K_S4.37 GB+0.2689
Q4_K_M4.58 GB+0.1754
Q4_14.78 GB+0.4511
Q5_05.21 GB+0.1316
Q5_K_S5.21 GB+0.1049
Q5_K_M5.33 GB+0.0569
Q5_15.65 GB+0.1062
Q6_K6.14 GB+0.0217
Q8_07.96 GB+0.0026

Reference values from koboldcpp’s GGUF and quantization documentation for Llama-3-8B; these are published reference values, not our measurements. Below Q4, quality drops quickly. Q4_K_M is the common default at about 31 percent of F16 size, and Q5_K_M and Q6_K cost 16 and 34 percent more memory than Q4_K_M, per nanocalcs’ quantization calculator; the same page is where the 4.58 GiB figure above comes from.

What actually fits at 4K, 8K and 16K context

Model classQ4_K_M weightsKV @4KKV @8KKV @16KTotalFits in 8 GB?
Qwen3 8B4.68 GB~1.0 GB~2.0 GB~4.0 GB~6.0 / ~7.0 / ~9.0 GBYes at 4K and 8K; overflows at 16K
Llama 3.1 8B4.58 GBnot statednot statednot statednot statedFits at Q4 (8B class); KV not stated
Ornith 1.0 9B~5.6 GBnot statednot stated~2.0 GBnot statedNo at 16K; lower contexts not stated
K2 Horizon 7B~7.4 GBnot statednot statednot statednot statedYes at 8K (largest in catalogue)
Qwen3 14B9.0 GBnot statednot statednot statednot statedNo (file alone exceeds 8 GB)
Qwen2.5 32Bnot statednot statednot statednot statednot statedNo (offloads; ESTIMATED 2 tok/s)
Llama 3.1 70B40.20 GiBnot statednot statednot statednot statedNo

The Qwen3 8B KV figures are fp16 estimates (ESTIMATED memory, not measured): about 1.0 GB at 4K, 2.0 GB at 8K, and 4.0 GB at 16K, for totals of about 6.0, 7.0, and 9.0 GB. A q8_0 KV cache roughly halves those figures. Most chat interactions stay under 8K context, which is why 8 GB cards survive in practice. Ornith 1.0 9B carries about 5.6 GB of weights and about 2.0 GB of KV at 16K, exceeding the roughly 7 GB usable budget, while K2 Horizon 7B at about 7.4 GB is the largest model among the 61 catalogued models that fit at 8K. Qwen3 14B is 9.0 GB before cache, and Llama 3.1 70B is 40.20 GiB, so neither is a VRAM-resident candidate. Sources: ModelFit’s quantization comparison, ModelFit’s RTX 5060 page, and LLMConfigurator’s RTX 5060 page.

The offload cliff: what happens when it doesn’t fit

Once weights do not fit, layers spill to system RAM and the speed collapse is severe. Documented real-world offload performance is 3–8 tok/s for 14B models versus 31–40 tok/s fully in VRAM on a 16 GB card, a 5–10× penalty. On the 8 GB RTX 5060 Ti, a 14B Q4_K_M model offloads to RAM at under 10 tok/s versus 32.7 tok/s fully in VRAM on the 16 GB variant. An 8 GB card running a half-offloaded 14B model often benchmarks worse than a clean 7B model fully on VRAM. Sources: RunAIHome’s 8 GB vs 16 GB comparison and CraftRigs’ 8 GB vs 16 GB analysis.

The MoE exception

Mixture-of-experts models keep only part of the network active, so an ESTIMATED 20B-class MoE model such as gpt-oss 20B (3.6B active) still reaches an ESTIMATED ~32 tok/s with partial offload on an 8 GB card, while a dense 32B model is ESTIMATED at about 2 tok/s with offload. The full weights still need residency or offload, so MoE is an exception to the size rule, not an exemption from memory. Sources: ModelFit’s RTX 5060 page and the 117-rig dataset.

Measured: how fast a 2026 8 GB card really is

ModelParamsSize (GB)Eval rate (tok/s)
deepseek-r11.5b1.1111.06
deepseek-r17b4.758.46
deepseek-r18b5.253.91
gemma3ne2b5.656.11
gemma3ne4b7.539.51
gemma31b0.815146.64
gemma34b3.380.72
qwen30.6b0.523210.97
qwen31.7b1.4156.39
qwen34b2.689.59
qwen38b5.261.70
llama3.18b4.958.07
llama3.21b1.3133.49
llama3.23b2.096.41
phi4-mini3.8b2.575.00
phi3.53.8b2.2113.89
phi2.7b1.6131.47
mistral7b4.472.88

MEASURED eval rates from Databasemart’s RTX 5060 Ollama benchmark: RTX 5060 8 GB, Ollama 0.9.5, all Q4, 24-core Xeon Platinum 8160, 64 GB RAM, Ubuntu 22, GPU utilisation 54–90%. Even 7B-class models such as Mistral and DeepSeek-R1 worked, albeit at reduced speed (under 75 tok/s). The measured 8B results of 54–62 tok/s (deepseek-r1 8b at 53.91 sits just below that band) sit inside the ESTIMATED 40–83 tok/s range for Llama 3.1 8B on the same card.

Estimates vs measurements: reading speed claims without getting fooled

MEASURED means a number came from a published benchmark table with a named rig; ESTIMATED means it was derived from memory bandwidth or model size, not a timed run. Apply that label to every speed and value number, including table rows and captions.

The bandwidth ceiling for an 8 GB card is ESTIMATED: 448 GB/s ÷ ~4.9 GB (Llama 3.1 8B Q4_K_M) ≈ 91 tok/s at 100% efficiency, and real results run about 60–70% of that. LLMConfigurator’s RTX 5060 page estimates Llama 3.1 8B at 40–83 tok/s on the RTX 5060, 113–234 tok/s on an RTX 5090, and 157–327 tok/s on an H100 80GB SXM. ModelFit’s RTX 5060 page is explicit that its speeds are estimates from memory bandwidth and model size, not measured benchmarks; it also notes that the RTX 5060 Ti 16 GB shares the same 448 GB/s and holds a 14B model in VRAM at an ESTIMATED 41 tok/s. Use ranges, not single numbers, when a source publishes a range.

Performance per dollar: two rankings, one direction

GPUVRAMLaunch MSRPThroughputValue scoreCohort
Arc A5808 GB$179~58 tok/s32ESTIMATED per-$100
Arc B58012 GB$249~52 tok/s21ESTIMATED per-$100
RTX 3060 Ti GDDR6X8 GB$399~69 tok/s17ESTIMATED per-$100
RTX 50608 GB$299~51 tok/s17ESTIMATED per-$100
RTX 5070 Ti16 GB$749~102 tok/s14ESTIMATED per-$100
RTX 5060 Ti8 GB$379~51 tok/s13ESTIMATED per-$100
RTX 306012 GB$329~41 tok/s12ESTIMATED per-$100
RTX 5060 Ti16 GB$429~51 tok/s12ESTIMATED per-$100
RTX 40608 GB$299~31 tok/s10ESTIMATED per-$100
RTX 509032 GB$1,999~203 tok/s10ESTIMATED per-$100
RTX 306012 GBnot stated42.0 tok/s127.66MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 5060 Ti16 GBnot stated51.4 tok/s119.81MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 5070 Ti16 GBnot stated87.5 tok/s116.82MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 507012 GBnot stated59.1 tok/s107.65MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 308010 GBnot stated74.2 tok/s106.15MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 407012 GBnot stated52.1 tok/s94.90MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 509032 GBnot stated145.3 tok/s72.69MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 4060 Ti16 GBnot stated34.3 tok/s68.74MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 409024 GBnot stated104.3 tok/s65.23MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)
RTX 309024 GBnot stated87.5 tok/s58.37MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL)

The two cohorts use different models and quantizations and are never averaged or normalized against each other. The first is ESTIMATED tok/s per $100 at launch MSRP from selfhost.directory’s GPU value page; the second is MEASURED tok/s per $1000 for Qwen3-8B @ Q4_K_XL from compareaihardware’s GPU value index. A used RTX 3090 gives about 3× more generation speed per euro than an RTX 5090 in the 117-rig dataset, and speculative decoding (MTP and friends) gave a median 1.76× speedup across ten same-hardware before/after pairs. Source: the 117-rig dataset.

When to jump to 16 GB instead

16 GB is the upgrade comparison here, not a deep dive. The RTX 5060 Ti 8 GB launched at $379 MSRP and was about $369 mid-2026; the 16 GB variant launched at $429 MSRP and was $499–$540 in June 2026, a real-world gap of roughly $130–$170. Both variants share 4,608 CUDA cores, 2,572 MHz boost, a 128-bit bus, 448 GB/s, and 180 W. A used RTX 3060 12 GB was about $220–250 on eBay in May 2026, at the cost of older, slower GDDR6 memory. MEASURED 16 GB speeds include Llama 3.1 8B at about 41 tok/s, Qwen3 8B at 48–51 tok/s, Qwen3 14B at 31–35 tok/s, and DeepSeek-R1-Distill-14B at 28–32 tok/s. Up to about 20B-class models — gpt-oss 20B as a MoE, plus dense 9B–12B models such as Qwen3.5 9B and Gemma 4 12B — a 16 GB card is the right buy. Sources: RunAIHome, CraftRigs, and the 117-rig dataset.

The 2026 buying verdict for 8 GB

Buy 8 GB if your budget ceiling is about $300, your models stay at 9B or smaller, and your context stays at 8K or less. The RTX 5060 8 GB at a $299 launch MSRP is the reference 2026 8 GB card in this guide; the Arc A580 8 GB shows a higher ESTIMATED 32 tok/s per $100, but that is an estimate from a different method than the MEASURED cohort. Do not buy 8 GB expecting dense 14B models, long context, or headroom: a 14B file is 9.0 GB before cache, and Qwen3 8B overflows 8 GB at 16K. If 14B matters, the cheapest credible answer is more VRAM, not a larger offload.

What this article can’t tell you

  • KV cache figures are estimates, not measurements; real KV use varies by model architecture and quantization.
  • There are no prompt-processing, time-to-first-token, multi-user, or fine-tuning numbers in the sources used here.
  • Beyond the Arc A580 and Arc B580 value rows, there are no AMD, Apple Silicon, or Intel Arc benchmark claims in this article.
  • Perplexity deltas are Llama-3-8B reference values only and do not transfer directly to other models.
  • IQ4_XS builds can differ by 5–10% in size for the same model, so on a card where a gigabyte decides, use the download size.
  • Prices are snapshots with the date windows given by their sources, not live quotes.

FAQ

Can an 8 GB card run Llama 3.1 8B? Yes. Its Q4_K_M file is 4.58 GB, and the measured eval rate on an RTX 5060 8 GB is 58.07 tok/s. KV figures for that model are not stated in the sources.

Can an 8 GB card run a 14B model? Not in VRAM. Qwen3 14B is 9.0 GB before cache, and offloaded 14B models run at 3–8 tok/s versus 31–40 tok/s fully in VRAM.

What quantization should I use on 8 GB? Q4_K_M is the common default with a +0.1754 perplexity increase on the Llama-3-8B reference. Q8_0 is 7.96 GB and leaves almost no room. Below Q4, quality drops quickly.

How much context fits? Most chat stays under 8K. Qwen3 8B totals about 7.0 GB at 8K and about 9.0 GB at 16K, so it overflows at 16K on an 8 GB card.

Are the speeds measured or estimated? Table 3 is MEASURED on a named rig. Table 4 and the bandwidth figures are ESTIMATED and are labelled as such. No number in this article is an unlabelled speed or value.

Should I buy 8 GB or 16 GB? If you need dense 14B models or longer context, 16 GB is the better buy despite a street-price gap of roughly $130–$170 over the 8 GB Ti. If you only need one 7B–9B model, 8 GB is sufficient.

The Bottom Line

MetricScoreBasis
Overall Score8/108 GB runs one 7B–9B model at Q4_K_M and 40–80 tok/s, but no dense model above ~9B stays in VRAM
Value7/10RTX 5060 8 GB: 17 ESTIMATED tok/s per $100 at a $299 launch MSRP, versus 32 for Arc A580 8 GB and 10 for RTX 4060 8 GB
Clarity9/1018 MEASURED eval-rate rows plus five tables with exact columns
Evidence9/1012 distinct cited sources, with every speed and value number labelled
Scope6/10Deliberately limited to 8 GB; no 16 GB deep dive, no AMD/Apple/Arc beyond two value rows, no TTFT

Verdict: 8 GB is a capable single-small-model tier in 2026, not a general-purpose local-AI tier. Buy it for one 7B–9B model at Q4_K_M and 8K context; buy more VRAM for anything larger.

How This Guide Was Built

Back to all posts