Eight gigabytes of VRAM is the floor for local LLM work in 2026, not a comfortable middle. It can run one small model well, and it fails cleanly once you ask for more. This guide is built from official documentation and published third-party benchmarks — we did not run these tests ourselves.
The short answer
An 8 GB card comfortably runs one 7B–9B model at Q4_K_M, with context up to about 8K, at roughly 40–80 tok/s on a 2026-era 8 GB card; no dense model larger than ~9B stays fully in VRAM. The measured 7B-class results on an RTX 5060 8 GB land at 58–73 tok/s, while the file for a 14B model is 9.0 GB before KV cache and a 70B file is 40.20 GiB. Those two facts are the whole story: 8 GB is a single-small-model tier.
How the 8 GB budget adds up
Usable VRAM is about 90% of 8 GB, or 7.2 GB, because the driver, framework, and allocator all take a slice. The budget equation is total = model file + KV cache + overhead. The file-size formula is File size = parameters × bits per weight ÷ 8. Llama 3.1 8B at Q4_K_M works out to 4.58 GiB, matching the published file size, and the practical rule of thumb is about 0.6 GB of VRAM per billion parameters at Q4_K_M. A file quant such as Q4_K_M mixes its base type with larger types for some tensors, so its average is slightly above the base; on Llama 3.1 8B, Q4_K_M measures about 4.89 effective bits per weight. Published builds vary by a few percent between sources — one table gives 4.52 GB for the same model where another gives 4.58 GiB — so treat any single figure as the build it was measured on.
One quantization table, no jargon wall
| Quant | File size | Perplexity increase |
|---|---|---|
| Q2_K | 2.96 GB | +3.5199 |
| Q3_K_S | 3.41 GB | +1.6321 |
| Q3_K_M | 3.74 GB | +0.6569 |
| Q3_K_L | 4.03 GB | +0.5562 |
| Q4_0 | 4.34 GB | +0.4685 |
| Q4_K_S | 4.37 GB | +0.2689 |
| Q4_K_M | 4.58 GB | +0.1754 |
| Q4_1 | 4.78 GB | +0.4511 |
| Q5_0 | 5.21 GB | +0.1316 |
| Q5_K_S | 5.21 GB | +0.1049 |
| Q5_K_M | 5.33 GB | +0.0569 |
| Q5_1 | 5.65 GB | +0.1062 |
| Q6_K | 6.14 GB | +0.0217 |
| Q8_0 | 7.96 GB | +0.0026 |
Reference values from koboldcpp’s GGUF and quantization documentation for Llama-3-8B; these are published reference values, not our measurements. Below Q4, quality drops quickly. Q4_K_M is the common default at about 31 percent of F16 size, and Q5_K_M and Q6_K cost 16 and 34 percent more memory than Q4_K_M, per nanocalcs’ quantization calculator; the same page is where the 4.58 GiB figure above comes from.
What actually fits at 4K, 8K and 16K context
| Model class | Q4_K_M weights | KV @4K | KV @8K | KV @16K | Total | Fits in 8 GB? |
|---|---|---|---|---|---|---|
| Qwen3 8B | 4.68 GB | ~1.0 GB | ~2.0 GB | ~4.0 GB | ~6.0 / ~7.0 / ~9.0 GB | Yes at 4K and 8K; overflows at 16K |
| Llama 3.1 8B | 4.58 GB | not stated | not stated | not stated | not stated | Fits at Q4 (8B class); KV not stated |
| Ornith 1.0 9B | ~5.6 GB | not stated | not stated | ~2.0 GB | not stated | No at 16K; lower contexts not stated |
| K2 Horizon 7B | ~7.4 GB | not stated | not stated | not stated | not stated | Yes at 8K (largest in catalogue) |
| Qwen3 14B | 9.0 GB | not stated | not stated | not stated | not stated | No (file alone exceeds 8 GB) |
| Qwen2.5 32B | not stated | not stated | not stated | not stated | not stated | No (offloads; ESTIMATED 2 tok/s) |
| Llama 3.1 70B | 40.20 GiB | not stated | not stated | not stated | not stated | No |
The Qwen3 8B KV figures are fp16 estimates (ESTIMATED memory, not measured): about 1.0 GB at 4K, 2.0 GB at 8K, and 4.0 GB at 16K, for totals of about 6.0, 7.0, and 9.0 GB. A q8_0 KV cache roughly halves those figures. Most chat interactions stay under 8K context, which is why 8 GB cards survive in practice. Ornith 1.0 9B carries about 5.6 GB of weights and about 2.0 GB of KV at 16K, exceeding the roughly 7 GB usable budget, while K2 Horizon 7B at about 7.4 GB is the largest model among the 61 catalogued models that fit at 8K. Qwen3 14B is 9.0 GB before cache, and Llama 3.1 70B is 40.20 GiB, so neither is a VRAM-resident candidate. Sources: ModelFit’s quantization comparison, ModelFit’s RTX 5060 page, and LLMConfigurator’s RTX 5060 page.
The offload cliff: what happens when it doesn’t fit
Once weights do not fit, layers spill to system RAM and the speed collapse is severe. Documented real-world offload performance is 3–8 tok/s for 14B models versus 31–40 tok/s fully in VRAM on a 16 GB card, a 5–10× penalty. On the 8 GB RTX 5060 Ti, a 14B Q4_K_M model offloads to RAM at under 10 tok/s versus 32.7 tok/s fully in VRAM on the 16 GB variant. An 8 GB card running a half-offloaded 14B model often benchmarks worse than a clean 7B model fully on VRAM. Sources: RunAIHome’s 8 GB vs 16 GB comparison and CraftRigs’ 8 GB vs 16 GB analysis.
The MoE exception
Mixture-of-experts models keep only part of the network active, so an ESTIMATED 20B-class MoE model such as gpt-oss 20B (3.6B active) still reaches an ESTIMATED ~32 tok/s with partial offload on an 8 GB card, while a dense 32B model is ESTIMATED at about 2 tok/s with offload. The full weights still need residency or offload, so MoE is an exception to the size rule, not an exemption from memory. Sources: ModelFit’s RTX 5060 page and the 117-rig dataset.
Measured: how fast a 2026 8 GB card really is
| Model | Params | Size (GB) | Eval rate (tok/s) |
|---|---|---|---|
| deepseek-r1 | 1.5b | 1.1 | 111.06 |
| deepseek-r1 | 7b | 4.7 | 58.46 |
| deepseek-r1 | 8b | 5.2 | 53.91 |
| gemma3n | e2b | 5.6 | 56.11 |
| gemma3n | e4b | 7.5 | 39.51 |
| gemma3 | 1b | 0.815 | 146.64 |
| gemma3 | 4b | 3.3 | 80.72 |
| qwen3 | 0.6b | 0.523 | 210.97 |
| qwen3 | 1.7b | 1.4 | 156.39 |
| qwen3 | 4b | 2.6 | 89.59 |
| qwen3 | 8b | 5.2 | 61.70 |
| llama3.1 | 8b | 4.9 | 58.07 |
| llama3.2 | 1b | 1.3 | 133.49 |
| llama3.2 | 3b | 2.0 | 96.41 |
| phi4-mini | 3.8b | 2.5 | 75.00 |
| phi3.5 | 3.8b | 2.2 | 113.89 |
| phi | 2.7b | 1.6 | 131.47 |
| mistral | 7b | 4.4 | 72.88 |
MEASURED eval rates from Databasemart’s RTX 5060 Ollama benchmark: RTX 5060 8 GB, Ollama 0.9.5, all Q4, 24-core Xeon Platinum 8160, 64 GB RAM, Ubuntu 22, GPU utilisation 54–90%. Even 7B-class models such as Mistral and DeepSeek-R1 worked, albeit at reduced speed (under 75 tok/s). The measured 8B results of 54–62 tok/s (deepseek-r1 8b at 53.91 sits just below that band) sit inside the ESTIMATED 40–83 tok/s range for Llama 3.1 8B on the same card.
Estimates vs measurements: reading speed claims without getting fooled
MEASURED means a number came from a published benchmark table with a named rig; ESTIMATED means it was derived from memory bandwidth or model size, not a timed run. Apply that label to every speed and value number, including table rows and captions.
The bandwidth ceiling for an 8 GB card is ESTIMATED: 448 GB/s ÷ ~4.9 GB (Llama 3.1 8B Q4_K_M) ≈ 91 tok/s at 100% efficiency, and real results run about 60–70% of that. LLMConfigurator’s RTX 5060 page estimates Llama 3.1 8B at 40–83 tok/s on the RTX 5060, 113–234 tok/s on an RTX 5090, and 157–327 tok/s on an H100 80GB SXM. ModelFit’s RTX 5060 page is explicit that its speeds are estimates from memory bandwidth and model size, not measured benchmarks; it also notes that the RTX 5060 Ti 16 GB shares the same 448 GB/s and holds a 14B model in VRAM at an ESTIMATED 41 tok/s. Use ranges, not single numbers, when a source publishes a range.
Performance per dollar: two rankings, one direction
| GPU | VRAM | Launch MSRP | Throughput | Value score | Cohort |
|---|---|---|---|---|---|
| Arc A580 | 8 GB | $179 | ~58 tok/s | 32 | ESTIMATED per-$100 |
| Arc B580 | 12 GB | $249 | ~52 tok/s | 21 | ESTIMATED per-$100 |
| RTX 3060 Ti GDDR6X | 8 GB | $399 | ~69 tok/s | 17 | ESTIMATED per-$100 |
| RTX 5060 | 8 GB | $299 | ~51 tok/s | 17 | ESTIMATED per-$100 |
| RTX 5070 Ti | 16 GB | $749 | ~102 tok/s | 14 | ESTIMATED per-$100 |
| RTX 5060 Ti | 8 GB | $379 | ~51 tok/s | 13 | ESTIMATED per-$100 |
| RTX 3060 | 12 GB | $329 | ~41 tok/s | 12 | ESTIMATED per-$100 |
| RTX 5060 Ti | 16 GB | $429 | ~51 tok/s | 12 | ESTIMATED per-$100 |
| RTX 4060 | 8 GB | $299 | ~31 tok/s | 10 | ESTIMATED per-$100 |
| RTX 5090 | 32 GB | $1,999 | ~203 tok/s | 10 | ESTIMATED per-$100 |
| RTX 3060 | 12 GB | not stated | 42.0 tok/s | 127.66 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 5060 Ti | 16 GB | not stated | 51.4 tok/s | 119.81 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 5070 Ti | 16 GB | not stated | 87.5 tok/s | 116.82 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 5070 | 12 GB | not stated | 59.1 tok/s | 107.65 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 3080 | 10 GB | not stated | 74.2 tok/s | 106.15 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 4070 | 12 GB | not stated | 52.1 tok/s | 94.90 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 5090 | 32 GB | not stated | 145.3 tok/s | 72.69 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 4060 Ti | 16 GB | not stated | 34.3 tok/s | 68.74 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 4090 | 24 GB | not stated | 104.3 tok/s | 65.23 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
| RTX 3090 | 24 GB | not stated | 87.5 tok/s | 58.37 | MEASURED per-$1000 (Qwen3-8B @ Q4_K_XL) |
The two cohorts use different models and quantizations and are never averaged or normalized against each other. The first is ESTIMATED tok/s per $100 at launch MSRP from selfhost.directory’s GPU value page; the second is MEASURED tok/s per $1000 for Qwen3-8B @ Q4_K_XL from compareaihardware’s GPU value index. A used RTX 3090 gives about 3× more generation speed per euro than an RTX 5090 in the 117-rig dataset, and speculative decoding (MTP and friends) gave a median 1.76× speedup across ten same-hardware before/after pairs. Source: the 117-rig dataset.
When to jump to 16 GB instead
16 GB is the upgrade comparison here, not a deep dive. The RTX 5060 Ti 8 GB launched at $379 MSRP and was about $369 mid-2026; the 16 GB variant launched at $429 MSRP and was $499–$540 in June 2026, a real-world gap of roughly $130–$170. Both variants share 4,608 CUDA cores, 2,572 MHz boost, a 128-bit bus, 448 GB/s, and 180 W. A used RTX 3060 12 GB was about $220–250 on eBay in May 2026, at the cost of older, slower GDDR6 memory. MEASURED 16 GB speeds include Llama 3.1 8B at about 41 tok/s, Qwen3 8B at 48–51 tok/s, Qwen3 14B at 31–35 tok/s, and DeepSeek-R1-Distill-14B at 28–32 tok/s. Up to about 20B-class models — gpt-oss 20B as a MoE, plus dense 9B–12B models such as Qwen3.5 9B and Gemma 4 12B — a 16 GB card is the right buy. Sources: RunAIHome, CraftRigs, and the 117-rig dataset.
The 2026 buying verdict for 8 GB
Buy 8 GB if your budget ceiling is about $300, your models stay at 9B or smaller, and your context stays at 8K or less. The RTX 5060 8 GB at a $299 launch MSRP is the reference 2026 8 GB card in this guide; the Arc A580 8 GB shows a higher ESTIMATED 32 tok/s per $100, but that is an estimate from a different method than the MEASURED cohort. Do not buy 8 GB expecting dense 14B models, long context, or headroom: a 14B file is 9.0 GB before cache, and Qwen3 8B overflows 8 GB at 16K. If 14B matters, the cheapest credible answer is more VRAM, not a larger offload.
What this article can’t tell you
- KV cache figures are estimates, not measurements; real KV use varies by model architecture and quantization.
- There are no prompt-processing, time-to-first-token, multi-user, or fine-tuning numbers in the sources used here.
- Beyond the Arc A580 and Arc B580 value rows, there are no AMD, Apple Silicon, or Intel Arc benchmark claims in this article.
- Perplexity deltas are Llama-3-8B reference values only and do not transfer directly to other models.
- IQ4_XS builds can differ by 5–10% in size for the same model, so on a card where a gigabyte decides, use the download size.
- Prices are snapshots with the date windows given by their sources, not live quotes.
FAQ
Can an 8 GB card run Llama 3.1 8B? Yes. Its Q4_K_M file is 4.58 GB, and the measured eval rate on an RTX 5060 8 GB is 58.07 tok/s. KV figures for that model are not stated in the sources.
Can an 8 GB card run a 14B model? Not in VRAM. Qwen3 14B is 9.0 GB before cache, and offloaded 14B models run at 3–8 tok/s versus 31–40 tok/s fully in VRAM.
What quantization should I use on 8 GB? Q4_K_M is the common default with a +0.1754 perplexity increase on the Llama-3-8B reference. Q8_0 is 7.96 GB and leaves almost no room. Below Q4, quality drops quickly.
How much context fits? Most chat stays under 8K. Qwen3 8B totals about 7.0 GB at 8K and about 9.0 GB at 16K, so it overflows at 16K on an 8 GB card.
Are the speeds measured or estimated? Table 3 is MEASURED on a named rig. Table 4 and the bandwidth figures are ESTIMATED and are labelled as such. No number in this article is an unlabelled speed or value.
Should I buy 8 GB or 16 GB? If you need dense 14B models or longer context, 16 GB is the better buy despite a street-price gap of roughly $130–$170 over the 8 GB Ti. If you only need one 7B–9B model, 8 GB is sufficient.
The Bottom Line
| Metric | Score | Basis |
|---|---|---|
| Overall Score | 8/10 | 8 GB runs one 7B–9B model at Q4_K_M and 40–80 tok/s, but no dense model above ~9B stays in VRAM |
| Value | 7/10 | RTX 5060 8 GB: 17 ESTIMATED tok/s per $100 at a $299 launch MSRP, versus 32 for Arc A580 8 GB and 10 for RTX 4060 8 GB |
| Clarity | 9/10 | 18 MEASURED eval-rate rows plus five tables with exact columns |
| Evidence | 9/10 | 12 distinct cited sources, with every speed and value number labelled |
| Scope | 6/10 | Deliberately limited to 8 GB; no 16 GB deep dive, no AMD/Apple/Arc beyond two value rows, no TTFT |
Verdict: 8 GB is a capable single-small-model tier in 2026, not a general-purpose local-AI tier. Buy it for one 7B–9B model at Q4_K_M and 8K context; buy more VRAM for anything larger.
How This Guide Was Built
- Sources: the 12 documentation and benchmark pages linked inline throughout this article, including koboldcpp, nanocalcs, modelvram, ModelFit quant compare, ModelFit RTX 5060, Databasemart, LLMConfigurator, RunAIHome, CraftRigs, selfhost.directory, compareaihardware, and the 117-rig dataset.
- All figures were checked against the saved sources on 2026-11-04.
- This guide is built from official documentation and published third-party benchmarks — we did not run these tests ourselves.
- Every speed and value number is labelled MEASURED or ESTIMATED; prices carry their source date windows; KV figures are estimates.