What Can You Actually Run on 12 GB VRAM in 2026
· 14 min read

What Can You Actually Run on 12 GB VRAM in 2026

Which local models fit on a 12 GB graphics card, why published speed numbers for the same card disagree, and whether a 16 GB card is the better buy.

What Can You Actually Run on 12 GB VRAM in 2026

12 GB in 2026: the awkward middle tier

Twelve gigabytes of VRAM is enough for useful local models and not enough for the larger ones. If you go shopping for 12 GB today, these are the three cards the sources in this guide price: the RTX 3060 12GB at $230, the RTX 4070 at $450, and the RTX 5070 at $650, all priced in hardware-corner.net’s comparison.

This guide answers three questions in order:

  1. What fits in 12 GB, and at which quantization?
  2. Why do published speeds for the same card disagree by roughly 3x or more?
  3. Is 12 GB worth buying in 2026, or is a 16 GB card the better purchase?

The three 12 GB cards, side by side

The table below collects every published figure hardware-corner.net reports for the three cards.

MetricRTX 3060 12GBRTX 4070RTX 5070
VRAM / memory type12 GB GDDR612 GB GDDR6X12 GB GDDR7
Memory bandwidth360 GB/s504 GB/s672 GB/s
ArchitectureAmpereAdaBlackwell
TDP170 W200 W250 W
Price$230$450$650
Price per GB VRAM$19.17$37.50$54.17
Qwen3 14B Q4_K gen @4k31.2 t/s42.5 t/s54.2 t/s
Qwen3 14B Q4_K gen @16k22.6 t/s32.7 t/s40.6 t/s
Qwen3 8B gen @4k55.2 t/s71.2 t/s85.8 t/s
Qwen3 8B gen @16k42.0 t/s52.1 t/s59.1 t/s
Qwen3 8B gen @32k31.9 t/s38.1 t/s43.6 t/s
Prompt processing, 14B @4k972.6 t/s2,099.9 t/s2,144.2 t/s
Prompt processing, 14B @16k678.2 t/s1,355.8 t/s1,315.2 t/s
Prompt processing, 8B @4k1,696.8 t/s3,564.1 t/s3,487.7 t/s
Prompt processing, 8B @16k1,119.2 t/s2,064.3 t/s1,600.8 t/s
Prompt processing, 8B @32k764.7 t/s1,116.7 t/s898.8 t/s
Relative gen score (RTX 3060 = 100%)100%145%180%
Price per t/s (14B @16k)$10.18$13.78$16.01

Source: RTX 3060 12GB, RTX 4070, and RTX 5070 on hardware-corner.net, benchmarks updated March 2026. Every cell above is that source’s own published figure.

The 3060 page gives two power supply figures. It lists “Suggested PSU 550 W,” and its FAQ recommends “a minimum 650W quality PSU” for a single card. This guide reports both because the source does not reconcile them.

Its 14B figure is also reported two ways: the page summary and its price-per-t/s calculation use 22.6 t/s at 16k, while the raw benchmark table lists 22.7. This guide uses 22.6 throughout so the number matches the price-per-t/s figure it quotes. The headline blocks on all three benchmark pages also label their 14B figure ”@ 4k Ctx” while showing the 16k number; the table above uses each page’s benchmark tables instead.

The NVIDIA RTX 5070 family page is the vendor page for the 5070 line. It publishes no local-LLM benchmark figures, so every speed in this guide comes from hardware-corner.net.

Question 1: What fits in 12 GB?

The fit list

Ollama’s Qwen3 tags page lists these download sizes: qwen3:4b at 2.5 GB, qwen3:8b at 5.2 GB, and qwen3:14b at 9.3 GB. All three files fit on a 12 GB card; how much context you get on top depends on the card and runtime, and tyolab.com’s 14B test needed --ctx-size 4096 to make room. hardware-corner.net reports that Qwen3 14B Q4_K runs “fully in VRAM” up to 16k context on the RTX 3060, and that it measured Qwen3 8B up to 32k.

promptquorum.com makes a broader claim in its budget GPU guide: “12GB+ VRAM fits every 7B-8B model at Q4/Q5 and most dense 13B-14B models at Q4.”

The do-not-try list

Several models waste an evening on a 12 GB card:

  • gpt-oss-20b. tyolab.com reports that “the openai_gpt-oss-20b model is about 11.5GB at Q4_K_M quantization. That leaves only 500MB for the KV cache, not viable,” and adds that it loaded only partially with CPU offload, with responses that were “too slow and wrong.” promptquorum.com says gpt-oss:20b “needs 16 GB, so it is just out of reach on a 12 GB card.”
  • qwen3:30b. The Ollama library lists it as a 19 GB download.
  • Llama 4 Scout. promptquorum.com describes it as a “17B-active/109B-total MoE that needs ~55 GB at Q4.”
  • DeepSeek-R1-Distill-Qwen-14B. tyolab.com reports it “wouldn’t load at ctx=4096.”
  • Kimi-Linear-48B-A3B-BPW2.6. tyolab.com’s failure was a corrupt download of about 8.1 GB, not a VRAM limit. Treat it as a bad file, not a hardware ceiling.

Why 14B’s fit depends on context and setup

A model needs memory for two things: the weights and the KV cache. The cache grows with the context length. tyolab.com reports that at --ctx-size 8192, “14B Q4_K_M models fail to load: the weights (~8.4GB) plus the KV cache exceed 12GB.” Dropping to --ctx-size 4096 loaded Qwen3-14B for that test.

The sources conflict here. hardware-corner.net reports Qwen3 14B running fully in VRAM at 16k. tyolab.com reports a load failure at 8k. The sources do not explain the difference. Runtime, quantization, and settings could each account for it.

The cache size formula comes from compilenrun.com. For Llama 3.1 8B at 8,192 tokens with an FP16 cache, 32 layers, 8 KV heads, and head dimension 128, the cache is 2 × 32 × 8 × 128 × 2 bytes × 8,192 = 1.0 GiB. compilenrun.com estimates the total near 6 GiB, including weights and buffers. At the full 131,072-token context, it says the cache alone “would be 16 GiB.”

llama.cpp can shrink the cache with -ctk q8_0 -ctv q8_0. compilenrun.com gives the flags but no savings figure, so this guide does not state one.

Quantization in 90 seconds

Quantization stores each weight in fewer bits. compilenrun.com lists the bits per weight: F16 at 16.0, Q8_0 at 8.5, Q6_K at 6.5625, Q5_K at 5.5, Q4_K at 4.5, IQ4_XS at 4.25, Q3_K at 3.4375, and Q2_K at 2.625. The llama.cpp quantize README puts Q4_K_M at 4.8944, and koboldcpp.dev rounds the same file to 4.90.

Q4_K_M is not exactly 4 bits. compilenrun.com explains that it “uses Q4_K for most tensors but promotes” several tensors to higher precision, including the output projection, which moves to Q6_K. The result is “4.89 bits per weight instead of 4.5” for Llama 3.1 8B.

The llama.cpp quantize README gives file sizes. Llama 3.1 8B drops from a 32.1 GB unquantized original to 4.9 GB at Q4_K_M. The 70B model drops from 280.9 GB to 43.1 GB, and the 405B model from 1,625.1 GB to 249.1 GB.

koboldcpp.dev reads download sizes from Hugging Face for Llama 3.1 8B Instruct, dated 14 Nov 2026. Q4_K_M is 4.6 GiB, Q5_K_M is 5.3 GiB, and Q8_0 is 8.0 GiB. The same page warns that small models cost more bits than their labels suggest. A Q4_K_M file costs “4.90 BPW on Llama 3.1 8B and 6.24 on a 0.5B model.” It also says plainly, “We have not measured the quality of each step.”

Below about 4 bits, generation stops getting faster, because unpacking the weights costs as much time as the memory it saves (compilenrun.com).

Question 2: Why does the same card get three different speed answers?

Published speeds for the RTX 3060 12GB running Qwen3 14B at 4-bit — Q4_K in hardware-corner.net’s tests, Q4_K_M in tyolab.com’s — range from 9 to 33 tokens per second. The sources disagree for three reasons.

  • Context length. hardware-corner.net’s own 3060 numbers drop from 31.2 t/s at 4k to 22.6 t/s at 16k. A figure taken at one context length does not transfer to another.
  • Measured versus estimated. hardware-corner.net reports 22.6 t/s at 16k context, measured. tyolab.com reports 33.4 t/s with the model tested at --ctx-size 4096. promptquorum.com gives 9 to 12 t/s for Ollama on an RTX 3060 with 16 GB of system RAM, and it states these are “estimates for Ollama on an RTX 3060 12GB with 16GB system RAM, not our own benchmark results.” It adds that “speeds vary ±15% depending on prompt length and context window.”
  • Runtime and system memory. promptquorum.com’s estimate uses Ollama on 16 GB of RAM. tyolab.com’s 33.4 t/s comes from llama.cpp built from source on a machine with 64 GB of DDR4.

Compare a 4k test with a 16k test, or a measured figure with an estimate, before comparing the numbers themselves.

How fast each 12 GB card really is

The table shows what each card reaches on Qwen3 at several context lengths. Generation speed rises with memory bandwidth, so the 5070 leads at every context length, the 4070 comes second, and the 3060 trails. Prompt processing tells a different story. The 4070 and 5070 process 14B prompts at 16k about twice as fast as the 3060 does (1,355.8 and 1,315.2 t/s against 678.2 t/s, per hardware-corner.net), and at 4k the gap is wider still (2,099.9 and 2,144.2 t/s against 972.6 t/s), which matters when you paste long documents.

The one formula that predicts a card’s top speed

compilenrun.com gives a ceiling for decode speed: “tokens per second ≤ memory bandwidth / model size in bytes.” Each generated token reads every weight once, so bandwidth sets the limit. The same source says real speeds reach “perhaps 60 to 80% of the bound.” Its example: at 100 GB/s with a 4.9 GB model, the ceiling is “capped near 20 tokens per second.”

This is arithmetic on published figures, not a measurement. Bandwidths come from hardware-corner.net, the roughly 8.4 GB weight size for 14B comes from tyolab.com, and the formula comes from compilenrun.com.

  • RTX 3060: 360 ÷ 8.4 ≈ 43 t/s ceiling. The measured 31.2 t/s is about 73% of that bound.
  • RTX 4070: 504 ÷ 8.4 ≈ 60 t/s ceiling. The measured 42.5 t/s is about 71% of that bound.
  • RTX 5070: 672 ÷ 8.4 ≈ 80 t/s ceiling. The measured 54.2 t/s is about 68% of that bound.

All three measured speeds land inside compilenrun.com’s 60 to 80% band. The 8.4 GB figure is weight size, not download size: substitute the 9.3 GB Ollama download for Qwen3 14B and the ceilings fall to about 39, 54 and 72 t/s, and the 3060’s measured 31.2 t/s would sit just above the 80% mark instead of inside it.

Question 3: Price per GB and price per t/s — is 12 GB worth buying in 2026?

Price per gigabyte is simple division: $230 ÷ 12 GB is $19.17, $450 is $37.50, and $650 is $54.17, all per hardware-corner.net. The 12 GB pool is the same on all three cards, so the price gap is what you pay for the faster memory and core.

Price per t/s favors the cheapest card in the table. hardware-corner.net’s price per t/s at 14B and 16k is $10.18 for the 3060, $13.78 for the 4070, and $16.01 for the 5070. Those figures divide each card’s listed average price by its measured speed, so they move with the street price: at the $320–$350 used range promptquorum.com lists for the 3060, the same arithmetic gives about $14–$16 per t/s, which is no longer the lowest in the table. hardware-corner.net did not publish price per t/s for Qwen3 8B, so this guide does not calculate one.

The used-market wrinkle

Used prices change the buying decision, and the published ranges are wide. promptquorum.com lists the RTX 3060 12GB at “$320–$350” used in both its body and its buy table, and calls used the better deal at that price. It also says the entry point “has risen from about $200 to about $300 during 2026,” naming DRAM and GDDR7 supply and AI-datacenter demand as reported contributors, and notes that its sources “disagree on how much each contributes.”

No source in this brief publishes used prices for the RTX 4070 or RTX 5070, so this guide gives none.

The 16 GB counter-argument

A 16 GB card is the main alternative to 12 GB. promptquorum.com cites the RTX 5060 Ti 16GB at “about $620 new, roughly 44% over MSRP” of $429. It says the card handles models up to 20B at Q4 and reaches “25–30 tok/sec” on 7B models. BuySellRAM’s consumer GPU market update, dated 2026-10-11, lists the card at “about $618 new and $649 used,” as cited by promptquorum.com.

hardware-corner.net’s relative score puts the RTX 5060 Ti 16GB at 146% of the 3060’s generation performance, close to the 4070’s 145%.

promptquorum.com also flags that workstation cards “that used to be budget bargains (RTX A4000, RTX A2000)… no longer belong on a budget list.”

FAQ: 12 GB VRAM in 2026

Can I run Qwen3 14B on 12 GB? Yes, with caveats. hardware-corner.net reports Qwen3 14B Q4_K running fully in VRAM up to 16k context on the RTX 3060. tyolab.com reports that the Q4_K_M weights and cache failed to load at 8192 context. The sources do not explain the difference.

Is the RTX 4070 good value for local LLMs? hardware-corner.net answers “No.” Its page says “the 12GB VRAM limitation restricts usability to smaller models.”

Why was my speed different from the table? See Question 2 above. The gap usually comes from context length, measured versus estimated figures, or the runtime and system RAM.

Does Qwen3 8B run at 64k on 12 GB? hardware-corner.net’s tables stop at 32k for Qwen3 8B on 12 GB. One line on its page mentions 64k, but no table supports it, so this guide reports 32k.

Can quantizing the KV cache save memory? llama.cpp supports -ctk q8_0 -ctv q8_0 for quantizing the cache (compilenrun.com). The source does not publish how many gigabytes this saves, so this guide does not state a figure.

Can I run a 30B model on 12 GB? Not in the sources’ view. The Ollama library lists qwen3:30b as a 19 GB download. promptquorum.com says gpt-oss:20b needs 16 GB.

The Bottom Line

  • Budget buyer: buy a used RTX 3060 12GB. It is the cheapest of these three 12 GB cards in hardware-corner.net’s comparison, and at hardware-corner.net’s $230 average market value its $10.18 per t/s at 14B and 16k is the lowest in the table. At the $320–$350 used range promptquorum.com quotes, that works out to about $14–$16 per t/s, so the advantage narrows, but it is still the cheapest of the three at hardware-corner.net’s listed prices. Used prices for the 4070 and 5070 are not published, so no used-price comparison is possible. promptquorum.com notes one cheaper 12 GB option, the RX 6700 XT at “$175–$315” used, which it pairs with AMD setup friction.
  • Mid-range buyer: be careful with the RTX 4070. hardware-corner.net answers “No” when asked whether the 4070 is good value for local LLMs, and notes that 16 GB cards reach similar speeds. Its 5070 page recommends the opposite trade, calling the 4070 “much more cost-effective” than the 5070 if you only run models that fit in 12 GB. Both verdicts come from the same source, so read your budget before choosing.
  • Top-tier buyer: buy the RTX 5060 Ti 16GB over the RTX 5070. The 5070 costs $650 per hardware-corner.net, and hardware-corner.net calls it “a poor value proposition for local LLMs” for models that fit in 12 GB. promptquorum.com cites the 5060 Ti at about $620 new. hardware-corner.net’s own 5070 page is the source for the upgrade case: spending a little more on a 16 GB card “opens up access to the 20B+ model tier.”

How This Guide Was Built

Every published figure in this guide is quoted from a named source. The speed ceilings, the percentage-of-ceiling figures and the used-price per-t/s arithmetic are this guide’s own division on those published figures, and they are labelled as such where they appear. No hands-on testing was performed for this article.

The sources do not establish these points:

  • Used prices for the RTX 4070 or RTX 5070.
  • Measured speeds for the RTX 5060 Ti 16GB. The 25 to 30 t/s figure comes from promptquorum.com, and the 146% score comes from hardware-corner.net’s relative comparison.
  • Gigabyte savings from quantizing the KV cache.
  • Quality loss at each quantization step. koboldcpp.dev says it has not measured this.
  • Why the Qwen3 14B fit differs between hardware-corner.net (16k) and tyolab.com (failure at 8k).
  • The weight size of Qwen3 14B. Ollama lists the download at 9.3 GB, while tyolab.com estimates the Q4_K_M weights at about 8.4 GB.
  • CodeIntel Log — code quality, debugging, and software engineering benchmarks
  • ToolBrain — tool reviews, LLM comparisons, and AI workflow guides

Cross-links automatically generated from None.

Back to all posts