What LLMs Can You Run With 12 GB of VRAM?

Quick Answer

A 12 GB GPU can comfortably run many current small and mid-sized quantized LLMs entirely in VRAM, including Gemma 4 E4B, Qwen3 8B, DeepSeek R1 Distill Qwen 7B, Ministral 8B, and Qwen2.5-Coder 7B in my testing.

On an RTX 4070 Ti with 12 GB of VRAM and 32 GB of system RAM, Gemma 4 E4B is the strongest all-around starting point from my tests: it stayed fully GPU-resident at 32K context while using 6.2 GB of VRAM and generated 88.5 tokens per second.

Current Calculator Picks for an RTX 4070 Ti 12 GB

This Ollama RTX 4070 Ti benchmark snapshot was captured on August 15, 2026. As a starting point, I used my LLM VRAM calculator, set to an RTX 4070 Ti and 32 GB of RAM, to get some initial estimates. The VRAM and speed columns are calculator estimates for that profile, not vendor requirements or measured benchmarks. Re-run the LLM VRAM Calculator & Local AI GPU Planner before choosing a model because its current model data and your settings can change the order.

Gemma 4 E4B is the strongest general-purpose recommendation in this snapshot. Qwen3 8B is the main alternative for chat, RAG, and agent experiments. DeepSeek R1 Distill Qwen 7B leaves more memory headroom for reasoning workloads, while Gemma 4 E2B is the smaller, faster option. Qwen3-VL-8B-Instruct is for image-aware workflows, but it may not achieve full GPU fit under some workloads.

To evaluate how well those calculator estimates held up in practice, I benchmarked the models on my RTX 4070 Ti system.

Real World Comparison

To help gather data for this article, I ran benchmark tests on my PC with the following specs:

• GPU: RTX 4070 Ti
• CPU: AMD Ryzen 5 5600
• RAM: 32 GB DDR4
• OS: Windows 11
• Ollama: 0.32.13

The tested Ollama tags were gemma4:e4b, qwen3:8b, deepseek-r1:7b, qwen2.5-coder:7b, nchapman/ministral-8b-instruct-2410:latest, qwen3-vl:8b, and gemma4:e2b.

Each model was tested three times with num_ctx configured to 8K and 32K. The prompts did not fill those windows equally: the 8K runs used roughly half of the available context, while the 32K runs began around 92% full. Each test used Q4_K_M quantization and a cold start, and I report the median of the three runs.

The table above is the cold-start Ollama measurements. The comparison below shows where the calculator matched those measurements and where it was optimistic at 32K context.

In summary, Gemma 4 E4B had the best tested balance of speed, VRAM use, and full GPU residency. At 32K context, it generated 88.5 tokens per second while using 6.2 GB of VRAM and remaining fully GPU-resident. That leaves substantial headroom on a 12 GB card rather than simply proving that the model can load. Qwen3 and Ministral are also viable, but use more of the available VRAM, while Qwen3-VL partially offloaded at 32K and dropped to 16.1 tokens per second.

That remaining headroom matters because model weights are only one part of VRAM usage. Increasing the context window also increases the K/V cache, which can change whether a model fits comfortably.

How Context Size Changes VRAM Usage

The model file is only one part of the memory budget. The calculator separates the estimate into three pieces:

• Model weights: The quantized parameters loaded for inference.
• K/V cache: Memory that grows with the active context window.
• Runtime overhead: Memory used by the runner beyond the model weights and cache.

For the same model and cache precision, K/V-cache memory scales approximately linearly with active context length, so moving from 8K to 32K requires roughly four times the cache memory before rounding. That is why a setup that appears to fit based on model-file size alone can offload or become impractical as context grows.

For example, the planner estimates Gemma 4 E4B at 5.6 GB total VRAM with an 8K context window and 5.9 GB at 32K. Model weights remain 4.8 GB and runtime overhead remains 0.6 GB. The displayed K/V-cache estimate rises from 0.2 GB to 0.5 GB.

Context is four times larger at 32K, and K/V-cache memory scales linearly with it. The rounded values do not visibly show an exact fourfold increase because the underlying 8K estimate is below 0.2 GB; when that underlying value is multiplied by roughly four, the 32K estimate displays as 0.5 GB.

Gemma 4 E4B still retains substantial headroom on a 12 GB card at 32K. The difference between 8K and 32K is relatively small for this model, but models with larger K/V-cache requirements can lose substantially more headroom as context grows.

What “Fits” Actually Means

The verdicts in this guide describe practical memory fit, not whether a runner can technically start a process.

• Comfortable: Fully GPU-resident with useful room for context and ordinary desktop use.
• Fits with tradeoffs: Requires lower quantization, reduced context, or less multitasking.
• CPU offload likely: May run by placing part of the model in system RAM, usually with a substantial speed cost.
• Not practical: Does not have enough GPU-resident memory for the listed model and context to be a sensible default.

System RAM remains useful for loading models, document indexes, and offloading layers. It does not make an oversized model perform as if it were entirely in VRAM. Treat offloading as an experiment to measure, not as a replacement for capacity.

How Current 12 GB Models Compare

Gemma 4 E4B IT is the current top-rated model in this calculator snapshot. Qwen3 8B is another current general-purpose choice with a smaller advertised maximum context window. Neither number guarantees that a particular Ollama tag, GGUF build, or desktop workload will use the same memory.

DeepSeek R1 Distill Qwen 7B is the reasoning-focused entry in the current list. Qwen3-VL-8B-Instruct is the image-aware option, and Gemma 4 E2B IT is the smaller fast alternative. The calculator also lists Qwen3.5 35B A3B as a 26 GB, “Barely runs” option for this profile; that is an offload experiment, not a 12 GB recommendation.

With Ollama, start by choosing an explicitly quantized tag that matches the memory budget, then check actual placement after loading it. Do not infer the expected memory use from a model family name or parameter count alone.

Recommended Starting Settings for a 12 GB GPU

Start conservatively, verify that the model remains GPU-resident, then raise the context window only if the workload needs it. These are starting points rather than quality rankings or universal settings.

The K/V-cache choice matters most as context rises. Q8_0 halves cache memory relative to F16 in the planner; Q4_0 saves more but may have an output-quality tradeoff. Keep the model quantization and K/V-cache setting distinct: one affects model weights and the other affects context memory.

Check Your Exact GPU and Workload

This page fixes the capacity at 12 GB. Your actual result also depends on GPU bandwidth, system RAM, model file, runner, cache settings, and workload. Use the planner to compare estimated weights, K/V cache, runtime overhead, fit, and speed for your setup.