Local LLM VRAM Calculator

Pick a model, a quantization, and a context length. See exactly how much GPU memory it needs — weights, KV cache, and overhead — and which GPU actually runs it, all the way up to the dual RTX 5090 ABUZ8 box.

🔒 100% in-browser — nothing you select is uploaded

1. The model you want to run

Real parameter counts and attention shapes (layers, KV heads) for accurate KV-cache math.
Billions of weights.
Transformer blocks.
kv_heads × head_dim (often 1024).

2. Quantization & context

Bits per weight. Q4_K_M is the popular default.
Many runtimes can store the cache in 8-bit.
Tokens held in memory at once.
Total VRAM needed
0 GB
weights + KV cache + overhead
Model weights
0 GB
at selected quant
KV cache
0 GB
at selected context
GPU VRAM Usable Fits? Headroom

Estimates, not guarantees. Weights = parameters × bytes-per-weight (set by the quant; values are typical GGUF bits-per-weight). KV cache = 2 × layers × kv-dim × context × KV-precision. Overhead ≈ 0.8 GB CUDA context plus a small activation allowance. Real usage shifts with the backend (llama.cpp / vLLM / Exllama), flash-attention, batch size, and whether you offload layers to CPU. GPU "usable" VRAM assumes ~1 GB reserved by the OS/driver; Mac unified memory assumes ~75% is addressable for the model.

Stop guessing. Run it on a box you own.

Knowing the VRAM is step one. QADIR OS is the sovereign layer on top: it loads the largest model your hardware can hold, routes the few calls it can't clear to a cloud API, and keeps your data on your machine. Own the floor, rent only the ceiling.

✓ You're on the list. We'll be in touch.

How much GPU memory a local LLM really needs

How much VRAM do I need to run an LLM?

Three things add up: the weights (parameters × bytes per weight, which the quantization sets), the KV cache (grows with context length and layer count), and about a gigabyte of runtime overhead. A 7-8B model at 4-bit needs roughly 5-6 GB; a 70B model at 4-bit needs roughly 40-45 GB. The calculator above breaks out all three for the exact model and settings you choose.

What does quantization actually cost me?

Quantization trades a little quality for a lot of memory. FP16 is 2 bytes per weight; Q4_K_M is about 0.6 bytes — a ~3.3× reduction — and is the most common choice because the quality loss is small. Drop to Q3 or Q2 and the model gets meaningfully worse, but it may be the difference between fitting and not fitting on your card.

Why does long context blow up my memory?

Every token in context is cached for every layer. The KV cache scales linearly with context length, so going from 8k to 128k context multiplies that part of memory by 16. On large models the KV cache can grow to rival the weights themselves — which is why a model that loads at 4k can OOM at full context. Storing the cache in 8-bit (above) halves it.

Is my data sent anywhere?

No. This calculator runs entirely in your browser. Nothing you select or type is uploaded, logged, or stored.