"Can my GPU run this model?" is the first question everyone asks about local AI, and most answers are either vibes or a spec sheet written for people who already know. Our free AI VRAM calculator gives you the real answer in seconds: pick a model size, pick a quantization, and see whether it fits in your card's memory. This guide explains the numbers behind it — so you understand why the answer is what it is, and what your options are when it's "no."
Rough working numbers for text models: a 7B model at Q4 quantization needs about 4–5GB of VRAM and runs comfortably on an 8GB card. A 13B at Q4 wants roughly 8–10GB — a 12GB card is the comfortable floor. A 70B at Q4 needs on the order of 40GB+, which means a workstation card, a multi-GPU rig, or aggressive CPU offloading with real speed penalties. Run the same models at full FP16 precision and the requirements roughly quadruple — a 7B jumps to ~14GB — which is exactly why almost nobody runs full precision at home. These are honest approximations, not marketing: actual usage shifts with context length, architecture, and your inference stack, which is why a calculator beats memorizing a chart.
Three things. Model weights are the big one — billions of parameters times bytes-per-parameter. At FP16 that's 2 bytes each; at Q4 it's roughly half a byte, which is the entire magic trick of quantization. The KV cache grows with your context window: every token in play holds attention keys and values in memory, so a 32k-token context can quietly claim gigabytes on top of the weights. Long-context work costs real memory, not just time. Overhead — CUDA buffers, the inference framework, your desktop compositor — typically shaves 1–2GB off what the box claims. An "8GB" card is a 6-something-GB card by the time your model loads.
Quantization stores model weights at lower precision — 4-bit instead of 16-bit — trading a small amount of output quality for a 3–4x cut in memory. For most real work the quality difference between FP16 and a good Q4 or Q5 quant is hard to notice; the difference between "runs on my machine" and "doesn't" is not. This is what the GGUF format was built for, and it's why the local AI world runs on quantized models. If GGUF is new to you, read our plain-English GGUF explainer — it's the file format you'll be downloading either way.
Diffusion models (image generation) care about resolution and batch size more than parameter count — SDXL-class models want 8–12GB to be comfortable, and video generation is hungrier still: current local video pipelines are happiest at 16–24GB, though offloading tricks can squeeze them into less if you tolerate the wait. The pattern holds everywhere: VRAM is the hard ceiling, speed is the negotiable part. If it fits, it runs; the only question is how fast.
In order of sanity: drop a quant level — Q4 instead of Q5, Q3 if you're desperate; quality degrades gradually, not off a cliff. Use a smaller model — a 7B that fits beats a 13B that's swapping to system RAM. Offload layers to CPU — most runtimes let you split a model between GPU and system memory; it works, at a real speed cost. Shrink your context window — reclaims KV cache instantly. What we won't tell you is "just buy a bigger GPU" — the whole point of running local is doing real work on hardware you already own, and a well-chosen quant on an 8GB card does more real work than most people expect.
ABUZ8 builds QADIR OS — a sovereign, local-first agentic operating system that runs AI on your machine, not on a rented meter. The first honest conversation in that world is hardware: what can your box actually run? That's why the VRAM calculator is free with no signup, like the rest of the tool library. Know your ceiling, pick your model, own your stack.
VRAM is the one spec that decides local AI: 8GB gets you a fast 7B, 12–16GB unlocks 13B-class models with room to breathe, 24GB opens serious territory, and quantization is the lever that makes all of it affordable. Check your card against the model you want before you download 40GB of weights — it takes ten seconds and it's the ten seconds that saves an evening.
Find your number first. Run the free AI VRAM calculator, then join early access for QADIR OS — the sovereign agentic OS built to run on the hardware you just measured. No card required.