GGUF VRAM Calculator

How the estimate is built

Three numbers are added together, and it is worth knowing which one is hurting you when a model does not fit.

The weights are the model file itself. This tool reads the real size of each GGUF from Hugging Face rather than working it out from bits per weight, because the calculated version is consistently wrong. llama.cpp stores token embeddings and the output tensor at higher precision than the rest of the model, and that overhead scales with vocabulary size rather than parameter count. On a small model with a large vocabulary the gap runs to well over a gigabyte.

The context is the KV cache, and it grows as the conversation fills up. This is where most calculators go wrong in 2026. Until recently every layer of a model cached the full context, so the arithmetic was simple. Qwen3.5 and its successors interleave linear attention layers, which hold a fixed-size state no matter how long the context gets, with a smaller number of full attention layers that behave the old way. Gemma 4 does something similar with sliding-window layers that only ever hold 1024 tokens. Counting every layer as full attention overstates Qwen3.8-27B at 128k context by roughly four times.

The overhead is the CUDA context and the compute buffers, and it lands near a gigabyte regardless of model size.

Reading the result

Take the largest quantisation that fits with room to spare. Quality tracks file size closely, and the quality cliff below roughly four bits per weight is far steeper than any gain above it. Given a choice between a large model squeezed into two bits and a smaller model at four, the smaller model wins nearly every time.

If nothing fits, you have three levers before buying hardware. Drop the KV cache to 8-bit, which roughly halves the context cost for very little quality. Shorten the context, since almost nobody needs 128k. Or accept partial offload, where llama.cpp keeps some layers in system RAM and the model still runs, just slowly.

If you want the reasoning behind the quantisation names themselves, the terms are explained here, and there is a companion piece onchoosing a quant for your VRAM, on what quality you actually lose, and on quantising the KV cache itself. For hardware, seewhat you need to run models locally.

Questions

What does this calculator actually measure?

Three things added together. The model file itself, taken from the real size of the GGUF on Hugging Face rather than an estimate. The KV cache, which is the memory your context window occupies and grows as you fill it. And roughly 1 GB of fixed overhead for the CUDA context and compute buffers.

Why is the KV cache smaller than other calculators say?

Because most 2026 models are no longer uniformly full-attention. Qwen3.5 and later interleave linear attention layers whose state does not grow with context, and Gemma 4 uses sliding-window layers capped at 1024 tokens. A calculator that assumes every layer caches the full context will overstate these models badly, by around 4x for Qwen3.8-27B at 128k.

Which quantisation should I pick?

The largest one that fits with headroom to spare. Quality rises with file size, and the drop-off below about 4 bits per weight is much steeper than the drop-off above it. If Q4_K_M and Q5_K_M both fit, take Q5_K_M. If nothing at 4 bits or above fits, a smaller model at Q4 will almost always beat a larger model at Q2.

Can I run a model that does not fit entirely in VRAM?

Yes. llama.cpp will offload some layers to system RAM and run the rest on the GPU, so the model still loads. It just runs much more slowly, because every token has to cross the PCIe bus. Treat a result that does not fit as a speed warning rather than a hard no.

Does quantising the KV cache help?

A lot, at long context. Dropping the cache from 16-bit to 8-bit halves that part of the budget and costs very little quality. At 128k context on a big model the cache can be larger than the weights, so it is often the first thing worth changing.

How accurate is this?

Close enough to make a buying or downloading decision, not close enough to run at 99% capacity. The published empirical work on predicting llama.cpp VRAM reports a median error of around 365 MiB and suggests a safety buffer of roughly 577 MB for high confidence. This tool leaves similar headroom and tells you when a result is marginal.