Quantization quality loss is small down to 4-bit and sharp below about 3-bit, and for code it depends more on the task than on the quant you picked. In the most detailed test in circulation, one tester's four 4-bit files of Qwen3.6-27B finished within two points of each other on HumanEval+, yet two of the same 4-bit files scored 13 to 16 points lower on MBPP+, a second coding benchmark. Q4 is a sound default for chat and still the right floor for code. What does not survive is the idea that a quant "keeps 99% of quality" in any single sense.
That gap between two coding tests is the main reason this page exists. Most guides quote one score per quant level, and the evidence here shows how much that one score hides.
Where Quantization Quality Loss Starts#
Quantization quality loss starts to bite at around 3-bit, which is the closest thing to a located cliff in the material. According to a Qwen3.6-27B benchmark posted to r/LocalLLaMA (2026), KLD rose above 0.1 from Q3_K_XL downwards, and the tester described the degradation from that point as "more drastic". Above that level, the same kind of test keeps finding differences small enough to argue about.
KLD, short for Kullback-Leibler divergence, measures how far the quantized model's word probabilities have drifted from the original's, and lower is better. It has become the metric of choice in this niche because it is more sensitive than checking whether an answer was right. This is because a model can still land on the correct answer while its confidence in each word has shifted, and KLD registers that shift.
The chart below marks the reported threshold. Only the line at 0.1 and its position at Q3_K_XL come from the benchmark, and the curve around them is drawn to show the direction rather than measured.
What a KLD of 0.1 Actually Tells You#
A KLD of 0.1 tells you the order of the quants and very little about what a reader would notice. The people who measure it say so themselves. In a thread on KV cache quantization ("Effect of KV quantization on KLD", r/LocalLLaMA), one tester reported differences of 0.005 and 0.01 and then asked what a 0.005 increase in KLD actually means. Nothing in the material answers that question.
KLD can be trusted for ranking, since a higher-precision file sits closer to the original than a lower one. It cannot be trusted as a measure of experience, because no study in the sources links a KLD value to the point where a person reading the output would object. As a result, the 0.1 line is best read as a landmark where several testers saw things get worse, and anyone quoting a KLD delta as though it settles the question is reading too much into it.
How Close Are the 4-Bit Quants to Each Other?#
The 4-bit quants sit very close together on quality and far apart on speed. The most detailed test in the material comes from one tester, a self-described first-timer, who ran twelve GGUF quantizations of Qwen3.6-27B on a single RTX 3090 (r/LocalLLM, "Qwen3.6-27B on RTX 3090, tested 12 GGUF quants", 2026). Each file was scored on HumanEval+, MBPP+, perplexity, throughput and needle-in-a-haystack retrieval, and the final phase used 164 of 378 tasks.
The table below shows the 4-bit results. Gemma-4-31B is a different and larger model that the tester ran as a comparison, and the four Qwen files finish within 1.9 points of each other on HumanEval+.
| Model and quantization | HumanEval base | HumanEval+ | Run time |
|---|---|---|---|
| Gemma-4-31B UD-Q4_K_XL | 97.0% | 94.5% | 34:02 |
| Qwen3.6-27B UD-Q4_K_XL (no MTP) | 95.7% | 92.1% | 19:17 |
| Qwen3.6-27B IQ4_NL-mtp | 95.1% | 91.5% | 9:39 |
| Qwen3.6-27B UD-Q4_K_XL-MTP | 95.1% | 90.9% | 11:07 |
| Qwen3.6-27B IQ4_XS-Q8nextn | 93.9% | 90.2% | 9:01 |
Table: one tester, one GPU, run times are wall-clock for the whole benchmark. Source: r/LocalLLM RTX 3090 test (2026).
The last column is where the real difference sits. Run times went from 9 minutes to 34, almost fourfold, while HumanEval+ moved by about four points across all five rows. The tester's own pick for a 24 GiB card was IQ4_NL-mtp as the best overall balance. The UD-Q4_K_XL build without MTP was called "the quality king if you don't mind ~2×" the time, and in this run it took 19 minutes against under 10 for IQ4_NL-mtp, for 0.6 points more.
Plotting score against time makes the trade-off easier to see than the table does.
For anyone iterating on code, the time matters more than it looks. This is because a model that is a fraction of a point better but twice as slow per attempt is not obviously better once a task takes several attempts.
Why Is Quantization Harder on Code Than on Chat?#
Quantization is harder on code because code has far fewer acceptable next words than prose, so the same small drift in probabilities shows up as a bug instead of a synonym. A sentence survives a near-synonym. A program does not survive one wrong variable name or an off-by-one boundary, and it simply fails to run.
The clearest evidence came when the same files met a second coding benchmark. On MBPP+, setups that scored in the 90s on HumanEval+ fell to about 78%, and the chart puts the two scores side by side for each file.
Nothing changed between those bars except the set of problems. Gemma-4-31B dropped 15.9 points and the Qwen build 12.9, which means a single "quality retained" figure for a quant level is close to meaningless for coding on its own. The fair question to ask of any such claim is which benchmark produced it, and the answer is usually one test on one model.
This does not make Q4 unsafe for code. Every file in that comparison was a 4-bit quant, so the gap sits between two tests and not between two quant levels. Notably, the sensible reading is that 4-bit remains the floor for coding, and that a quant should be judged on the developer's own code rather than on whichever benchmark a model card quotes. Cache settings are a separate decision, covered in the guide to KV cache quantization.
When Is a Difference Between Quants Just Noise?#
A difference between quants is noise once it is smaller than the test can resolve, and the RTX 3090 run reached that point at Q5. All of the Q5 variants tied at 90.9% on HumanEval+ in the final phase, and the tester concluded that "the differences between them are noise". Time spent choosing between Q5 builds of the same model is better spent on a larger model, more context or a better prompt.
The same run found a much larger drop that had nothing to do with quantization. An abliterated finetune at Q5_K_M scored 75% with multiple timeouts, against 90.9% for the standard Q5 builds.
That fall of nearly 16 points is bigger than anything a quant level caused in the same test. As a result, when a local model starts writing bad code, the variant is the first thing to check and the compression the second. An abliterated build changes the model itself, and in this test code is where the change showed.
How to Read the Next Quantization Benchmark#
The next quantization benchmark can be read with four questions, and between them they cover nearly every claim in this cluster.
- Which model, and which size? A cliff found on a 27B model says little about a 4B one.
- Which benchmark? HumanEval+ and MBPP+ disagreed by up to 16 points on the same files.
- How many tasks? The RTX 3090 test screened on 20 tasks and confirmed on 164, which are very different signals.
- Is the gap bigger than the noise? If variants tie, they tie.
For local use the evidence supports a short set of rules. Stay at 4-bit or above, because below roughly 3-bit the divergence rises for every task and code has the least room for it. Within 4-bit, choose for speed, since the quality spread is small and the time spread is not. Use a standard build rather than an abliterated one, and test on your own code before trusting any of it.
Which file fits a particular card is worked out in the quantization explainer, which also decodes names like Q4_K_M and GGUF, and the VRAM calculator checks whether a quant fits your GPU before anything is downloaded.
Sources#
- Qwen3.6-27B quantization benchmark (r/LocalLLaMA)
- Quality comparison between Qwen 3.6 27B quantizations (r/LocalLLaMA)
- The Quantization Cliff, When Does Compression Break Code Intelligence? (PromptInjection)
- Qwen 3.6 27B GGUF quality benchmark (LocalBench)
- Qwen3.6-27B quantization benchmark writeup (huy.rocks, 29 May 2026)
- KLD-guided quantization (Hugging Face blog)
- Introduction to weight quantization (Maxime Labonne)
- Unsloth Dynamic GGUFs on Aider Polyglot (Unsloth docs)
- Quantization notes (bartowski)
Last reviewed: 18 October 2026. This page now includes the former quantization for coding post. Every benchmark cited is one or two models on one tester's hardware, and no controlled study across model families exists in the sources. Re-checked quarterly.
