How bad do the results of using less than full BF16 quantization get? Kaitchup just posted results of his benchmarks for Qwen3.8 27B for quants from different tests – from Q4 to Q1.

Below 95% accuracy recovery, I consider the degradation too large. At that point, you are probably better off using a smaller model at a healthier quantization level rather than forcing a heavily quantized 27B model to fit on your GPU.

Additional discussion here.