{"id":17225,"date":"2026-09-03T13:12:09","date_gmt":"2026-09-03T20:12:09","guid":{"rendered":"https:\/\/mattfife.com\/?p=17225"},"modified":"2026-09-03T13:23:40","modified_gmt":"2026-09-03T20:23:40","slug":"qwen3-8-27b-quantized-accuracy-chart","status":"publish","type":"post","link":"https:\/\/mattfife.com\/?p=17225","title":{"rendered":"Qwen3.8-27b quantized accuracy chart"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

How bad do the results of using less than full BF16 quantization get? Kaitchup just posted results of his benchmarks for Qwen3.8 27B<\/a> for quants from different tests – from Q4 to Q1.<\/p>\n\n\n\n

\n

Below 95% accuracy recovery, I consider the degradation too large. At that point, you are probably better off using a smaller model at a healthier quantization level rather than forcing a heavily quantized 27B model to fit on your GPU.<\/p>\n<\/blockquote>\n\n\n\n

Additional discussion here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

How bad do the results of using less than full BF16 quantization get? Kaitchup just posted results of his benchmarks for Qwen3.8 27B for quants from different tests – from Q4 to Q1. Below 95% accuracy recovery, I consider the degradation too large. At that point, you are probably better off using a smaller model at a healthier quantization level rather than forcing a heavily quantized 27B model to fit on your GPU. Additional discussion here.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[28,9],"tags":[],"class_list":["post-17225","post","type-post","status-publish","format-standard","hentry","category-ai","category-cool"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4WECr-4tP","jetpack-related-posts":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts\/17225","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=17225"}],"version-history":[{"count":3,"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts\/17225\/revisions"}],"predecessor-version":[{"id":17229,"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts\/17225\/revisions\/17229"}],"wp:attachment":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17225"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=17225"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=17225"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}