Verification results for the quantization models of Qwen3.8 27B show that the 4-bit (Q4_K_M) version maintains performance equivalent to the full model (BF16) in Terminal-Bench 2.1, a coding benchmark for agents. The 4-bit model has a size of 17GB, making it operational on consumer GPUs with 24GB of memory, such as the RTX 4090, while maintaining a context of approximately 64,000 tokens.

On the other hand, performance degradation occurs non-linearly as quantization progresses. While 2-bit models show score decreases in some benchmarks, they maintain a practical level of utility. However, performance collapses dramatically in 1-bit models, with benchmarks such as GPQA Diamond dropping to levels comparable to random guessing.

Additionally, it has been pointed out that although this model features a "reasoning_effort" function to adjust the depth of inference, it tends to perform excessive inference under default settings. In response, it has been suggested that the combination of methods such as Multi-Token Prediction (MTP) may significantly improve inference speed.

Source: