Params
27B
Fastest decode
370 tok/s
RTX 5090 · 4-bit MLX format + DFlash2 · TensorFold 0.6.6
Refusal
Official
Setups
17
Best reported pair · not live serving speed ↗
Runs on
Quoted one-stream decode on real boxes.
- RTX 50904bit370 tok/s
- RTX PRO 6000 Blackwell Max-QNVFP4272.8 tok/s
- RTX 5090EXL3258 tok/s
- RTX 5090IQ4_XS186 tok/s
- RTX A5000NVFP4177.6 tok/s
- RTX PRO 6000 Blackwell Max-QNVFP4151.4 tok/s
- RTX A5000NVFP4139.3 tok/s
- DGX Spark4-bit102.9 tok/s
- M5 Pro 64GB4-bit89.9 tok/s
- DGX SparkNVFP472 tok/s
- M6 mini 32GB4-bit61.9 tok/s
- DGX SparkNVFP442.3 tok/s
- RTX 4090 24GBUD-Q4_K_XL40.7 tok/s
- DGX SparkNVFP431.8 tok/s
- DGX SparkNVFP428.7 tok/s
- RTX 5060 Ti 16GBIQ3_XXS23 tok/s
- RTX 5060 Ti 16GBUD-Q2_K_XL19 tok/s
Get it running
- 1. DownloadPuts the weights on this computer.
huggingface-cli download Qwen/Qwen3.8-27B - 2. Start itUse this only if the download is GGUF.
llama-server -hf Qwen/Qwen3.8-27B - Open the weightsHugging Face ↗
- One box · 27B, SGLangRecipe ↗