Qwen3.8-Flash-Next: EXL3 ~600 vs NVFP4 ~150 tok/s reported concurrent decode. 🤯
This is a AI post classified by Jev as AI infra & evals (a model release), kept by the AI Radar because it carries real work, not commentary.
Qwen3.8-Flash-Next: EXL3 ~600 vs NVFP4 ~150 tok/s reported concurrent decode. 🤯 Both on DGX Spark GB10. The interesting part is what the engines actually do with those compressed weights. This will be my last informational post on Qwen3.8-Flash-Next as im cooking up something new for you all! 👨🏻🍳 3.05 bpw EXL3 through my ExLlamaV3 fork. NVFP4 through vLLM. 𝗧𝗛𝗘 𝗗𝗘𝗖𝗢𝗗𝗘 𝗗𝗜𝗙𝗙𝗘𝗥𝗘𝗡𝗖𝗘 Balanced workload, EXL3 / NVFP4: Single-request decode: 78.3 / 31.0 tok/s Per-request decode at 8 requests in flight: 75.9 / 18.9 tok/s Time per output token at that load: 13.2 / 53.4 ms Reported concurrent dec
Posted by Cruz (2.1k followers) 2 h ago · 28 likes · 1.1k views · view the original post on X. Kept by the AI Radar as AI infra & evals. Tools mentioned: qwen3.8-flash-next-exl3-dgx-spark-recipe, exllamav3.
More AI work like this
- Kimi K3 is now on Amazon Bedrock! — @Kimi_Moonshot
- 继续调优，测试了 MTP 步数和 FR-Spec。 — @wei_wang
- Join us on October 7th for an exclusive webinar on Cloudflare OS. We’re discussing the… — @Cloudflare
- our team shipped some major improvements to https://cfo.ai's cache efficiency this morning — @blader
- MSI announces EdgeMesa N AI+ — @MiaAI_lab
- Good news: Tako Search is free in @vercel_dev AI Gateway. — @TakoViz
- Meet Husky: a Model-Specific Inference (MSI) engine up to 4.5× faster than Apple's MLX — @0xSigil
- LLMs are built out of neural networks, so it was easy to extend to the fruit fly… — @0xdoug
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 41k posts from 5k X accounts over the last 14 days, 1.7k tools, 19 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-22 04:47 UTC. Full method.