继续调优，测试了 MTP 步数和 FR-Spec。
This is a AI post classified by Jev as AI infra & evals (a model release), kept by the AI Radar because it carries real work, not commentary.
继续调优，测试了 MTP 步数和 FR-Spec。 还是单张 RTX PRO 6000 Blackwell 96GB，功耗限制 350W，16 并发共享约 100 万 token KV 池。 MTP 1 步在 8 路和 16 路下能把综合吞吐提高约 22% 和 24%，还可以多释放约 1.08GiB 显存，但 1 到 4 路会慢约 7%。为了兼顾日常低并发请求，最终继续使用 MTP 3 步。 FR-Spec 的问题出在词表。 现成的 64K 草稿词表对中文覆盖率只有约三成，中文任务反而慢了三成多。我们重新收集中英文回答、代码和工具调用格式，按词频制作了 64K、96K 和 128K 三套混合词表。 四类任务在 1、2、4、8、16 路并发下完整测试后，96K 的综合效果最好，相对关闭 FR-Spec 提升约 5.3%。64K 和 128K 分别提升 4.9% 和 4.1%。 词表更大，覆盖率更高，速度却不一定更快。草稿计算成本也会跟着增加，96K 正好落在这套模型和硬件的平衡点上。 现在已经把 MTP 3 步和混合 96K 词表保留在线。 最新 Prose 实测达到 单路 252.9 tok/s 8 路总吞吐 819.4 tok/s 16 路总吞吐 1190.2 tok/s 相比上一轮，单路提高约 8.2%，16 路总吞吐提高约 11.8%。 单张 350W
Posted by Wei (6.8k followers) 1 h ago · 2 likes · 416 views · view the original post on X. Kept by the AI Radar as AI infra & evals.
More AI work like this
- Join us on October 7th for an exclusive webinar on Cloudflare OS. We’re discussing the… — @Cloudflare
- our team shipped some major improvements to https://cfo.ai's cache efficiency this morning — @blader
- MSI announces EdgeMesa N AI+ — @MiaAI_lab
- Good news: Tako Search is free in @vercel_dev AI Gateway. — @TakoViz
- Meet Husky: a Model-Specific Inference (MSI) engine up to 4.5× faster than Apple's MLX — @0xSigil
- LLMs are built out of neural networks, so it was easy to extend to the fruit fly… — @0xdoug
- Welcome @prisminference to Respan Gateway 🚀 — @RespanAI
- Meet Wally. — @RunAnywhereAI
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 40.8k posts from 5k X accounts over the last 14 days, 1.7k tools, 19 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-22 02:45 UTC. Full method.