按 token 计费的前提是 token 一样贵，但真实情况是：前面一个 token 会抬高后面所有 token 的能耗。
This is a AI post classified by AGK Intelligence as AI research (research), kept by the AI Radar because it carries real work, not commentary.
按 token 计费的前提是 token 一样贵，但真实情况是：前面一个 token 会抬高后面所有 token 的能耗。 每个 token 进 KV-cache 之后，都会让后续 token 更贵，所以能耗沿序列非线性增长，哪怕 token 看起来完全一样。这篇给出了一个四参数的解析式，SIMPLE，把推理能耗拆成 prefill 和 decode，再把 decode 拆成 KV-cache 成本和下一 token 计算，不用针对每个模型单独拟合。 数字挺硬：13 个密集模型、6 个家族、H100 上，R² 0.969、MAPE 11.97%，比之前的 SOTA 低 3.1 到 3.5 倍，且不需要逐模型标定。 更有用的是它给出的结构性结论。prefill 能耗随 hidden dim 二次增长，所以模型越大，长输入的账越难看；GQA 分组越激进，每 token 能耗最多能降到标准 MHA 的三分之一。 判断：这类工作真正的价值不是预测多准，而是告诉你改哪个超参数最省电。以后算推理成本该按这个模型算，而不是数 token。 边界也明确：只覆盖 dense 模型，MoE 不在内；硬件只验了 H100；能耗依赖真实 trace 的测量。 https://arxiv.org/abs/2611.03330
Posted by 青稞社区 (3.7k followers) 3 h ago · 2 likes · 317 views · view the original post on X. Kept by the AI Radar as AI research. Tools mentioned: arXiv.org.
More AI work like this
- 3rd place ARC-AGI-2 Public Leaderboard submission open sourced by @gonedarragh and… — @GregKamradt
- this is free f*cking gold — @Mr_Buzzoni
- Zyphra's new paper proposes Zyphra Hierarchical Attention (ZHA), which organizes the… — @askalphaxiv
- New AI rigger that handles hard stuff like skirts, capes, hair and sleeves. — @Stefan_3D_AI
- How transformers work. — @pmddomingos
- Sparse attention reduces computation by selecting relevant tokens, but finding those… — @askalphaxiv
- Hello to everyone from all over the world, — @rlaope
- Re: Inkling pretrained on 12T tokens — @hsu_steve
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.2k posts from 5k X accounts over the last 14 days, 10.6k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-07 16:45 UTC. Full method.