Zyphra's new paper proposes Zyphra Hierarchical Attention (ZHA), which organizes the…
This is a AI post classified by AGK Intelligence as AI research (research), kept by the AI Radar because it carries real work, not commentary.
Zyphra's new paper proposes Zyphra Hierarchical Attention (ZHA), which organizes the KV-cache into a tree of compressed summaries. Queries search the tree from top to bottom, narrowing down which tokens deserve attention. With just 1B tokens of additional distillation training, ZHA preserves most of the performance of full attention on models up to 35B parameters. On RULER at 256K context, it achieves 93.3% vs 96% accuracy, while attending to just 1-6% of the KV-cache. https://www.alphaxiv.org/abs/2610.42112
Posted by alphaXiv (60.6k followers) 6 h ago · 84 likes · 4.4k views · view the original post on X. Kept by the AI Radar as AI research. Tools mentioned: alphaXiv.
More AI work like this
- 3rd place ARC-AGI-2 Public Leaderboard submission open sourced by @gonedarragh and… — @GregKamradt
- 按 token 计费的前提是 token 一样贵，但真实情况是：前面一个 token 会抬高后面所有 token 的能耗。 — @qingke_ai
- this is free f*cking gold — @Mr_Buzzoni
- New AI rigger that handles hard stuff like skirts, capes, hair and sleeves. — @Stefan_3D_AI
- How transformers work. — @pmddomingos
- Sparse attention reduces computation by selecting relevant tokens, but finding those… — @askalphaxiv
- Hello to everyone from all over the world, — @rlaope
- Re: Inkling pretrained on 12T tokens — @hsu_steve
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.2k posts from 5k X accounts over the last 14 days, 10.6k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-07 16:45 UTC. Full method.