Sparse attention reduces computation by selecting relevant tokens, but finding those…
This is a AI post classified by AGK Intelligence as AI research (research), kept by the AI Radar because it carries real work, not commentary.
Sparse attention reduces computation by selecting relevant tokens, but finding those tokens becomes extremely expensive at ultra-long contexts, sometimes accounting for 98% of attention's prefill time. This paper Latent Sparse Attention (LSA) groups similar tokens into clusters and organizes them into a searchable hierarchy, letting queries to find relevant information without scanning the entire context. At 10M tokens, LSA makes one attention layer 22.7x faster during prefill and 4.6x faster during decoding than GLM-5.3-Flash's original sparse attention, while maintaining similar accuracy.
Posted by alphaXiv (60.6k followers) 11 h ago · 114 likes · 5.3k views · view the original post on X. Kept by the AI Radar as AI research. Tools mentioned: alphaXiv.
More AI work like this
- 3rd place ARC-AGI-2 Public Leaderboard submission open sourced by @gonedarragh and… — @GregKamradt
- 按 token 计费的前提是 token 一样贵，但真实情况是：前面一个 token 会抬高后面所有 token 的能耗。 — @qingke_ai
- this is free f*cking gold — @Mr_Buzzoni
- Zyphra's new paper proposes Zyphra Hierarchical Attention (ZHA), which organizes the… — @askalphaxiv
- New AI rigger that handles hard stuff like skirts, capes, hair and sleeves. — @Stefan_3D_AI
- How transformers work. — @pmddomingos
- Hello to everyone from all over the world, — @rlaope
- Re: Inkling pretrained on 12T tokens — @hsu_steve
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.2k posts from 5k X accounts over the last 14 days, 10.5k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-07 16:40 UTC. Full method.