depth-hybrid-attention
depth-hybrid-attention is Project overview & code. It is ranked #126 on the AI Radar, in AI research, first seen 3 days ago and shared in 4 posts (37.9k views).
What depth-hybrid-attention says about itself
Contribute to Nanbeige/Depth-Hybrid-Attention development by creating an account on GitHub.
GitHub - Nanbeige/Depth-Hybrid-Attention · GitHub Skip to content Navigation Menu Sign in Search / Sign in Sign up Appearance settings You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Repository navigation Depth-Hybrid-Attention Public Notifications You must be signed in to change notification settings Fork 1 ( 1 ) Star 7 ( 7 ) You must be signed in to star a repository About No description, website, or topics provided. Resources Readme…
What people said about depth-hybrid-attention on X
We introduce Depth Hybrid Attention (DHA), which incorporates depth-wise value mixing into hybrid architectures with very little additional compute and cache overhead. By allowing layers to mix information across depth, DHA broadens each layer’s view and increases the model’s expressive capacity. Our DHA method…
— @nanbeige, 3 days ago · 342 likes · see the post
Nanbeige now does interesting architecture research
— @teortaxesTex, 2 days ago · 92 likes · see the post
🔥牛逼！性能直接碾压！Hybrid Attention终于不“缩水”了：BOSS直聘南北阁开源 Depth Hybrid Attention (DHA)，短上下文平均分暴涨，KV Cache几乎零增加！ 它在混合注意力架构里加入了“跨层值混合”，让每一层都能有选择地借用更早层的信息，模型表达能力明显变强，却几乎不增加计算和缓存开销。 核心优势 🔹短上下文基准全面领先：DWA平均39.68、DDA平均40.92，均超过对应的Full Attention和Hybrid基线 🔹生成任务涨幅尤其明显（GSM8K、CMATH等） 🔹长上下文检索和损失曲线也更好 🔹完整保留Hybrid Attention的低KV…
— @NFT_Chen, 2 days ago · 38 likes · see the post
Alternatives to depth-hybrid-attention
- flashprove — Further progress in provable inference will likely be an important part of improving trust in AI deployment. To that…
- moe-blogpost — A hands-on guide to training Mixture-of-Experts models
- liberai.com — Technical blog
- Science's Last Exam — Can AI do research, end to end? Science's Last Exam defines and measures whether an AI agent can carry a research goal…
- science-last-exam — Science's Last Exam. Contribute to Gen-Verse/Science-Last-Exam development by creating an account on GitHub.
- www-cdn.anthropic.com — Technical report
depth-hybrid-attention in numbers
- Rank on the AI Radar: #126 of 10389
- Shared in 4 posts by 3 accounts: @nanbeige, @teortaxesTex, @NFT_Chen
- 37.9k views on those posts
- First seen 3 days ago, last shared 17 h ago
- Pricing seen by AGK Intelligence: open source
- Market: AI research
FAQ
What is depth-hybrid-attention?
Project overview & code It was first shared on X 3 days ago and is ranked #126 on the AI Radar.
Is depth-hybrid-attention free?
It is open source.
Who shared depth-hybrid-attention?
3 accounts on X, including @nanbeige, @teortaxesTex, @NFT_Chen, in 4 posts totalling 37.9k views.
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.7k posts from 5k X accounts over the last 14 days, 10.4k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-06 13:30 UTC. Full method.