Epoch AI researcher Michelle Campeau on why so many AI benchmarks have errors in the…
This is a AI post classified by Jev as AI research (a launch), kept by the AI Radar because it carries real work, not commentary.
Epoch AI researcher Michelle Campeau on why so many AI benchmarks have errors in the first place: "A lot of people will write an answer and then write a question and send that pair to someone else to review. When you're horse blinders in on the answer, the answer can look right, but you don't understand what other types of answers would have also been right." "A lot of benchmarks these days are vibe coded. You can vibe code something that works to maybe 50% of what you actually want pretty quickly." "Taking that from working sort of okay to actually being a really high-quality benchmark is
Posted by MTS (510.3k followers) 18 h ago · 23 likes · 11k views · view the original post on X. Kept by the AI Radar as AI research.
More AI work like this
- We built AI that actually knows aging biology. Longevity-LLM (9B params) beats OpenAI… — @biogerontology
- We need more examples like this in the open-source RL ecosystem — @adithya_s_k
- "Verbalizing Subliminal Learning Effects Using Text Optimization" — @askalphaxiv
- “What Does Privileged Information Add to On-Policy Self-Distillation?” — @askalphaxiv
- “Score Centering Stabilizes Off-Policy Reinforcement Learning” — @askalphaxiv
- Banger paper from Google Cloud AI Research. — @dair_ai
- If you liked the MiMo-V2.6 livestreamed RL run, don’t forget to check out Marin’s… — @adi_baradwaj
- Astra solved this hand-object reconstruction and tracking problem in one shot😂 — @YuXiang_IRVL
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 27.1k posts from 4.8k X accounts over the last 14 days, 1.2k tools, 19 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-19 17:40 UTC. Full method.