novelty-eval
novelty-eval is Benchmark and controlled study of LLM-based novelty judges for research ideas. Code and data for "Old Ideas, Novel Problems" (arXiv 2610.02022) - allenai/novelty-eval. It is ranked #3777 on the AI Radar, in AI research, first seen 10 days ago and shared in 1 post (12.3k views).
GitHub - allenai/novelty-eval: Benchmark and controlled study of LLM-based novelty judges for research ideas. Code and data for "Old Ideas, Novel Problems" (arXiv 2610.02022) · GitHub Skip to content Navigation Menu Sign in Appearance settings Search / Sign in Sign up Appearance settings You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Repository navigation novelty-eval Public Notifications You must be signed in to change notification…
What people said about novelty-eval on X
LLMs are increasingly used to judge whether AI-generated research ideas are novel. But are they any good at it? Our new eval reveals even frontier LLMs can reach different conclusions about similar ideas depending on how they're asked to evaluate them. 🧵 https://github.com/allenai/novelty-eval
— @allen_ai, 10 days ago · 145 likes · see the post
Alternatives to novelty-eval
- academy.dair.ai — Chat with Paper
- alphaXiv — Now on @askalphaxiv
- flashprove — Further progress in provable inference will likely be an important part of improving trust in AI deployment. To that…
- moe-blogpost — A hands-on guide to training Mixture-of-Experts models
- liberai.com — Technical blog
- Science's Last Exam — Can AI do research, end to end? Science's Last Exam defines and measures whether an AI agent can carry a research goal…
novelty-eval in numbers
- Rank on the AI Radar: #3777 of 10412
- Shared in 1 post by 1 account: @allen_ai
- 12.3k views on those posts
- First seen 10 days ago, last shared 10 days ago
- Pricing seen by AGK Intelligence: open source
- Market: AI research
FAQ
What is novelty-eval?
Benchmark and controlled study of LLM-based novelty judges for research ideas. Code and data for "Old Ideas, Novel Problems" (arXiv 2610.02022) - allenai/novelty-eval It was first shared on X 10 days ago and is ranked #3777 on the AI Radar.
Is novelty-eval free?
It is open source.
Who shared novelty-eval?
1 account on X, including @allen_ai, in 1 post totalling 12.3k views.
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.5k posts from 5k X accounts over the last 14 days, 10.4k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-06 16:50 UTC. Full method.