llm-bench-rig
llm-bench-rig is Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable cards. - notwitcheer/llm-bench-rig. It is ranked #73 on the AI Radar, in AI infra & evals, first seen 10 days ago and shared in 11 posts (45.1k views).
llm-bench-rig/reports/toolcall-v1.md at main · notwitcheer/llm-bench-rig · GitHub Skip to content Navigation Menu Sign in Appearance settings Search / Sign in Sign up Appearance settings You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Repository navigation Files Expand file tree main / toolcall-v1.md Copy path Blame More file actions Blame More file actions Latest commit History History History 95 lines (73 loc) · 14.8 KB main /…
What people said about llm-bench-rig on X
which file of Qwen3.8-27B should you download if you want it to use tools, the way an agent like Hermes Agent does? I tested six files of the same model, from 22.9 GB down to 5.9 GB, on 100 tasks where the model has to keep calling tools over several turns. TL;DR: the 3-bit file (11.9 GB) is as good as the 6-bit one…
— @witcheer, 10 days ago · 201 likes · see the post
a 12 GB card already gets Hermes Agent through 27 of 30 agent tasks on a local model. a 32 GB card gets 30. 30 agent tasks, run offline in Hermes Agent: renaming code across files, solving a git merge conflict, finding the bad commit in a 14-commit history, fixing five bugs behind a 40-test suite. TL;DR, the best…
— @witcheer, 5 days ago · 203 likes · see the post
A 4B model with a real to-do list. Spark-X2.6-4B passed 25/30 offline tasks in @witcheer's Hermes test, with 6,116 MiB peak VRAM at 64K context. Measured on an RTX 5090; placed in the author's 8GB memory tier. Worth reproducing on your own setup.
— @SparkLLM, 1 days ago · 59 likes · see the post
Alternatives to llm-bench-rig
- Spark-X2.6 — 6 open routing models from 0.3B to 27B, built to make decisions and extract exactly what you need in a single call. 🤖
- fireworks.ai — Start building through our API, no dedicated GPUs needed
- OrcaRouter — Claim yours
- aimlapi.com — Crawl Reddit with your AI agent >
- LongCatAI — Take a closer look at what it can do
- Kimi API Platform — Kimi API Platform, providing the 2.8-trillion-parameter Kimi K3 large language model API with a 1M-token context…
llm-bench-rig in numbers
- Rank on the AI Radar: #73 of 10392
- Shared in 11 posts by 3 accounts: @witcheer, @SparkLLM, @PaulGug
- 45.1k views on those posts
- First seen 10 days ago, last shared 1 days ago
- Pricing seen by AGK Intelligence: open source
- Market: AI infra & evals
FAQ
What is llm-bench-rig?
Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable cards. - notwitcheer/llm-bench-rig It was first shared on X 10 days ago and is ranked #73 on the AI Radar.
Is llm-bench-rig free?
It is open source.
Who shared llm-bench-rig?
3 accounts on X, including @witcheer, @SparkLLM, @PaulGug, in 11 posts totalling 45.1k views.
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.7k posts from 5k X accounts over the last 14 days, 10.4k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-06 13:40 UTC. Full method.