llm-inference-capacity-handbook
llm-inference-capacity-handbook is 大模型推理案头手册（开源版）：给定 GPU 算力，一个模型能扛多少 QPS？三层模型 × 三面墙 × 排队论 × 开环压测 - shtjww/llm-inference-capacity-handbook. It is ranked #857 on the AI Radar, in AI infra & evals, first seen 2 h ago and shared in 1 post (318 views).
GitHub - shtjww/llm-inference-capacity-handbook: 大模型推理案头手册（开源版）：给定 GPU 算力，一个模型能扛多少 QPS？三层模型 × 三面墙 × 排队论 × 开环压测 · GitHub Skip to content Navigation Menu Sign in Search / Sign in Sign up Appearance settings You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Repository navigation llm-inference-capacity-handbook Public Notifications You must be signed in to change notification settings Fork 0 ( 0 ) Star 185 ( 185 ) You must be signed in to star…
What people said about llm-inference-capacity-handbook on X
📈 压箱底的干货来了，llm-inference-capacity-handbook 帮你算清楚一件事：手里这些 GPU，到底扛得住多少 QPS。 llm-inference-capacity-handbook 是一本讲大模型推理容量规划的中文手册，开源的是 v8.0 节选，从第 0 章总纲一路讲到第 9 章集群与基础设施。 很多团队估容量，是压测跑满吞吐，或者拿 Decode 吞吐除以输出长度算 QPS。手册直接点破这两个坑：闭环压测的吞吐相当于把利用率顶到 100%，排队时间趋于无穷，不能当容量；忽略 Prefill 开销，在 2048/512 负载下大约会高估 4 倍。 它把问题拆成三层：物理层看…
— @Ryrenz, 2 h ago · 4 likes · see the post
Alternatives to llm-inference-capacity-handbook
- Nous Portal — Great time to try it out
- TensorFold — Setup guide
- Spark-X2.6 — 6 open routing models from 0.3B to 27B, built to make decisions and extract exactly what you need in a single call. 🤖
- AI/ML API — Crawl Reddit with your AI agent >
- OrcaRouter — Claim yours
- together.ai — Together AI
llm-inference-capacity-handbook in numbers
- Rank on the AI Radar: #857 of 10548
- Shared in 1 post by 1 account: @Ryrenz
- 318 views on those posts
- First seen 2 h ago, last shared 2 h ago
- Pricing seen by AGK Intelligence: open source
- Market: AI infra & evals
FAQ
What is llm-inference-capacity-handbook?
大模型推理案头手册（开源版）：给定 GPU 算力，一个模型能扛多少 QPS？三层模型 × 三面墙 × 排队论 × 开环压测 - shtjww/llm-inference-capacity-handbook It was first shared on X 2 h ago and is ranked #857 on the AI Radar.
Is llm-inference-capacity-handbook free?
It is open source.
Who shared llm-inference-capacity-handbook?
1 account on X, including @Ryrenz, in 1 post totalling 318 views.
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 65.2k posts from 5k X accounts over the last 14 days, 10.5k tools, 19 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-07 16:40 UTC. Full method.