Have you ever had a local coding agent flying at the start of a session and crawling an…
This is a AI post classified by Jev as AI infra & evals (a tutorial), kept by the AI Radar because it carries real work, not commentary.
Have you ever had a local coding agent flying at the start of a session and crawling an hour later? It's not heat or RAM but the KV cache. The agent appends everything to the context every turn: files, tool outputs, all of it. The server caches that prefix, so normally only the new tokens get processed. But when something early in the chat changes (e.g. the agent trims old tool outputs) or the request lands on a different cache slot, that cache is useless. It rereads the whole conversation from scratch. With 200k+ tokens that can be a ~2 minutes read before it even starts replying. How to
Posted by Javier • priv/acc (4.4k followers) 1 h ago · 4 likes · 220 views · view the original post on X. Kept by the AI Radar as AI infra & evals.
More AI work like this
- Qualcomm announces it most powerful Snapdragon 8 Elite Extreme Gen 6 chip built for… — @CNET
- The https://MLX.fast community has finally doubled the speed of Qwen 3.8 Flash Next to… — @TheDavidTai
- 🚀 MCDMA is working between my Mac Studio & DGX Sparks, and now it's time to bring it to… — @ashxhart
- 🚀 MCDMA has RDMA working between my Mac Studio and DGX Sparks. — @ashxhart
- Deploy DiffusionGemma-Jev on Cloud Run with just a single command. — @Saboo_Shubham_
- Update for GLM 5.3 Flash EXL3 (2x Spark) releasing soon! — @plotarmordev
- before fraud protection I was spending $15/hour on http://classifier.dev — @michael_chomsky
- 762 commits. 315 contributors. 104 first-timers. vLLM v0.30.0 is live. 🎉 — @vllm_project
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 44.3k posts from 5k X accounts over the last 14 days, 1.8k tools, 19 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-23 13:36 UTC. Full method.