your rtx 3060 was running bonsai 2 at 26 tok/s this morning and now does 40 tok/s, i…
This is a AI post classified by Jev as Open & local models (a tool drop), kept by the AI Radar because it carries real work, not commentary.
your rtx 3060 was running bonsai 2 at 26 tok/s this morning and now does 40 tok/s, i spent the day inside the prismml llama.cpp fork so your rtx 3060 can rejoice. the kernel that reads the weights was leaving two thirds of the card's threads idle on the biggest matrices. i fixed it, tested it against the cpu reference, the outputs are bit for bit the same, and both PRs are open on their repo. then i grafted the qwen 3.8 mtp head back onto the file, the one they dropped in the export, and with a batch invariance switch speculative decoding is lossless: 50 tok/s, twice what the stock fork does
Posted by Sudo su (36.2k followers) 1 h ago · 38 likes · 2.3k views · view the original post on X. Kept by the AI Radar as Open & local models.
More AI work like this
- dear rtx 3060 owners, and every 12gb card behind it. bonsai 2 27b dense went from 26 to… — @sudoingX
- Nimble is able to process images! — @madiator
- Woah...it claims to be open Jev. And it's 100% opensource. — @Saboo_Shubham_
- Meet DeepSeek-V4.1-Flash: a blazing fast image-text-to-text model. It takes both images… — @HuggingModels
- Meet Edge0-35B-A3B-preview: a 35B parameter MoE model built for edge inference. It runs… — @HuggingModels
- Putting Jeff-1 out into the world, this is an attempt at a Jev like model, built on qwen… — @DJLougen
- Train a 9M parameter language model from scratch in five minutes. — @tom_doerr
- Meet DeepSeek-R1-Distill-Qwen-Base-7B, a 7B text generation model fine-tuned for… — @HuggingModels
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 27.2k posts from 4.8k X accounts over the last 14 days, 1.2k tools, 19 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-19 18:08 UTC. Full method.