Viewing profile — i_am_rocoe

I'm working on a variation of MTP that recovers PP TPS (back to the same as with MTP disabled), keeping most of MTP's benefits to TG TPS. https://news.ycombinator.com/item?id=48700…

Thank you!

[dead]

I've been running Qwen3.6-35B-A3B locally on llama.cpp and noticed that prompt processing throughput gets too low with MTP. I got nerd-sniped. What started as curiosity turned into…

If you want to run the PoC locally to replicate: Clone the masked-nextn-skip-catchup branch: https://codeberg.org/rocoe/llama.cpp/src/branch/masked-nextn... Run llama-server with a…

I've been running Qwen3.6-35B-A3B locally on llama.cpp and noticed that prompt processing throughput gets too low with MTP. I got nerd-sniped. I'm not a C++ dev, I know almost noth…