Viewing profile — ani17

Author here. I wanted to understand what vLLM and llama.cpp are actually doing under the hood, but the codebases are massive. So I wrote a stripped down version from scratch to see…

The blog walks through why your first token is always the slowest, why output tokens cost 5x more, and how stuff like speculative decoding and chunked prefill actually work, from t…

Author here. A bit more context: By day I'm a systems engineer building AI networking infrastructure. So I kept ending up in conversations where I'm not exactly able to wrap my hea…

Definitely an alternative solution. For the purpose of this script, I wouldn't prefer that though.

I use Google Meet frequently with the "Share a tab" feature. The main pain point is that when I switch to a different tab, I often forget to click the "Share this tab instead" butt…

It's insane if the data is accurate. Only time will tell

You forgot "Middle Out" by Pied Piper!

[dead]

[dead]

thanks for sharing!