The lesson-style README is a great approach. Breaking down LLM inference into digestible steps makes the codebase approachable even for people who haven't touched CUDA before.
Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
11–20 of 26 posts
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#12Thanks for sharing this. As someone currently researching LLMs, I'm sure I'll be referencing this quite a bit going forward.
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#13[flagged]
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#14[flagged]
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#15Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#16I feel like I learned twice as much in 10 minutes reading this than I did reading LLM for Dummies. Thank you
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#17I am looking at a plain and simple C implemented LLM inference, and/or x86_64 assembly implemented, and/or AMD GPU RDNA assembly.
Anybody?
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#18I am looking at a plain and simple C implemented LLM inference, and/or x86_64 assembly implemented, and/or AMD GPU RDNA assembly. Anybody?
I heard once that c++ can become assembly at some point if you type the right things in. :)
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#19README is in my opinion (author here) the most interesting - I wrote it to help others build useful mental model to be able to recreate the project yourself, without need to even read my code
I am not super familiar with C and CUDA, so I read solely for the README and enjoyed it supremely. The blend of cheerful walking through instructive examples and your philosophical takes on how to approach the exercise to get the most out of it put me in a great mood. You captured that special upbeat attitude that comes about when you're doing something as well as you can just because it's so legitimately interesting to you.
Re: Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
#20[flagged]