I love learning new concepts by playing around with them (think explorable explanations (https://en.wikipedia.org/wiki/Explorable_explanation), and I got more and more interested in running local LLMs on my own hardware. Since we live in the age of "speak, and your side project will be done" (joking, but LLMs are really helpfull for me when it comes to frontend stuff and for brainstorming in general!), I built the tool I needed myself :)
WatchMachineGo is basically a visual simulator that simulates how a local LLM model gets loaded, prefilled and then used for inference, while showing the effects of different hardware parameters like memory bandwitch or setups like no GPU, two GPUs and so on.
All free and without ads, forever.
I plan to open source it too, but want to think about how first, still.
Note: It is still under construction!
If you check it out: Thank you very much and I hope that it will be time well spent! :)
Show HN: WatchMachineGo – A visualizer to show hardware performing LLM inference
watchmachinego.com