71–80 of 174 posts
Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier. Then, some criticism. I probably don't get it, but I think the HN headline does your project a disservice. Your project does not demystify anything (see below) and it diverges from your project's claim, too. Furthermore, I think you claim too much on your github. "This project exists to show that…
https://bbycroft.net/llm has 3d Visualization of tiny example LLM layers that do a very good job at showing what is going on ( https://news.ycombinator.com/item?id=38505211 )
Could it be possible to train LLM only through the chat messages without any other data or input? If Guppy doesn't know regular expressions yet, could I teach it to it just by conversation? It's a fish so it wouldn't probably understand much about my blabbing, but would be interesting to give it a try. Or is there some hard architectural limit in the current LLM's, that the training needs to be done offline and with…
How many parameters would you need for that?
[flagged]
I was going to suggest implementing RoPE to fix the context limit, but realized that would make it anatomically incorrect.
How does this compare to Andrej Karpathy's microgpt ( https://karpathy.github.io/2026/02/12/microgpt/ ) or minGPT ( https://github.com/karpathy/minGPT )?
I like the idea, just that the examples are reproduced from the training data set. How does it handle unknown queries?