I could fork it and create TrumpLM. Not a big leap, I suppose.
probably 8M params are too much even :)
Show HN: I built a tiny LLM to demystify how language models work
51–60 of 174 posts
Re: Show HN: I built a tiny LLM to demystify how language models work
#52I like the idea, just that the examples are reproduced from the training data set. How does it handle unknown queries?
[deleted]
Re: Show HN: I built a tiny LLM to demystify how language models work
#53Could it be possible to train LLM only through the chat messages without any other data or input?
If Guppy doesn't know regular expressions yet, could I teach it to it just by conversation? It's a fish so it wouldn't probably understand much about my blabbing, but would be interesting to give it a try.
Or is there some hard architectural limit in the current LLM's, that the training needs to be done offline and with fairly large training set.
Re: Show HN: I built a tiny LLM to demystify how language models work
#54* How creating dataset? I download it but it is commpresed in binary format.
* How training. In cloud or in my own dev
* How creating a gguf
Re: Show HN: I built a tiny LLM to demystify how language models work
#55* How creating dataset? I download it but it is commpresed in binary format. * How training. In cloud or in my own dev * How creating a gguf
```
uv run python -m guppylm chat
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/user/gupik/guppylm/guppylm/__main__.py", line 48, in
main()
File "/home/user/gupik/guppylm/guppylm/__main__.py", line 29, in main
engine = GuppyInference("checkpoints/best_model.pt", "data/tokenizer.json")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/gupik/guppylm/guppylm/inference.py", line 17, in __init__
self.tokenizer = Tokenizer.from_file(tokenizer_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Re: Show HN: I built a tiny LLM to demystify how language models work
#56Why are there so many dead comments from new accounts?
Because despite what HN users seem to think, HN is a LLM-infested hellscape to the same degree as Reddit, if not more.
Re: Show HN: I built a tiny LLM to demystify how language models work
#57This is really great! I've been wanting to do something similar for a while.
Re: Show HN: I built a tiny LLM to demystify how language models work
#58Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier.
Then, some criticism. I probably don't get it, but I think the HN headline does your project a disservice. Your project does not demystify anything (see below) and it diverges from your project's claim, too. Furthermore, I think you claim too much on your github. "This project exists to show that training your own language model is not magic." and then just posts a few command line statements to execute. Yeah, running a mail server is not magic, just apt-get install exim4. So, code. Looking at train_guppylm.ipynb and, oh, it's PyTorch again. I'm better off reading [2] if I'm looking into that (I know, it is a published book, but I maintain my point).
So, in short, it does not help the initiated or the uninitiated. For the initiated it needs more detail for it to be useful, the uninitiated more context for it to be understood. Still a fun project, even if oversold.
[1] https://spreadsheets-are-all-you-need.ai/ [2] https://github.com/rasbt/LLMs-from-scratch
Re: Show HN: I built a tiny LLM to demystify how language models work
#59* How creating dataset? I download it but it is commpresed in binary format. * How training. In cloud or in my own dev * How creating a gguf
You sound like Guppy. Nice touch.
Re: Show HN: I built a tiny LLM to demystify how language models work
#60[dead]