Show HN: I built a tiny LLM to demystify how language models work • Hacker News Board

Re: Show HN: I built a tiny LLM to demystify how language models work

#161

Post by dominotw » Mon, Apr 06, 2026, 10:00 PM UTC

libria wrote: post #156

stronglikedan wrote: post #148

Earlier quoted context omitted.

> If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on. My non-coder but technically savvy boss has been doing this lately to great success. It's nice because I spend less time on it since the model has taken my place for the most part.

> since the model has taken my place for the most part Hah, you realize the same thing is going on in your boss's head right? The pie chart of Things-I-Need-stronglikedan-For just shrank tiny bit...

my last employer was using ai to rank developers on most impactful code their prs are shipping.

Re: Show HN: I built a tiny LLM to demystify how language models work

#162

Post by mudkipdev » Tue, Apr 07, 2026, 4:58 AM UTC

algoth1 wrote: post #87

This really makes me think if it would be feasible to make an llm trained exclusively on toki pona ( https://en.wikipedia.org/wiki/Toki_Pona )

People have made toki pona translation models before, not exclusively trained though

Re: Show HN: I built a tiny LLM to demystify how language models work

#163

Post by hahooh » Tue, Apr 07, 2026, 6:42 AM UTC

haha funny, but really cool project. why fish tho lol.

Re: Show HN: I built a tiny LLM to demystify how language models work

#164

Post by [deleted] » Tue, Apr 07, 2026, 9:34 AM UTC

fg137 wrote: post #64

How does this compare to Andrej Karpathy's microgpt ( https://karpathy.github.io/2026/02/12/microgpt/ ) or minGPT ( https://github.com/karpathy/minGPT )?

[deleted]

Re: Show HN: I built a tiny LLM to demystify how language models work

#165

Post by devsteru » Tue, Apr 07, 2026, 9:35 AM UTC

totetsu wrote: post #61

https://bbycroft.net/llm has 3d Visualization of tiny example LLM layers that do a very good job at showing what is going on ( https://news.ycombinator.com/item?id=38505211 )

Thanks for sharing

Re: Show HN: I built a tiny LLM to demystify how language models work

#166

Post by inanutshellus » Tue, Apr 07, 2026, 1:11 PM UTC

zkmon wrote: post #86

_2fnr wrote: post #50

[flagged]

Meaning/goal of life is to reproduce. Food (and everything else) is only a means to it. Reproduction is the only root goal given by nature to any life form. All resources and qualities are provided are only to help mating.

I don't get why anyone downvoted you but maybe we can "all get along" by saying:

   "the meaning of life is to continue living."

Thus the short-term answer is "food" and "reproduction" is the long-term answer.

Re: Show HN: I built a tiny LLM to demystify how language models work

#167

Post by BorisMelnik » Tue, Apr 07, 2026, 2:38 PM UTC

thomasfl wrote: post #111

Is there some documentation for this? The code is probably the simplest (Not So) Large Language Model implementation possible, but it is not straight forward to understand for developers not familiar with multi-head attention, ReLU FFN, LayerNorm and learned positional embeddings. This projects shares similarities with Minix. Minix is still used at universities as an educational tool for teaching operating system des…

I haven't heard minix in so long!

Re: Show HN: I built a tiny LLM to demystify how language models work

#168

Post by tombelieber » Tue, Apr 07, 2026, 3:23 PM UTC

looking forward to try it, great job

Re: Show HN: I built a tiny LLM to demystify how language models work

#169

Post by bigmadshoe » Tue, Apr 07, 2026, 5:28 PM UTC

stronglikedan wrote: post #149

bigmadshoe wrote: post #137

Earlier quoted context omitted.

LLMs can tell you what the code does but not why the developer chose to do it that way. Also, large codebases are harder to understand. But projects like these are simple to discuss with an LLM.

> LLMs can tell you what the code does but not why the developer chose to do it that way. Do LLMs not take comments into consideration? (Serious question - I'm just getting into this stuff)

They do. Think of it like a very intelligent but somewhat unreliable engineer you can hire to look at your code. They have no context about the codebase beyond what’s written in the source code, or any docs you give them.

What I meant was the docs might provide explanations about the problems the codebase solves, design decisions, the abstractions chosen, etc that wouldn’t live in a particular source file. Any discussion someone has with an LLM about the codebase will lack this context in the explanations given if docs don’t exist.

Re: Show HN: I built a tiny LLM to demystify how language models work

#170

Post by Propelloni » Tue, Apr 07, 2026, 9:11 PM UTC

jadengeller wrote: post #71

Propelloni wrote: post #58

Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier. Then, some criticism. I probably don't get it, but I think the HN headline does your project a disservice. Your project does not demystify anything (see below) and it diverges from your project's claim, too. Furthermore, I think you claim too much on your github. "This project exists to show that…

this comment seems to be astroturfing to sell a course

What do you mean, the LLM from Scratch book?