Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a tiny LLM to demystify how language models work

github.com

Post reply on HN
Print view View original

141–150 of 174 posts

Re: Show HN: I built a tiny LLM to demystify how language models work

#141

Post by arcanemachiner »

dominotw wrote: post #134
achenatx wrote: post #117

Earlier quoted context omitted.

give the code to an LLM and have a discussion about it.

does this work? there is no more need for writing high level docs?

> does this work?

Absolutely. If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on.

> there is no more need for writing high level docs?

Absolutely not. That would be like exploring a cave without a flashlight, knowing that you could just feel your way around in the dark instead.

Code is not always self-documenting, and can often tell you how it was written, but not why.

Re: Show HN: I built a tiny LLM to demystify how language models work

#142

Post by andoando »

vixen99 wrote: post #106
andoando wrote: post #37

Earlier quoted context omitted.

Id highly disagree with that. Were all living in the same shared universe, and underlying every intelligence must be precisely an understanding of events happening in this space-time.

What does 'precisely' mean? Everyone has the same understanding of events - a precise one?

No I am saying the basis of intelligence must be shared, not that we have the same exact mental model.

I might for example say a human entered a building, a bat might on the other hand think "some big block with two sticks moved through a hole", but both are experiencing a shared physical observation, and there is some mapping between the two.

Its like when people say, if there are aliens they would find the same mathematical constants thet we do

Re: Show HN: I built a tiny LLM to demystify how language models work

#147

Post by algoth1 »

MarkusQ wrote: post #127
algoth1 wrote: post #87

This really makes me think if it would be feasible to make an llm trained exclusively on toki pona ( https://en.wikipedia.org/wiki/Toki_Pona )

There isn't enough training data though, is there? The "secret sauce" of LLMs is the vast amount of training data available + the compute to process it all.

I think you could probably feed a copy of a toki pona grammar book to a big model, and have it produce ‘infinite’ training data

Re: Show HN: I built a tiny LLM to demystify how language models work

#148

Post by stronglikedan »

arcanemachiner wrote: post #141
dominotw wrote: post #134

Earlier quoted context omitted.

does this work? there is no more need for writing high level docs?

> does this work? Absolutely. If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on. > there is no more need for writing high level docs? Absolutely not. That would be like exploring a cave without a flashlight, knowing that you could just feel your way around in the dark instead. Code is not always self-documenting,…

> If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on.

My non-coder but technically savvy boss has been doing this lately to great success. It's nice because I spend less time on it since the model has taken my place for the most part.

Re: Show HN: I built a tiny LLM to demystify how language models work

#149

Post by stronglikedan »

bigmadshoe wrote: post #137
dominotw wrote: post #134

Earlier quoted context omitted.

does this work? there is no more need for writing high level docs?

LLMs can tell you what the code does but not why the developer chose to do it that way. Also, large codebases are harder to understand. But projects like these are simple to discuss with an LLM.

> LLMs can tell you what the code does but not why the developer chose to do it that way.

Do LLMs not take comments into consideration? (Serious question - I'm just getting into this stuff)

Post reply on HN