Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How LLMs work

0xkato.xyz

Post reply on HN
Print view View original

261–270 of 293 posts

Re: How LLMs work

#261

Post by LatencyKills »

tinktank wrote: post #255
LatencyKills wrote: post #144

Earlier quoted context omitted.

I have a BS in CS (and have been in the field for 25 years). I couldn't understand the transformer architecture until I built a few myself. Here are the books I worked through. I now feel I have a very good understanding of modern LLMs. https://www.amazon.com/Build-Large-Language-Model-Scratch/dp... https://www.amazon.com/Build-DeepSeek-Scratch-Abhijit-Dandek...

Has it given you enough of an understanding that you can pick up and follow research papers or did you have to do more to achieve that?

I went this route because I had difficulty visualizing the content of the Attention Is All You Need paper. After going through both books, I can now understand every part of that paper.

I'm currently working on a robotics project that uses Nvidia's GR00T N1 model, and I was able to understand the research paper. [0]

[0]: https://arxiv.org/abs/2503.14734

Re: How LLMs work

#262

Post by [deleted] »

layla5alive wrote: post #213
YeGoblynQueenne wrote: post #164

Earlier quoted context omitted.

I'd be very careful assuming something is not in an LLM's training set. Those data sets are truly vast. And, from experience, people tend to miss a lot of their content. As a for instance, back in the day some academics wrote a paper that compared GPT 3.5 to a couple of inductive programming systems (including one of mine) on solving programming problems in a certain well-known esoteric language which I shall call "L…

I could answer the same query the same way as a child.

[deleted]

Re: How LLMs work

#263

Post by enrich717 »

Great write-up, thanks! I went from zero to a partial understanding after the first read. Will definitely come back to this to fully wrap my head around it.

Re: How LLMs work

#264

Post by tinktank »

LatencyKills wrote: post #261
tinktank wrote: post #255

Earlier quoted context omitted.

Has it given you enough of an understanding that you can pick up and follow research papers or did you have to do more to achieve that?

I went this route because I had difficulty visualizing the content of the Attention Is All You Need paper. After going through both books, I can now understand every part of that paper. I'm currently working on a robotics project that uses Nvidia's GR00T N1 model, and I was able to understand the research paper. [0] [0]: https://arxiv.org/abs/2503.14734

Thank you for the information.

Re: How LLMs work

#265

Post by otabdeveloper4 »

layla5alive wrote: post #249
otabdeveloper4 wrote: post #231

Earlier quoted context omitted.

The LLM predicts next token one at a time. (Stochastically.) This is a literal truth. Deal with it.

It's a literal truth that predicting the next token one at a time does not preclude intelligence on the other side of the decode function. Deal with it.

Yes, for a certain vague notion of "intelligence". Even homes, televisions and watches are "smart" now.

Re: How LLMs work

#266

Post by stevenhuang »

TheHalfDeafChef wrote: post #157
jmalicki wrote: post #151

Earlier quoted context omitted.

Performance enhancements are huge though. If you can make the existing model faster, you can then save your inference budget to then make your model bigger, which then makes it smarter. A lot of how smart the models can be comes down to budget. If you can make your existing thing cheaper, you can instead make it bigger for the same price.

Not really “smarter” though? It’s just a big probability engine. (Not trying to flame bait or anything. I just wouldn’t call LLM as exhibiting intelligence. It is great at making connections based on probability but doesn’t have a semantic understanding of what it is doing)

You do realize modern neuroscience considers the human brain as "just" a probability engine and that intelligence may well be the ability for an organism to predict well.

> doesn’t have a semantic understanding of what it is doing

I hope you realize this is an area of open, active research.

Re: How LLMs work

#267

Post by hackinthebochs »

Borealid wrote: post #260
hackinthebochs wrote: post #248

Earlier quoted context omitted.

>The point is that the output is text that is statistically correlated with the input. But we can simply note that this description applies to any machine learning algorithm. Yet LLMs are lightyears better than, say, Markov chains. What people are after is something that elucidates the features of LLMs that allow them to be so productive over what came before.

There is absolutely nothing stopping someone from distilling a modern LLM into a very effective Markov chain. The physical size of the model would explode because a context window containing C tokens of size B would need B^C Markov prior states, but the actual output would be a deterministic version of the LLM's with top-n n=1 sampling. In other words, a Markov chain and a Transformer model are exactly equivalent in…

>In other words, a Markov chain and a Transformer model are exactly equivalent in power

Nonsense. Markov chains treat the past context as a single unit, an N-tuple with no internal structure. LLMs leverage the internal structure of the context which allows a large class of generalization that Markov chains necessarily miss.

Re: How LLMs work

#268

Post by LearnYouALisp »

barrenko wrote: post #72
aabdi wrote: post #30

this is hard to read... it goes all over the place. i'm not actually sure who your target audience is. there's too many side tangents. just like, structure it plz. 1. customer feels bad cuz they don't understand how llms work 2. provide high level abstracted explanation (don't dive into concepts yet) 3. provide breakdown guide of overall set of components. 4. walk through each component. don't side track. no need to…

It's just slop.

Without seeing it, most likely answer based on any apparent confusion and the current broader situation

Re: How LLMs work

#269

Post by bonoboTP »

sirsinsalot wrote: post #116
redox99 wrote: post #100

Earlier quoted context omitted.

In the 90s you didn't have norm layers, residuals, attention, and some more. So you're missing a lot of the building blocks that make LLMs. It's not a matter of just having the compute.

I think the attention mechanism is so simple but so revolutionary that people forget it. Like the best leaps in thinking, once it is made, is is immediately obvious and intuitive.

Yes, but it wasn't invented from nothing in 2017. Soft attention existed in other applications like information retrieval, Nonlocal networks had similar ideas as well. But it wasn't seen or used as a fundamental building block. But it wasn't something out of the blue either.

Re: How LLMs work

#270

Post by layla5alive »

otabdeveloper4 wrote: post #265
layla5alive wrote: post #249

Earlier quoted context omitted.

It's a literal truth that predicting the next token one at a time does not preclude intelligence on the other side of the decode function. Deal with it.

Yes, for a certain vague notion of "intelligence". Even homes, televisions and watches are "smart" now.

You evaded any understanding of my point — mistaking the API for the implementation is like seeing a manuscript beside a typewriter and concluding the typewriter wrote it. You wouldn’t do that; why make the same category error here?
Post reply on HN