Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How LLMs work

0xkato.xyz

Post reply on HN
Print view View original

71–80 of 293 posts

Re: How LLMs work

#71

Post by barrenko »

Maledictus wrote: post #44
10GBps wrote: post #6

I learned TCP/IP by watching and reading raw packets over packet radio at 1200 baud. I've noticed the same thing is possible if you watch the output of a slow LLM. Eventually you start to see the machinery. input tokens = output tokens, it's math. I can't exactly predict the tokens generated but I can see how they are formed. It's a lot like chess. You can't see every possible move but the mechanism is understandable…

How would I set this up?

I'd recommend to maybe also specifically watching Karpathy's videos and focusing on the early parts where he specifically deals with tokenization / embeddings generation (which gets really overlooked), and he does this in most of his videos.

Re: How LLMs work

#72

Post by barrenko »

aabdi wrote: post #30

this is hard to read... it goes all over the place. i'm not actually sure who your target audience is. there's too many side tangents. just like, structure it plz. 1. customer feels bad cuz they don't understand how llms work 2. provide high level abstracted explanation (don't dive into concepts yet) 3. provide breakdown guide of overall set of components. 4. walk through each component. don't side track. no need to…

It's just slop.

Re: How LLMs work

#73

Post by bspammer »

vocram wrote: post #68

Saying an article is of inferior quality just because editing was AI-assisted is like saying a book is lower quality just because it was printed rather than written by hand

No? One affects the actual text and the other doesn’t.

Re: How LLMs work

#74

Post by lateral_cloud »

vocram wrote: post #68

Saying an article is of inferior quality just because editing was AI-assisted is like saying a book is lower quality just because it was printed rather than written by hand

AI assisted is a stretch. And that analogy isn't even close to being relevant

Re: How LLMs work

#75

Post by antirez »

malwrar wrote: post #10

Back when ChatGPT came out, I was so shocked by how _good_ it was for an “AI” product that I simply had to know how it worked. Over the next month I ended up drawing out a block diagram on a whiteboard I have in my office, with the math involved next to each step in the blackboard. I’d puzzle about each step along the way, and the triumph of completing the drawing was also that of this sense of deep understanding. I…

There is a different way to look at this: that is, actually the Transformer is a minimal complication of what the based model is: in theory the neural network could be just a huge FFN, which is anyway the part of the Transformer that does the heavy lifting. But this would be impossibile to train both numerically and computationally, so the Transformer encodes enough priors for it to work: the causal attention, and the math tricks like the residuals and so forth. But the bottom line of all this is that the Transformer works because of the incredible semantical power of simple/huge FFNs.

Re: How LLMs work

#76

Post by ekunazanu »

malwrar wrote: post #10

Back when ChatGPT came out, I was so shocked by how _good_ it was for an “AI” product that I simply had to know how it worked. Over the next month I ended up drawing out a block diagram on a whiteboard I have in my office, with the math involved next to each step in the blackboard. I’d puzzle about each step along the way, and the triumph of completing the drawing was also that of this sense of deep understanding. I…

> This was openai’s entire breakthrough. Making this particular model architecture larger leads to emergent capabilities

Basically, the bitter lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

Re: How LLMs work

#77

Post by formerly_proven »

otabdeveloper4 wrote: post #40
10GBps wrote: post #15

Earlier quoted context omitted.

Yep. It's nearly identical to the neural nets we were using in the 90s. Back then even a supercomputer wasn't big enough or fast enough to do what we do today. I have to wonder though. Is this all a human brain is? A similar thing to an LLM just scaled exponentially larger. I mean a brain is not just neurons with simple connections to each other. The neurons, axons, dendrites, , etc in a brain are all holding and pro…

> I mean a brain is not just neurons with simple connections to each other. No, it's not. There are many animals that have extremely complex and even learned behaviour that have literally zero neurons. Clearly "neurons" is an oversimplification just-so story, not a scientific theory.

Do you consider fungi animals or do you perhaps mean animals that don't have a brain/CNS?

Re: How LLMs work

#78

Post by GardenLetter27 »

malwrar wrote: post #10

Back when ChatGPT came out, I was so shocked by how _good_ it was for an “AI” product that I simply had to know how it worked. Over the next month I ended up drawing out a block diagram on a whiteboard I have in my office, with the math involved next to each step in the blackboard. I’d puzzle about each step along the way, and the triumph of completing the drawing was also that of this sense of deep understanding. I…

It's not just the architecture but also the data - the decoder only approach lets you train in parallel over blocks of text (no RNN serial waiting), that allows you train on much, much more data.

Re: How LLMs work

#79

Post by janalsncm »

lionkor wrote: post #65

It sucks that this article is clearly LLM edited, with common phrases like "same shape as", "the intuition: ", and the "tiny explainer" which clearly generalized from a prompt accidentally. Good article, but when sharing it I will have to preface "yes it's slop, but it's a good explanation". Absolutely embarrassing that the author didn't catch that these LLM-isms are a (and here I'll use one) bad signal. In fact, I w…

I don’t think it’s absolutely embarrassing. First of all, the point of the author writing at all is to aid understanding, not produce prose. So from that standpoint, what would be embarrassing would be to include incorrect facts that suggest a fundamental misunderstanding of the topic.

From my read, it is fine. The brief history of LLMs is complicated since every single component has papers introducing enhancements. So it’s easy to ignore them or get bogged down with details.

The author appears to be a security researcher learning about LLMs for the purpose of defending against common attacks. So this piece is that person giving themselves a crash course on the topic. The fact that they cleaned up their notes with an LLM is frankly completely irrelevant.

Re: How LLMs work

#80

Post by stalfie »

This article describes how Transformers work, but not really how LLMs work. Explaining the underlying architecture gives you about as much insight into how a modern LLM behaves as an breakdown of neuronal biochemistry and a few pathways does for the brain. Meaning, almost no insight at all.
Post reply on HN