Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How LLMs work

0xkato.xyz

Post reply on HN
Print view View original

271–280 of 293 posts

Re: How LLMs work

#271

Post by podocarp »

FergusArgyll wrote: post #229
miki123211 wrote: post #166

There's one thing I wish people understood about LLMs, and it doesn't really have anything to do with what's inside the neural network part. It's the fact that LLMs can only write in one direction — forward. When you are writing an essay and realize midway through a sentence that what you've written doesn't make sense, you go back and edit. An LLM can't do that, the only thing it can do is keep on generating. Because…

I get what you're saying but to be slightly pedantic etc. Why can't an llm tool call `delete(index: int)` or `replace(from_index: int, to_index: int, string: str)` and then it can go back and edit just the way we can? We also first made the mistake and only afterwards noticed we actually want to change something

It can, that's why things like planning and reasoning are so effective. Planning effectively gives them a scratchpad they can keep modifying till they're satisfied without being constrained into the "only go forward" output they usually produce. Reasoning is also the same idea. Although I have on rare occasions seen in plans models saying "blabla above is wrong, I should instead do …" instead of actually fixing the plan, but that's another problem

Re: How LLMs work

#272

Post by podocarp »

hackinthebochs wrote: post #155
Borealid wrote: post #90

Earlier quoted context omitted.

Your casual understanding is imprecise. At all times the LLM is, indeed, predicting the next token. Anything it does emerges from that. It did not "figure anything out". It predicted that text describing the use of a radial gradient was likely to follow text describing your problem.

>At all times the LLM is, indeed, predicting the next token The point is that saying they're just "predicting the next token" is not at all explanatory nor providing insight. Saying the brain is just firing action potentials gives you no understanding about how the brain does what it does or what the space of its capabilities are. Similarly, predicting the next token tells you nothing about the capabilities of LLMs.

I think it's a perfectly fine one liner explanation. If a kid asks why grass is green, do you stop explaining when you say chlorophyll is green, or do you go on to explain electron hybridization and all the spectra stuff, or do you go further to explain the structure of our eyes and why we perceive that reflected light as green? Also why green? Why not red? Do you have to explain that? It all depends on the audience, the context, and how much space you have to explain as well as how much you know. For you and more experienced people of course this is not sufficient and so you need to know more being "predict tokens" and so that opens up follow up questions like "how does it do that".

Re: How LLMs work

#273

Post by Borealid »

hackinthebochs wrote: post #267
Borealid wrote: post #260

Earlier quoted context omitted.

There is absolutely nothing stopping someone from distilling a modern LLM into a very effective Markov chain. The physical size of the model would explode because a context window containing C tokens of size B would need B^C Markov prior states, but the actual output would be a deterministic version of the LLM's with top-n n=1 sampling. In other words, a Markov chain and a Transformer model are exactly equivalent in…

>In other words, a Markov chain and a Transformer model are exactly equivalent in power Nonsense. Markov chains treat the past context as a single unit, an N-tuple with no internal structure. LLMs leverage the internal structure of the context which allows a large class of generalization that Markov chains necessarily miss.

No, not nonsense.

Both are a lookup table whose key is the entire context window and whose value is a probability distribution for what the next token should be.

You can say the choice of probability distribution in the value is "leveraging the internal structure of the context" or not, but the same tokens in two different orders are two different lookup keys and saying it's impossible to achieve some result with a Markov chain is factually incorrect.

https://arxiv.org/pdf/2410.02724 describes the equivalence formally.

Re: How LLMs work

#274

Post by podocarp »

stevenhuang wrote: post #241
Borealid wrote: post #220

Earlier quoted context omitted.

I completely agree that humans sometimes are not applying reasoning to things. I'm not trying to argue a model can not "reason" or have "cognition", whatever those things are. I'm only saying that it's absolutely the case that whatever those things are, they come from its mechanism of predicting one token at a time ad infinitum, and that throwing away a deep understanding in favor of a shallow one is foolish. Just be…

A dominant theory for human cognition is predictive coding https://en.wikipedia.org/wiki/Predictive_coding

Not and expert but how does this explain planning or anything creative? That's just generating things according to the world model with no error correction afterwards.

Re: How LLMs work

#275

Post by podocarp »

Marha01 wrote: post #122
AltruisticGapHN wrote: post #88

I don't like how most LLM explainer articles and videos say that essentially a LLM " predicts the next word". I'm a developer but not very good at maths and I still don't understand any of it. A LLM clearly has some "visual" capacity. You ask Gemini to build something with Canvas and it's able to reason about the shape of things. Like recently I waanted a checkbox that has like a gradient flowing around the edge. It…

LLMs fundamentally work by predicting the next word (token). But that should not be used to diminish their potential capabilities. It's like saying that human brains "just predict (or produce) the next electrical impulse". Fundamentally correct, but says nothing about the potential emergent capabilities of scaled-up systems that work like that. Emergent properties of complex systems should not be diminished just beca…

True, but there are also some who push it too far. As with all things, take it with moderation.

Re: How LLMs work

#276

Post by podocarp »

layla5alive wrote: post #249
otabdeveloper4 wrote: post #231

Earlier quoted context omitted.

The LLM predicts next token one at a time. (Stochastically.) This is a literal truth. Deal with it.

It's a literal truth that predicting the next token one at a time does not preclude intelligence on the other side of the decode function. Deal with it.

To be honest, if this is intelligence, then it's really boring. We can't even simulate the brain of a 1000 neuron nematode. You're telling me we can't even run nematode.exe but somehow we have already created intelligence?

Re: How LLMs work

#277

Post by hackinthebochs »

Borealid wrote: post #273
hackinthebochs wrote: post #267

Earlier quoted context omitted.

>In other words, a Markov chain and a Transformer model are exactly equivalent in power Nonsense. Markov chains treat the past context as a single unit, an N-tuple with no internal structure. LLMs leverage the internal structure of the context which allows a large class of generalization that Markov chains necessarily miss.

No, not nonsense. Both are a lookup table whose key is the entire context window and whose value is a probability distribution for what the next token should be. You can say the choice of probability distribution in the value is "leveraging the internal structure of the context" or not, but the same tokens in two different orders are two different lookup keys and saying it's impossible to achieve some result with a M…

That paper doesn't prove the equivalence of Transformers and Markov chains, it uses Markov chains as a theoretical model to understand the behavior of Transforms. The expressivity of the model matters, and Transformers just are more expressive than Markov chains.

>but the same tokens in two different orders are two different lookup keys

This is necessarily true for Markov chains and not necessarily true for Transformers. Transformers learn invariance over certain kinds of semantically irrelevant transformations. The Markov chain simply has to learn each input variant independently, resulting in an explosion of state space and data requirements compared to the functionally equivalent transformer. Expressive power matters.

I really don't get people's love for saying X is "just" Y (it's just a Markov chain, it's just a Kernel method). It's a strange pathology to focus on the superficial similarity while downplaying the boost in expressive power from where the models diverge.

Re: How LLMs work

#278

Post by Chu4eeno »

stevenhuang wrote: post #266
TheHalfDeafChef wrote: post #157

Earlier quoted context omitted.

Not really “smarter” though? It’s just a big probability engine. (Not trying to flame bait or anything. I just wouldn’t call LLM as exhibiting intelligence. It is great at making connections based on probability but doesn’t have a semantic understanding of what it is doing)

You do realize modern neuroscience considers the human brain as "just" a probability engine and that intelligence may well be the ability for an organism to predict well. > doesn’t have a semantic understanding of what it is doing I hope you realize this is an area of open, active research.

Didn't neuroscience some big scandals about bad statistics and overstating their findings (in addition to normal issues like replication)? Look up at least the "dead salmon study" (hint: it's related to fMRI, and you can probably guess its conclusions from its nickname). The "Voodoo Correlations" and "Cluster Failure" papers are also a bit eye-opening.

In general we (humans) need to be humble about the limitations of our knowledge about how we function, it's an insanely complicated problem.

Re: How LLMs work

#279

Post by Chu4eeno »

locknitpicker wrote: post #259
galaxyLogic wrote: post #216

Earlier quoted context omitted.

The big breakthrough is we can interact with the agents using natural language - because of the LLM. It is the combination of LLM and agent-harnesses that make it look really smart. Agent-harness is a programmatic device that lets us tap into the vast knowledge in the LLM. It is probabaly true that many TV-commentators fail to appreciate this fact and therefore think LLMs are super-intelligent. No, it is the combinat…

> The big breakthrough is we can interact with the agents using natural language - because of the LLM. Without ReAct and tool calling, all you have is a chatbot. That's useful, but it's just a chatbot. ReAct loops and tool calling is what unblocks high value usecases. It enables systems to actually address free-form problem statements, gather data that is not a part of their training set, inspect the current state of…

It's not very novel, though, it's a fairly obvious step once you get something that can operate iteratively and largely independent, there were a ton of people trying to get LLMs to loop on their own even before deepseek r1.

And I remember talking about goal directed behavior (which what people are calling "agents" now don't seem to properly have) and autonomous operation decades ago in the intelligent agent course at uni, including react loops.

So no, the huge step with LLMs really was just that attention mechanism from that translation paper everyone forgot until Google brought its marketing to it, everything else is either just optimization/scaling, more money or old ideas suddenly relevant.

Re: How LLMs work

#280

Post by Chu4eeno »

antirez wrote: post #75
malwrar wrote: post #10

Back when ChatGPT came out, I was so shocked by how _good_ it was for an “AI” product that I simply had to know how it worked. Over the next month I ended up drawing out a block diagram on a whiteboard I have in my office, with the math involved next to each step in the blackboard. I’d puzzle about each step along the way, and the triumph of completing the drawing was also that of this sense of deep understanding. I…

There is a different way to look at this: that is, actually the Transformer is a minimal complication of what the based model is: in theory the neural network could be just a huge FFN, which is anyway the part of the Transformer that does the heavy lifting. But this would be impossibile to train both numerically and computationally, so the Transformer encodes enough priors for it to work: the causal attention, and th…

Well, it seems obvious (in hindsight) that attention is vital to actually do anything useful with all those matrices getting smashed together in increasingly unhinged ways in more and more layers.

Which is incidentally more or less the only thing I remember about Global Workspace Theory (attention facilitating consciousness in a way iirc).

Post reply on HN