Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do LLMs have emergent properties?

johndcook.com

Post reply on HN
Print view View original

81–90 of 121 posts

Re: Why do LLMs have emergent properties?

#81

Post by zmmmmm »

This seems superficial and doesn't really get to the heart of the question. To me it's not so much about bits and parameters but a more interesting fundamental question of whether pure language itself is enough to encompass and encode higher level thinking.

Empirically we observe that an LLM trained purely to predict a next token can do things like solve complex logic puzzles that it has never seen before. Skeptics claim that actually the network has seen at least analogous puzzles before and all it is doing is translating between them. However the novelty of what can be solved is very surprising.

Intuitively it makes sense that at some level, that intelligence itself becomes a compression algorithm. For example, you can learn separately how to solve every puzzle ever presented to mankind, but that would take a lot of space. At some point it's more efficient to just learn "intelligence" itself and then apply that to the problem of predicting the next token. Once you do that you can stop trying to store an infinite database of parallel heuristics and just focus the parameter space on learning "common heuristics" that apply broadly across the problem space, and then apply that to every problem.

The question is, at what parameter count and volume of training data does the situation flip to favoring "learning intelligence" rather than storing redundant domain specialised heuristics? And is it really happening? I would have thought just looking at the activation patterns could tell you a lot, because if common activations happen for entirely different problem spaces then you can argue that the network has to be learning common abstractions. If not, maybe it's just doing really large scale redundant storage of heurstics.

Re: Why do LLMs have emergent properties?

#82

Post by kevinsync »

anon373839 wrote: post #66

I remain skeptical of emergent properties in LLMs in the way that people have used that term. There was a belief 3-4 years ago that if you just make the models big enough, they magically acquire intelligence. But since then, we’ve seen that the models are actually still pretty limited by the training data: like other ML models, they interpolate well between the data they’ve been trained on, but they don’t generalize…

My hot take is that what some people are labeling as "emergent" is actually just "incidental encoding" or "implicit signal" -- latent properties that get embedded just by nature of what's being looked at.

For instance, if you have a massive tome of English text, a rather high percentage of it will be grammatically-correct (or close), syntactic and understandable, because humans who speak good English took the time to write it and wrote it how other humans would expect to read or hear it. This, by its very nature, embeds "English language" knowledge due to sequence, word choice, normally-hard-to-quantify expressions (colloquial or otherwise), etc.

When you consider source data from many modes, there's all kinds of implicit stuff that gets incidentally written.. for instance, real photographs of outer space or deep sea would only show humans in protective gear, not swimming next to the Titanic. Conversely, you won't see polar bears eating at Chipotle, or giant humans standing on top of mountains.

There's a statistical probability of "this showed up enough in the training data to loosely confirm its existence" / "can't say I ever saw that, so let's just synthesize it" aspect of the embeddings that one person could interpret as "emergent intelligence", while another could just-as-convincingly say it's probabilistic output that is mostly in line with what we expect to receive. Train the LLM on absolute nonsense instead and you'll receive exactly that back.

Re: Why do LLMs have emergent properties?

#83

Post by gessha »

lordnacho wrote: post #21

What seems a bit miraculous to me is, how did the researchers who put us on this path come to suspect that you could just throw more data and more parameters at the problem? If the emergent behavior doesn't appear for moderate sized models, how do you convince management to let you build a huge model?

There’s that Sinclair quote:

It Is Difficult to Get a Man to Understand Something When His Salary Depends Upon His Not Understanding It

Re: Why do LLMs have emergent properties?

#84

Post by pixl97 »

hibikir wrote: post #36
TheCoreh wrote: post #27

Earlier quoted context omitted.

This is perhaps why it took us this long to get to LLMs, the underlying math and ideas were (mostly) there, and even if the Transformer as an architecture wasn't ready yet, it wouldn't surprise me if throwing sufficient data/compute at a worse architecture wouldn't also produce comparable emergent behavior There needed to be someone willing to try going big at an organization with sufficient idle compute/data just si…

But we got here step by step, as other interesting use cases came up by using somewhat less compute. Image recognition, early forms of image generation, AlphaGo, AlphaZero for chess. All earlier forms of deep neural networks that are much more reasonable than training a top of the line LLM today, but seemed expensive at the time. And ultimately a lot of this also comes from the hardware advancements and the math adva…

> and how much cheaper the compute was getting.

I remember back in the 90s how scientists/data analysts were saying that we'd need exaflop scale systems to tackle certain problems. I remember thinking how foreign that number was when small systems were running maybe tens of megaFLOPS. Now we have systems starting to zettaflops (FP8 so not exact comparison).

Re: Why do LLMs have emergent properties?

#85

Post by pixl97 »

nthingtohide wrote: post #41

What do you think about this analogy? A simple process produces a Mandelbrot set. A simple process (loss minimization through gradient descent) produces LLMs. So what plays the role of 2D-plane or dense point grid in the case of LLMs? It is the embeddings, (or ordered combinations of embeddings ) which are generated after pre-training. In case of a 2D plan, the closeness between two points is determined by our numeri…

> the most peculiar one is that by playing formal games with an input mathematical text, one can get an output text which seemingly carries new knowledge.

Or biologically, DNA/RNA behaves in a similar manner.

Re: Why do LLMs have emergent properties?

#86

Post by disambiguation »

zmmmmm wrote: post #81

This seems superficial and doesn't really get to the heart of the question. To me it's not so much about bits and parameters but a more interesting fundamental question of whether pure language itself is enough to encompass and encode higher level thinking. Empirically we observe that an LLM trained purely to predict a next token can do things like solve complex logic puzzles that it has never seen before. Skeptics c…

Good take, but while we're invoking intuition, something is clearly missing in the fundamental design given real brains don't need to consume all the worlds literature before demonstrating intelligence. There's some missing piece w.r.t self learning and sense making. The path to emergent reasoning you lay out is interesting and might happen anyway as we scale up, but the original idea was to model these algorithms in our own image in the first place - I wonder if we won't discover that missing piece first.

Re: Why do LLMs have emergent properties?

#87

Post by pixl97 »

PaulDavisThe1st wrote: post #67
Workaccount2 wrote: post #52

Earlier quoted context omitted.

The challenge there would be showing that humans have this thing called intelligence. You yourself are just outputting ephemeral actions that rise out of your subconscious. We have no idea what that system feeding our output looks like (except it's some kind of organic neural net) and hence there isn't really a basis for discriminating what is and isn't intelligent besides "if it solves problems, it has some degree o…

To return an old but still good analogy ... If you want to understand how birds fly, the fact that planes also fly is near useless. While a few common aerodynamic principles apply, both types of flight are so different from each other that you do not learn very much about one from the other. On the other hand, if your goal is just "humans moving through the air for extended distances", it doesn't matter at all that a…

doesn't by itself tell you that they are working in remotely the same way or capable of the same sorts of things.

I believe any intelligence that reaches 'human level' should be capable of nearly the same things with tool use, the fact it accomplishes the goal in a different way doesn't matter because the systems behavior is generalized. Hence the term (artificial) general intelligence. Two different general intelligences built on different architectures should be able to converge on similar solutions (for example solutions based on lowest energy states) because they are operating in the same physical realm.

An AGI and an HGI should be able to have convergent solutions for fast air travel, ornithopters, and drones.

Re: Why do LLMs have emergent properties?

#88

Post by pixl97 »

scopemouthwash wrote: post #32

The authors haven’t demonstrated emergence of LLMs. If I write a piece of code and it does what I programmed it to do that’s not emergence. LLMs aren’t doing anything unexpected yet. I think that’s the smell test because emergence is still subjective.

Are you writing the neural networks for LLMs?

Re: Why do LLMs have emergent properties?

#89

Post by pixl97 »

xboxnolifes wrote: post #63
RigelKentaurus wrote: post #59

I'm not a techie, so perhaps someone can help me understand this: AFAIK, no theoretical computer scientist predicted emergence in AI models. Doesn't that suggest that the field of theoretical computer science (or theoretical AI, if you will) is suspect? It's like Lord Kelvin saying that heavier-than-air flying machines are impossible a decade before the Wright brothers' first flight.

Has emergent behavior ever been predicted prior to it being observed in other theoretical fields?

I believe it's been predicted in traffic planning and highway design and tested in via simulation and in field experiments. Use of self driving cars to modify traffic behaviors and decrease traffic jams is a field of study these days.

Re: Why do LLMs have emergent properties?

#90

Post by creer »

gyomu wrote: post #68

The reasoning in the article is interesting, but this struck me as a weird example to choose: > “The real question is how can we predict when a new LLM will achieve some new capability X. For example, X = “Write a short story that resonates with the social mood of the present time and is a runaway hit” Framing a capability as something that is objectively measurable (“able to perform math on the 12th grade level”, “a…

That it seems hard (impossible) or not clear intuitively how to go about it, to us humans, is what makes the question interesting. In a way. The other questions are interesting but a different class of interesting. At any rate, both good for this question. Either way this becomes "what would we need to estimate this emergence threshold?".
Post reply on HN