Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Position: LLMs Can't Jump

openreview.net

Post reply on HN
Print view View original

91–100 of 233 posts

Re: Position: LLMs Can't Jump

#91

Post by wizzwizz4 »

ben_w wrote: post #64
jebarker wrote: post #51

Earlier quoted context omitted.

Just for the sake of clarity: all LLMs up to today are still only doing next-token/autocomplete. The training process got additional stages to shape the model weights, but standalone LLMs are still deployed essentially identically.

If you gave GPT-2 a question and ended with a "?", it might answer, but also it might write several more questions in a similar category. IMO, the mechanism isn't the important thing, the behaviour is. If you look at the step-by-step, we are also looking for the next word or motor action (and for whoever is about to suggest that we humans plan ahead, Transformer-based LLMs have been shown to also do this); as this is…

If you frame it like so:

   Where do birds go when it rains?
   They
then GPT-2 generally doesn't write more questions.

Re: Position: LLMs Can't Jump

#92

Post by _superposition_ »

What I find absolutely fascinating about this paper is that recently I made the leap that physical representation was a necessary ingredient for invention based on my own experience (lack of abundance of evidence) So I intuitively agree with the premise. It's kinda meta.

Re: Position: LLMs Can't Jump

#93

Post by reliablereason »

buzzin__ wrote: post #68
reliablereason wrote: post #27

Clearly LLMs cant do leaps of intuition since their "intuition" is locked after training ends. The only way a LLM can come up with new ideas if the "idea" appeared as a generalisation durring training or if it was achieved using reason in chain of thought.

Or some randomness is aomehowntroduced in the output, which happens after every word, unless you set the temperature to zero.

That would not be intuition that is just randomness. Intuition is not randomness.

A jump in intuition comes from automatic processes reorganising the relational structure of conceptual models. There is no reorganisation of the model durring inference.

Re: Position: LLMs Can't Jump

#95

Post by vatsachak »

ACCount37 wrote: post #33

It's a very shaky position, and the empirical track record of "LLMs can't..." is in itself a reason to call it into doubt. Every "can't" of this nature was followed by a discovery of "they can, just poorly", and then by that "poorly" improving steadily generation to generation. The paper doesn't provide a way to measure or quantify this elusive "jumping" capability, not even as an approximation. It just throws "can't…

Okay here's something LLMs can't. They can't solve problems that are longer than ~10 pages of math. They also can't maintain codebases without supervision. It's because they are have no memory and use various tricks to supplant that fact.

Re: Position: LLMs Can't Jump

#96

Post by inerte »

Maybe LLMs can't, but another form of AI will. I hope nobody is interpreting this as "nothing will never be as good as us".

I see similar thinking in stories of how humanity got here. Religion has thousands of years adapting to this problem, every time we explain something, the goal post moves. Catholics today accept evolution (or least the church does), but it is the "jump" from monkeys to humans where God is the only explanation.

Just 5 years ago we didn't have a technology that knows more about everything than even most experts. We keep coming up with benchmark after benchmark and LLM/AI keeps destroying them. Now we've moved the benchmark to "the jump". Again, maybe it's LLMs or the way we currently do them that can't do this, but eventually something will.

Re: Position: LLMs Can't Jump

#97

Post by glitchc »

gus_massa wrote: post #59
defgeneric wrote: post #24

Worth reposting a follow-up tweet from the author Tom Zahavy [1] after this made the rounds on X/Twitter recently: > A few reflections on my "LLMs Can’t Jump" paper: > My position paper recently got some traction here, so I wanted to share a few thoughts and clarify a few things. > First things first: some people are framing this as "DeepMind is throwing cold water on AI for science" or claiming the paper argues LLMs…

>> Specifically, I focused on the equivalence principle—a key axiom that Einstein formulated through thought experiments grounded in his physical intuition. It's weird because the equivalence principle is very unintuitive. Aristotle's Mechanics does not have it. It took almost two thousand years to discover inertia that is the most simple version of the equivalence principle. Einstein understood the idea of the the e…

> If we continue with Einstein, the photoelectric effect does not aperar in real life.

Of course it does. How do you think your phone camera works?

Re: Position: LLMs Can't Jump

#98

Post by john_strinlai »

jimbo808 wrote: post #72
quantum_mcts wrote: post #30

The popular retelling of how Einstein created Special Relativity to "Resolve the contradictions of Michelson-Morly experiments" is very reductive to the history of the question. The epitome is the quote from the paper: > From the two postulates, Einstein derived the Lorentz trans- formation ... If Einstein derived them, who is "Lorentz"? The groundwork for Special Relativity was the study of electrodynamics and symme…

I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived, when he doesn’t even belong in that conversation. If Einstein never existed, the fields he was in were already moving strongly in the directions of his conclusions. If Maxwell or Newton had never existed, the world today would be quite different.

>I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived

fair, there's lots of really awesome scientists, many of which not talked about even 1/100th as often as einstein

>when he doesn’t even belong in that conversation.

suddenly, the pendulum has swung way too far in the other direction.

Re: Position: LLMs Can't Jump

#99

Post by kurthr »

CyLith wrote: post #84
gus_massa wrote: post #59

Earlier quoted context omitted.

>> Specifically, I focused on the equivalence principle—a key axiom that Einstein formulated through thought experiments grounded in his physical intuition. It's weird because the equivalence principle is very unintuitive. Aristotle's Mechanics does not have it. It took almost two thousand years to discover inertia that is the most simple version of the equivalence principle. Einstein understood the idea of the the e…

That’s funny, when I first learned about the equivalence principle, my first thought was “of course!” I have always found it to be very intuitive. The great leap is being able to frame it that way.

I worked on the early iPod scroll wheel, before there were advertisements for it and it was in common use. I found the UI interaction odd and unintuitive, particularly the "menu" button and the dead end you hit when "playing". Of course by the time the demonstrator ads came out and everyone was talking about how easy it was use, I'd already spent 10s of hours on it, and it WAS second nature. Being first and embedded in the culture of the time has huge UI advantages.

(Note that the HP Chipmunk 9836 also had a scroll selector wheel in 84)

Most people are UI bigots. Once they get used to a first something, they expect everything to work that way and hate learning anew. They get stuck on keyboards, mice, trackpoint nubs, trackpads, trackballs, scroll wheels, or touchscreens and refuse to move on. Of course there are 'objective' performance tests for each including Fitt's test of accuracy and latency, as well as, cognitive load. I guess once you have a hammer, every screw looks like a nail.

So I'd be a bit careful with the "of course!". It may well be obvious only because that is the first mental model you latch onto.

Re: Position: LLMs Can't Jump

#100

Post by ben_w »

wizzwizz4 wrote: post #91
ben_w wrote: post #64

Earlier quoted context omitted.

If you gave GPT-2 a question and ended with a "?", it might answer, but also it might write several more questions in a similar category. IMO, the mechanism isn't the important thing, the behaviour is. If you look at the step-by-step, we are also looking for the next word or motor action (and for whoever is about to suggest that we humans plan ahead, Transformer-based LLMs have been shown to also do this); as this is…

If you frame it like so: Where do birds go when it rains? They then GPT-2 generally doesn't write more questions.

Generally. Sometimes it still did, in my experience.
Post reply on HN