Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Position: LLMs Can't Jump

openreview.net

Post reply on HN
Print view View original

131–140 of 233 posts

Re: Position: LLMs Can't Jump

#131

Post by ACCount37 »

wnmurphy wrote: post #128
ACCount37 wrote: post #122

Earlier quoted context omitted.

You have committed the classic blunder of confusing your abstraction layers. "Probabilistic next word prediction" and "humor" sit about as far apart as "modulating airflow with meat flaps" and "humor" do. One is an interface through which an action is performed and the other is a highly abstract capability. Would you claim that a podcast comedian is fundamentally incapable of being funny because all he ever does is w…

Humor requires a sudden orthogonal leap from context. That's what a punchline is. I think you're saying that you can eventually train models to arrive at that destination by training on existing jokes, effectively encoding these leaps as probabilities. In that case, the model isn't actually making an intuitive/comedic leap; they're just following new probability chains in attempting to approximate examples they've se…

> Try to get a frontier model to write a clever, funny joke which hasn't been seen before.

This is what I refer to when it comes to larger models like Fable 5 being funnier. They are more capable of doing that. They can deliver that "sudden orthogonal leap from context" of yours more reliably.

It's not a "fundamental inability" and never was. If you crank the scale up and a capability appears, "current architecture" was never the problem.

Re: Position: LLMs Can't Jump

#132

Post by jimbo808 »

kskdkwkdjw wrote: post #115
jimbo808 wrote: post #72

Earlier quoted context omitted.

I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived, when he doesn’t even belong in that conversation. If Einstein never existed, the fields he was in were already moving strongly in the directions of his conclusions. If Maxwell or Newton had never existed, the world today would be quite different.

Oh boy. I just cannot believe we have reached a point where someone would find it acceptable to have a temper tantrum over Einstein’s fame. What a depressingly peculiar time to be alive. Reddit is over that way, my friend. You might find the crowd there more amenable to this nonsense.

Calling my comment a “temper tantrum” is the exact kind of behavior you’d expect of a Redditor. You may be projecting.

Einstein’s fame isn’t the problem. It’s the narrative that he is somehow the greatest scientist ever, when it’s just transparently false when you look at his actual contributions and the historical momentum of the fields he contributed to. There’s nothing wrong with his fame, the problem is that it overshadows actual juggernauts, like Maxwell and Newton. Maxwell not being famous at all among the ordinary public is a great tragedy, when he genuinely is in the conversation as having have been the most important physicist to have ever lived.

Re: Position: LLMs Can't Jump

#134

Post by jimbo808 »

dguest wrote: post #101
jimbo808 wrote: post #72

Earlier quoted context omitted.

I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived, when he doesn’t even belong in that conversation. If Einstein never existed, the fields he was in were already moving strongly in the directions of his conclusions. If Maxwell or Newton had never existed, the world today would be quite different.

Yeah, once you dig into economics, technology and science of the era it often feels like the big discoveries were inevitable. That doesn't make these people any less great, it's just to say there are many more great thinkers that people are forgetting about. But I'd like more explanation as to why Maxwell or Newton are any less replaceable: all 4 of "Maxwell's equations" have other names attached to them (the excepti…

[dead]

Re: Position: LLMs Can't Jump

#135

Post by joefourier »

jebarker wrote: post #67
ben_w wrote: post #64

Earlier quoted context omitted.

If you gave GPT-2 a question and ended with a "?", it might answer, but also it might write several more questions in a similar category. IMO, the mechanism isn't the important thing, the behaviour is. If you look at the step-by-step, we are also looking for the next word or motor action (and for whoever is about to suggest that we humans plan ahead, Transformer-based LLMs have been shown to also do this); as this is…

I agree completely - behaviorally the models have changed drastically due to RLHF, RLVR and now maybe even more so due to agentic harnesses. But the mechanism of prediction hasn’t changed, that was all I was clarifying.

What about multi-token prediction and speculative diffusion? That’s a different mechanism of prediction, even if it serves only to accelerate decoding.

Re: Position: LLMs Can't Jump

#136

Post by Mistletoe »

dooglius wrote: post #35
Mistletoe wrote: post #12

I feel like you could just add some noise or randomness to the LLM and start approximating the leaps that the human mind uses to solve and understand unrelated things. Maybe that’s naive, it’s just coming from my organic computer in my skull.

Technically true, but the counter argument would be that the probability of this working would be ~ 2^(-(entropy_of_leap)) for an LLM (presumably intractable) and a human would succeed at a higher probability.

If we could just get the LLMs to take showers and dream, we would get some novel thoughts coming.

Re: Position: LLMs Can't Jump

#137

Post by oliculipolicula »

eru wrote: post #80
kergonath wrote: post #73

Earlier quoted context omitted.

> You can (re-) derive a lot of existing stuff. Einstein was aware of Lorentz and the transform. He was aware of Poincaré as well. He knew the state of the art for his time.

Re-deriving the start of the art cleanly with fewer postulates is good!

And so I'm skeptical of TFA's claim that "creativity _isn't [just]_ compression"

Imho there's also a clean argument against the existence of LLM-understanding: chatbots have been unable to summarize to experts (see my reply to you in the other thread) their own findings.

Even after prolonged interrogation. They were unable to _compress_ their own findings. Thus they might not actually understand what they have actually done. (They might barely pass an oral thesis defense)

(One may object-- that proofs aren't data that can be "compressed". But then doesn't the process of abduction generalise the very idea of data? to.. ?)

Re: Position: LLMs Can't Jump

#138

Post by emp17344 »

killerstorm wrote: post #124

This is literally an opinion of one dude which is not backed by any kind of quantitative evidence. It's actually possible to answer this question rigorously: 1. Define a scientific result which qualifies as a "jump". They should be frequent enough that they happen every year - otherwise one might say humans can't jump either. 2. Identify all such "jumps" in articles published in 2026, and use LLM with 2025 knowledge…

LLMs most definitely are limited. What’s your position, that LLMs have no limits? That’s obviously wrong, and the fact you hold such an unreasonable position may be why you react so strongly to pieces like this one.

Re: Position: LLMs Can't Jump

#139

Post by rowyourboat »

jvanderbot wrote: post #20
elar_verole wrote: post #17

Earlier quoted context omitted.

I think this can't work because an LLM needs too much data, and before the internet there probably just wasn't enough to get close to what we have now

Even simpler: Can GPT-2 anticipate and build Gwen/Deepseek? I think the answer is almost trivially "no", so I wonder what changed?

That's not really a fair comparison, no? Modern LLMs are much more capable than GPT-2. We'ld need a modern LLM trained on exclusively old data, and that might be impossible

Re: Position: LLMs Can't Jump

#140

Post by bigstrat2003 »

inerte wrote: post #96

Maybe LLMs can't, but another form of AI will. I hope nobody is interpreting this as "nothing will never be as good as us". I see similar thinking in stories of how humanity got here. Religion has thousands of years adapting to this problem, every time we explain something, the goal post moves. Catholics today accept evolution (or least the church does), but it is the "jump" from monkeys to humans where God is the on…

> Just 5 years ago we didn't have a technology that knows more about everything than even most experts.

We still don't have that. LLMs have shown time and time again that they don't know a single thing and are incapable of reasoning.

Post reply on HN