Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Position: LLMs Can't Jump

openreview.net

Post reply on HN
Print view View original

201–210 of 233 posts

Re: Position: LLMs Can't Jump

#201

Post by dandaka »

vatsachak wrote: post #192
ACCount37 wrote: post #110

Earlier quoted context omitted.

Now, how long before someone rolls out some sort of 10M context hybrid attention active context management monstrosity and ruins this guy's "can't"? Start the clock. My opinion of claims like "LLMs need memory to manage codebases" has also hit the dumpster bin a while ago. Why would knowing how to make a maintainable change to a codebase require any more "memory" than knowing how to play an optimal chess move? The co…

Okay then why do software engineers still exist. And if companies are making an error keeping people on instead of LLMs, why isn't there an all agent company making money off of it?

1/ LLMs are doing increasingly bigger share of work of SWE with an increasing success rate

2/ SWE are doing way more than "writing code", and since those areas are less "computational verifiable" (what is a good architecture that will stand in 5y?) and more "connected to real world" (what are the requirements?), LLMs struggle with them

Re: Position: LLMs Can't Jump

#202

Post by RugnirViking »

wnmurphy wrote: post #106

I had a related insight, but in the domain of humor [1]. LLMs are inherently probabilistic, and there's currently no mechanism for producing an orthogonal directional change in the path traced through a latent space which is also contextually relevant (landing on a punch line). In other words, LLMs are fundamentally incapable of making intuitive/orthogonal leaps in context. It might be possible to add this capability…

"there's currently no mechanism for producing an orthogonal directional change in the path"

idk the harnesses adding in "but wait -- " or some variation every few lines in the thinking trace seems to do great at this.

Re: Position: LLMs Can't Jump

#203

Post by HelloUsername »

nativeit wrote: post #173
HelloUsername wrote: post #23

Earlier quoted context omitted.

You posted this twice now. Why not just edit your original comment?

I don’t need to edit my original comment, it doesn’t contain any mistakes. All six of the links across three comments are each distinct, separate posts going to either the same article, or others with identical headlines.

Ah sorry, I meant this specific link: https://news.ycombinator.com/item?id=49136070, you posted it twice. And I also meant it could've been one post, to avoid nesting.

Re: Position: LLMs Can't Jump

#204

Post by Enginerrrd »

reliablereason wrote: post #27

Clearly LLMs cant do leaps of intuition since their "intuition" is locked after training ends. The only way a LLM can come up with new ideas if the "idea" appeared as a generalisation durring training or if it was achieved using reason in chain of thought.

I think this is a bad way to look at it. LLMs can probably conceive of most things that are representable within the embedding space.

Ordinarily in mathematics there’s a TON of papers to write just combining low level problems with different techniques. Better still, and often considered groundbreaking is borrowing techniques from other fields and adapting them or creating analogous methods to solve problems. A lot of landmark papers have been written this way. This is also what transformers are sort of good at within other contexts. They have super human breadth so I’m hopeful they’ll become real assets in math for a long time. Though the leaps necessary to adapt a technique in a nonobvious way might be too much for a while longer. We’ll see.

Truly novel techniques are quite rate indeed and I don’t know if LLMs can represent them faithfully in their embedding space or not. My inclination is that they probably can most of the time, but I don’t know. Mathematicians would describe such thins as “alien”.

Re: Position: LLMs Can't Jump

#205

Post by scotty79 »

gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

The entirety of our understanding of reality we owe to lossy transformations. Also called modelling.

Re: Position: LLMs Can't Jump

#206

Post by scotty79 »

lukifer wrote: post #147
gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

The lossy compression of language is why we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning. While there can be subtle semantic differences in real codebases (using "null" to mean "unknown" in one context, versus "intentionally blank" in another), there is a much tighter coupling of semantics to meaning (low ambiguity) compared to "love" in English (let…

> we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning

Math is pure reasoning and stock LLM is already better at it than 99.9% of humans.

And as for human language, agentic LLM can produce a perfectly human text. The fact that AI texts have tells is just because default settings are the same for millions of people using given LLM and almost everybody just pretty much on-shots the text instead of doing it agentically with anti-slop detection and rephrasing in the loop.

Re: Position: LLMs Can't Jump

#207

Post by andai »

andai wrote: post #178
gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

> The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought. The psychical entities which seem to serve as elements in thought are certain signs and more or less clear images which can be “voluntarily” reproduced and combined… The above-mentioned elements are, in my case, of visual and some of muscular type. Conventional words or other signs have to be sought fo…

Although I imagine that requires contact with actual reality? i.e. I would expect a higher degree of such insights to come from RLVR.

Then again maybe the philosophy department has a different opinion :)

Re: Position: LLMs Can't Jump

#208

Post by nullbio »

ACCount37 wrote: post #131
wnmurphy wrote: post #128

Earlier quoted context omitted.

Humor requires a sudden orthogonal leap from context. That's what a punchline is. I think you're saying that you can eventually train models to arrive at that destination by training on existing jokes, effectively encoding these leaps as probabilities. In that case, the model isn't actually making an intuitive/comedic leap; they're just following new probability chains in attempting to approximate examples they've se…

> Try to get a frontier model to write a clever, funny joke which hasn't been seen before. This is what I refer to when it comes to larger models like Fable 5 being funnier. They are more capable of doing that. They can deliver that "sudden orthogonal leap from context" of yours more reliably. It's not a "fundamental inability" and never was. If you crank the scale up and a capability appears, "current architecture"…

These capabilities that emerge are not "orthogonal leaps". They are very much just more of the same thing.

Re: Position: LLMs Can't Jump

#209

Post by rzk »

gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

This might be of interest:

Building AGI Using Language Models – https://bmk.sh/2020/08/17/Building-AGI-Using-Language-Models...

Re: Position: LLMs Can't Jump

#210

Post by potamic »

jimbo808 wrote: post #72
quantum_mcts wrote: post #30

The popular retelling of how Einstein created Special Relativity to "Resolve the contradictions of Michelson-Morly experiments" is very reductive to the history of the question. The epitome is the quote from the paper: > From the two postulates, Einstein derived the Lorentz trans- formation ... If Einstein derived them, who is "Lorentz"? The groundwork for Special Relativity was the study of electrodynamics and symme…

I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived, when he doesn’t even belong in that conversation. If Einstein never existed, the fields he was in were already moving strongly in the directions of his conclusions. If Maxwell or Newton had never existed, the world today would be quite different.

Leading physicists at one time appeared to disagree. Newton and Maxwell do come at #2 and #3 though.

http://news.bbc.co.uk/2/hi/science/nature/541840.stm

Post reply on HN