Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Position: LLMs Can't Jump

openreview.net

Post reply on HN
Print view View original

191–200 of 233 posts

Re: Position: LLMs Can't Jump

#191

Post by epihelix »

gabbagool wrote: post #145
gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

Or, ask yourself the following question: If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it. I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The…

You have to be there to have a subjective experience of the Grand Canyon, for whatever that's worth. (This is essentially the qualia distinction between us and LLMs, writ large.)

But in terms of objective knowledge about the Grand Canyon, reading every article and scholarly work would give you a much better grounding. Unless you're a geologist/ecologist doing actual field research there, actually seeing the Grand Canyon would add little, if anything, to your factual knowledge and understanding of the site.

Re: Position: LLMs Can't Jump

#192

Post by vatsachak »

ACCount37 wrote: post #110
vatsachak wrote: post #95

Earlier quoted context omitted.

Okay here's something LLMs can't. They can't solve problems that are longer than ~10 pages of math. They also can't maintain codebases without supervision. It's because they are have no memory and use various tricks to supplant that fact.

Now, how long before someone rolls out some sort of 10M context hybrid attention active context management monstrosity and ruins this guy's "can't"? Start the clock. My opinion of claims like "LLMs need memory to manage codebases" has also hit the dumpster bin a while ago. Why would knowing how to make a maintainable change to a codebase require any more "memory" than knowing how to play an optimal chess move? The co…

Okay then why do software engineers still exist. And if companies are making an error keeping people on instead of LLMs, why isn't there an all agent company making money off of it?

Re: Position: LLMs Can't Jump

#193

Post by Hunpeter »

cwmoore wrote: post #183
lukifer wrote: post #147

Earlier quoted context omitted.

The lossy compression of language is why we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning. While there can be subtle semantic differences in real codebases (using "null" to mean "unknown" in one context, versus "intentionally blank" in another), there is a much tighter coupling of semantics to meaning (low ambiguity) compared to "love" in English (let…

I have come across a concept that given a file of compressed text files, adding a new text to it expands it more or less depending on how different the new text is from the compressed. The compression series sounds interesting! I think the deltas are growing at different rates.

> given a file of compressed text files, adding a new text to it expands it more or less depending on how different the new text is from the compressed.

The 3b1b videos mention how this concept may be used to find similarities between different languages. Researchers have been able to get results that closely resemble how languages are usually grouped into families.

Re: Position: LLMs Can't Jump

#194

Post by cezart »

gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

In this context llm's remind me of the anecdote of Agassiz and the fish:

"A post-graduate student equipped with honours and diplomas went to Agassiz to receive the final and finishing touches. The great man offered him a small fish and told him to describe it. Post-Graduate Student: “That’s only a sun-fish” Agassiz: “I know that. Write a description of it.” After a few minutes the student returned with the description of the Ichthus Heliodiplodokus, or whatever term is used to conceal the common sunfish from vulgar knowledge, family of Heliichterinkus, etc., as found in textbooks of the subject. Agassiz again told the student to describe the fish. The student produced a four-page essay. Agassiz then told him to look at the fish. At the end of the three weeks the fish was in an advanced state of decomposition, but the student knew something about it."

sourced from: https://nabeelqu.co/understanding

Re: Position: LLMs Can't Jump

#195

Post by unsupp0rted »

cezart wrote: post #194
gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

In this context llm's remind me of the anecdote of Agassiz and the fish: "A post-graduate student equipped with honours and diplomas went to Agassiz to receive the final and finishing touches. The great man offered him a small fish and told him to describe it. Post-Graduate Student: “That’s only a sun-fish” Agassiz: “I know that. Write a description of it.” After a few minutes the student returned with the descriptio…

What was the student meant to describe that wasn’t covered in the essay?

Re: Position: LLMs Can't Jump

#196

Post by Dylan16807 »

AnthonyMouse wrote: post #180
Dylan16807 wrote: post #168

Earlier quoted context omitted.

There's so much natural wonder in the world in and around your home. It's not that the value of the grand canyon is low in an absolute sense, it's that in a relative sense the gap between "empty numb void" and "hiking at home (plus grand canyon videos)" is far far greater than the gap between "hiking at home (plus grand canyon videos)" and "actual grand canyon".

> in a relative sense the gap between "empty numb void" and "hiking at home (plus grand canyon videos)" is far far greater than the gap between "hiking at home (plus grand canyon videos)" and "actual grand canyon". Have you ever been to a place which is really high up? You're above the trees and can not only see as far as the curvature of the earth allows you to, but are high enough that the curvature of the earth al…

I haven't been that high but I've been in places with some pretty impressive views. And the grand canyon doesn't give you that mountaintop experience either.

Back to the topic, I'm sure it's very impressive, but I think you're underestimating just how impressive our baseline is.

Re: Position: LLMs Can't Jump

#197

Post by morgoths_bane »

unsupp0rted wrote: post #195
cezart wrote: post #194

Earlier quoted context omitted.

In this context llm's remind me of the anecdote of Agassiz and the fish: "A post-graduate student equipped with honours and diplomas went to Agassiz to receive the final and finishing touches. The great man offered him a small fish and told him to describe it. Post-Graduate Student: “That’s only a sun-fish” Agassiz: “I know that. Write a description of it.” After a few minutes the student returned with the descriptio…

What was the student meant to describe that wasn’t covered in the essay?

No man ever fishes the same fish twice, for he and the fish change. -Heraklitos when eating a tuna sandwich.

Re: Position: LLMs Can't Jump

#198

Post by bryanrasmussen »

schubidubiduba wrote: post #169
bryanrasmussen wrote: post #164

Earlier quoted context omitted.

while I said I was not a scientific historian I am well enough read that I can say no, the tradition of citing all sources is a relatively modern invention. Definitely not widespread in Aristotle's time, and not extensively practiced by Aristotle himself who very seldom names any source and their work, but I was not sure if in Newton's time it was already beginning to be established as I have not actually read many s…

You need to differentiate between citing everything, and citing a select few prior works which are integral to your own. For the latter, it is very natural to mention and cite them, unless you have motive not to do so.

that, however, was not what was said - people were arguing about people not citing, and Newton not citing, and I asked if it was expected to cite them at that time. At which point another person argued that yes it was and that even back in Aristotle's time it was natural to cite, which the use of Aristotle, the guy whose predecessors seemed to almost all be either "some guy", "people say", or "those pesky Pythagoreans", as an example of someone who was naturally prone to cite put me a bit on edge.

So it seems you have established that no, Newton was not expected to cite everyone in his time, very well, this now puts the question further up the chain over my entry into the conversation - "Were the people Newton neglected to cite actually integral to his work?"

My definition of integral would be, two possible definitions:

1. do I write something here that someone reading it would say, "What?! What do you mean, explain yourself, how did you arrive at this surprising bit of information?!!" then it is integral.

2. Am I doing this based on some work that someone did without which I would definitely not be doing this? If I am doing it with knowledge of the prior work but whether or not the prior work existed I would still be doing it, the prior work is not actually integral.

Re: Position: LLMs Can't Jump

#200

Post by nullbio »

lukifer wrote: post #147
gabbagool wrote: post #143

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

The lossy compression of language is why we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning. While there can be subtle semantic differences in real codebases (using "null" to mean "unknown" in one context, versus "intentionally blank" in another), there is a much tighter coupling of semantics to meaning (low ambiguity) compared to "love" in English (let…

I think LLMs perform better at code than human language tasks because there's no clear way to eval human language tasks in a non-ambiguous or concrete manner. Any sort of eval that happens around language is transformation tasks, which have deterministic properties or goals. The fuzzy side of human communication can only be modeled probabilistically because there are no clear boundaries. Human communication is more like a felt mutual agreement where the correct interpretation is generally determined by popularity.
Post reply on HN