Live data from Hacker News

Position: LLMs Can't Jump

openreview.net

221–230 of 233 posts

Re: Position: LLMs Can't Jump

#221

Earlier quoted context omitted.

> Try to get a frontier model to write a clever, funny joke which hasn't been seen before. This is what I refer to when it comes to larger models like Fable 5 being funnier. They are more capable of doing that. They can deliver that "sudden orthogonal leap from context" of yours more reliably. It's not a "fundamental inability" and never was. If you crank the scale up and a capability appears, "current architecture"…

These capabilities that emerge are not "orthogonal leaps". They are very much just more of the same thing.

Frankly, I doubt the existence of "orthogonal leaps" as a distinct category with distinct properties.

Just another emergent capability - one that follows exactly the same patterns any other emergent capability does.

Re: Position: LLMs Can't Jump

#222

Earlier quoted context omitted.

Leading physicists at one time appeared to disagree. Newton and Maxwell do come at #2 and #3 though. http://news.bbc.co.uk/2/hi/science/nature/541840.stm

That voting has obviously been done by people who have never read the original texts of the people for whom they have voted. Einstein is indeed one of the most important physicists of the 20th century, but it is really meaningless to try to establish a hierarchy between physicists, because their works are like open-source programs, each work is based on the works of the predecessors and it would have been impossible…

> he could extract the essential information from the much more confuse sources that were available at his time

Wasn't this the case with Einstein's relativity as well? From what I understand the maths for it was already developed for many years, but Einstein was the first to tie it all together into a cohesive theory. Ideas like there being no absolute time, speed of light being constant in all frames, seem like a pretty big conceptual leap. I don't know how scientists at the time reacted to it, but defying Newtonian laws at a time when it was considered absolutely sacrosanct, must have been a big deal.

We see the same with general relativity, with ideas like saying gravity and acceleration is the same thing. I think what really impresses people is the radical nature of these ideas, which completely upheaval long-standing, prevailing notions. To do something like that once would be pretty remarkable by itself. But to do it twice, and in both cases, resulting in a new framework that will become the basis for nearly everything else, surely must put them in the conversation for the greatest at the very least.

Re: Position: LLMs Can't Jump

#223

This is literally an opinion of one dude which is not backed by any kind of quantitative evidence. It's actually possible to answer this question rigorously: 1. Define a scientific result which qualifies as a "jump". They should be frequent enough that they happen every year - otherwise one might say humans can't jump either. 2. Identify all such "jumps" in articles published in 2026, and use LLM with 2025 knowledge…

I would argue that the whole concept of a "jump" is contrary to the need of quantitative evidence.

The idea that everything needs to be proven with evidence is what an LLM excels at. If you give it a task such as "analyze this massive dataset and demonstrate that is proves or disproves this concept" or "find a counter argument to this theory", it will dutifully run through the information and give an answer. If you give it the task of "invent a new style of music", it will do poorly.

It will not do poorly from of lack of ability, nor lack of originality, it will do poorly as "style of music" is a human concept, "new" is a human concept and "invent" is a human concept. Being human concepts, these are things without quantitative value. There is no truth or certainty to what is "invented", as others may call it "discovered" or "developed" (developed in the sense of progressing from a previous work). While a trained ear my call a song "Breakcore" and not "Gabber", another would clump it into "EDM".

The only solution to this is to have human origin. As humans will tell the tale of how an idea came about, inspiring similar thought patterns in peers and earning the title of "new". This to me seems to happen more often with art than science or engineering. Although, it is hard to tell if it is due to increased creativity, or ability to convey thoughts.

All of this is impossible for a non-human to do, as they will give evidence to support the conclusion rather than the tale. This will turn into a possible addition to the knowledge of humanity, but can never earn acceptance as a "jump", as humans care little for evidence. Humans are inherently social beings and, as a side effect, will never acknowledge a non-human contribution.

An easy way to put this is a literal jump. The humans at Merriam-Webster define it in part as "to spring into the air". However, a quick peruse of the comments on a video about the "World's Highest Jumping Robot" will find disagreements about the definition of "jumping" and "robot". The device does spring into the air, but it does so without human legs, human movement, or human consideration for safety. Should you instead look at a video on the highest jumping bipedal robot, you will instead find nearly full acceptance that what happened was a "jump" (it should be noted that there is much disagreement in comments about if such videos are AI generated). By my understanding of the world, this is due to human ability to excite brain paths of an action when viewing another entity preforming the action. So if the viewer's brain does not fire the signal of a jump motion happening, the viewer will seek any rationale to explain the action as not being a jump.

TL;DR: It is all based on vibes, and LLMs give the wrong vibe.

Re: Position: LLMs Can't Jump

#225
post #152

Earlier quoted context omitted.

And so I'm skeptical of TFA's claim that "creativity _isn't [just]_ compression" Imho there's also a clean argument against the existence of LLM-understanding: chatbots have been unable to summarize to experts (see my reply to you in the other thread) their own findings. Even after prolonged interrogation. They were unable to _compress_ their own findings. Thus they might not actually understand what they have actual…

I'm not sure, many chat bots summarise just fine as far as I can tell, on subjects I know a lot about.

I observe this too. It's just on the breakthroughs chatbots independently achieved , ie stuff not in the training data..?

Note that there are some differences between the different breakthroughs. The ones that humans were close to solving already were better explained imho

Re: Position: LLMs Can't Jump

#226

Earlier quoted context omitted.

Leading physicists at one time appeared to disagree. Newton and Maxwell do come at #2 and #3 though. http://news.bbc.co.uk/2/hi/science/nature/541840.stm

That voting has obviously been done by people who have never read the original texts of the people for whom they have voted. Einstein is indeed one of the most important physicists of the 20th century, but it is really meaningless to try to establish a hierarchy between physicists, because their works are like open-source programs, each work is based on the works of the predecessors and it would have been impossible…

> That voting has obviously been done by people who have never read the original texts of the people for whom they have voted.

That's a rather wild claim, given that the article says

> The survey was conducted among 100 of today's leading physicists.

Re: Position: LLMs Can't Jump

#227

Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…

The lossy compression of language is why we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning. While there can be subtle semantic differences in real codebases (using "null" to mean "unknown" in one context, versus "intentionally blank" in another), there is a much tighter coupling of semantics to meaning (low ambiguity) compared to "love" in English (let…

I also think they are one and the same. Because a network is a markov chain, with a huge, lossily compressed, state transition table. (that the chains weren't designed by a human is irrelevant to what they are).

given that, you can very mechanically convert it to a statistical compressor by just attaching an entropy coder. This gets very good compression.

Re: Position: LLMs Can't Jump

#228

Earlier quoted context omitted.

But he didn't have one :) I guess unless you use the raw images, I think the cameras fix a lot of the noise caused by the Poisson distribution. I imagine a movie where Einstein get bored in the patent office and decide to open a photography shop, perhaps with very grainy underexposed photography is possible to notice the quantization of photons, and then he gets a Nobel prize. Another bad idea for a movie is a waterm…

There were already prior reported cases of the phenomenon. Einstein provided the mathematical foundation explaining it, in particular that the discharge is quantized, indicating that it came from an electron. Millikan proved it a couple of years later with his oil drop experiment and by measuring the elementary charge of an electron, garnered his own Nobel: https://en.wikipedia.org/wiki/Oil_drop_experiment

I agree, I agree, but I think the guy only read about it and if lucky measured it once in a lab class. I'm trying to imagine a fake world were he could get enough experience in real life to justify the article claim "grounded in his physical intuition".

Re: Position: LLMs Can't Jump

#229

Earlier quoted context omitted.

I don't understand all this dismissal of Einstein. What you just described is that he published a unified theory of Brownian motion, developed a new way of conceptualizing scientific theories via principles with which he derived special and general relativity, contributed foundational work to quantum mechanics, and more, but no, he was not one of humanities' greatest scientists.

Have you even read what I have written? What I have written is not a dismissal of Einstein. On the contrary, I have said that Einstein is the source of a few extremely important discoveries, but those are not the theories cited by most people, who have never read the original work of Einstein but speak about him from hearsay. As I have written above, when speaking about Einstein, almost nobody mentions the discovery…

[flagged]

Re: Position: LLMs Can't Jump

#230

Earlier quoted context omitted.

You need to differentiate between citing everything, and citing a select few prior works which are integral to your own. For the latter, it is very natural to mention and cite them, unless you have motive not to do so.

that, however, was not what was said - people were arguing about people not citing, and Newton not citing, and I asked if it was expected to cite them at that time. At which point another person argued that yes it was and that even back in Aristotle's time it was natural to cite, which the use of Aristotle, the guy whose predecessors seemed to almost all be either "some guy", "people say", or "those pesky Pythagorean…

> I asked if it was expected to cite them at that time.

Newton was accusing Leibniz plagiarising his work on calculus and engaged in a bitter, lifelong priority dispute over the invention of calculus.

Post reply on HN