This is literally an opinion of one dude which is not backed by any kind of quantitative evidence. It's actually possible to answer this question rigorously: 1. Define a scientific result which qualifies as a "jump". They should be frequent enough that they happen every year - otherwise one might say humans can't jump either. 2. Identify all such "jumps" in articles published in 2026, and use LLM with 2025 knowledge…
Position: LLMs Can't Jump
141–150 of 233 posts
Re: Position: LLMs Can't Jump
#142Earlier quoted context omitted.
I agree completely - behaviorally the models have changed drastically due to RLHF, RLVR and now maybe even more so due to agentic harnesses. But the mechanism of prediction hasn’t changed, that was all I was clarifying.
What about multi-token prediction and speculative diffusion? That’s a different mechanism of prediction, even if it serves only to accelerate decoding.
Re: Position: LLMs Can't Jump
#143So, I do sort of buy into this idea that Einstein was simulating the world and running experiments on those simulations in ways that were beyond what you could encode in natural language. Will AI be capable of doing this, if it is bounded by training data that is composed almost entirely on language? One might argue that if AI is training on a lossy encoding/representation of the human experience, how will it be able to simulate anything beyond that experience? Unless it does so in a way that we manage to do when we image objects beyond 3D. But now I'm just rambling.
Re: Position: LLMs Can't Jump
#144Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
Re: Position: LLMs Can't Jump
#145Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it.
I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The point is that maybe the human experience is more vast than what is written about it.
Re: Position: LLMs Can't Jump
#146Interesting!
Induction Vs. Deduction Vs. Abduction!
(You know, if you like Logic, Philosophy, Law, or... just plain different ways to think/reason about something! :-) )
Re: Position: LLMs Can't Jump
#147Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
With apologies if this is common knowledge at this point, 3Blue1Brown has been doing an excellent series on compression, and its relationship to intelligence (or more controversially, that they are one and the same): https://www.youtube.com/watch?v=l6DKRf-fAAM
But that also throws in sharp relief, that there is vastly more to the human experience than intelligence alone: qualia, desire, gut instinct, intuition, emergent creativity. (Whether the "God of the Gaps" for the delta between capabilities of human vs AIs is fixed, or diminishing, or even shrinking to zero, remains an open experiment we're all living through.)
Re: Position: LLMs Can't Jump
#148Earlier quoted context omitted.
I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived, when he doesn’t even belong in that conversation. If Einstein never existed, the fields he was in were already moving strongly in the directions of his conclusions. If Maxwell or Newton had never existed, the world today would be quite different.
Einstein did a lot, he not only published the special and general theories of relativity, he also contributed to quantum physics and was the first to deduce the size of an atom using statistical arguments, essentially proving the existence of atoms.
The size of an atom was computed for the first time 40 years earlier, in 1865, by Johann Josef Loschmidt, who was an Austrian scientist.
Loschmidt had determined the value of what today is called "Avogadro's number", despite the fact that Avogadro had no idea about how one could find the value at that number. The contribution of Avogadro had been the law that ideal gases have the same number of molecules per volume when the pressure and the temperature are the same, but he did not know how that number might be determined.
Based on the results of Loschmidt, which were also used by Maxwell in his works on statistical physics, a few years later, in 1874, George Johnstone Stoney computed the value of the elementary electric charge, which he later named "electron".
Loschmidt was the one who first measured and weighed the atoms, establishing their existence beyond any reasonable doubt. Before him, many believed that the hypothesis of Dalton about the existence of atoms is just a convenient theory for explaining the laws of chemistry, but some other better explanation might be discovered in the future.
Einstein published an explanation of the Brownian motion, which verified that everything fits in the known theories and it matches the expected values. It did not contain any novel number or law, but it was a useful confirmation of the current theories.
In my opinion, the greatest achievement of Einstein has been the paper published by him in which he revealed the existence of the stimulated emission of electromagnetic waves by hot bodies, besides their well known absorption and spontaneous emission.
This paper had tremendous practical applications, leading to the discovery of masers and lasers, without which the modern technology would have never existed.
While Einstein's theory of gravitation was more intellectually challenging, its practical applications are minor, almost negligible. Moreover, even after more than a century we do not know exactly how adequate it is for describing gravitation.
What is certain is that Einstein's theory of gravitation cannot be anything else but an approximation, as it is based on averaging the distribution of matter in space, without taking into account the concentration of matter into elementary particles.
Another contribution of Einstein that is more important than his most frequently cited works is his support for Satyendra Nath Bose, which lead to the recognition of the Bose-Einstein statistics, which together with the Fermi-Dirac statistics plays an essential role in quantum physics.
Re: Position: LLMs Can't Jump
#149Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
Re: Position: LLMs Can't Jump
#150Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
Or, ask yourself the following question: If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it. I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The…