Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Position: LLMs Can't Jump

openreview.net

Post reply on HN
Print view View original

11–20 of 233 posts

Re: Position: LLMs Can't Jump

#11

Post by Normal_gaussian »

jvanderbot wrote: post #2

Came for: "A computer once beat me at chess, but it was no match for me at kick boxing." TFA was actually about leaps of intuition, sadly. One of the experiments I've heard proposed around here is to somehow create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.

The curious case here is how much of a description do we give it of itself? That would almost certainly dominate success rates.

My feeling is that a prompt would have to provide a vague description of a program that meaningfully passes something like a Turing test, an API to conform to, an expectation of novel construction (no 'ifs all the way down'), and then a requirement to search broadly and pursue promising ideas and not get hung up on the philosophy. Anything more precise feels like it would corrupt the test, but as it is that description feels doomed to loop before even trying the interesting parts.

Re: Position: LLMs Can't Jump

#12

Post by Mistletoe »

I feel like you could just add some noise or randomness to the LLM and start approximating the leaps that the human mind uses to solve and understand unrelated things. Maybe that’s naive, it’s just coming from my organic computer in my skull.

Re: Position: LLMs Can't Jump

#13

Post by Der_Einzige »

Mistletoe wrote: post #12

I feel like you could just add some noise or randomness to the LLM and start approximating the leaps that the human mind uses to solve and understand unrelated things. Maybe that’s naive, it’s just coming from my organic computer in my skull.

Hahaha you just derived temperature from first principles.

Turns out temperature is pretty bad too, you can find ways to sample from deeper in the distribution without distorting it. Great example is XTC (exclude top choices), In a few weeks/months it'll also have a proper scholarly paper with peer review.

Re: Position: LLMs Can't Jump

#14

Post by Spacecosmonaut »

sobiolite wrote: post #10

The theory is that creative leaps in theoretical physics require a grounding in sensory experience, but the obvious counter-argument is that humans can make creative leaps in abstract fields without such sensory grounding. They do address this at the end, saying "In abstract domains such as Mathematics or Computer Science, the Sense Experience (E) may be grounded in high-dimensional topology or have other goals such…

Isnt the sensory grounding even in abstract cases some (limited) intuition that simulates in a mental world model?

Re: Position: LLMs Can't Jump

#15

Post by ModernMech »

Normal_gaussian wrote: post #11
jvanderbot wrote: post #2

Came for: "A computer once beat me at chess, but it was no match for me at kick boxing." TFA was actually about leaps of intuition, sadly. One of the experiments I've heard proposed around here is to somehow create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.

The curious case here is how much of a description do we give it of itself? That would almost certainly dominate success rates. My feeling is that a prompt would have to provide a vague description of a program that meaningfully passes something like a Turing test, an API to conform to, an expectation of novel construction (no 'ifs all the way down'), and then a requirement to search broadly and pursue promising idea…

I wonder if we could just tell it to invent itself without any description and see if it can I introspect enough through its own interface to figure out what it is.

Re: Position: LLMs Can't Jump

#16

Post by bob001 »

sobiolite wrote: post #10

The theory is that creative leaps in theoretical physics require a grounding in sensory experience, but the obvious counter-argument is that humans can make creative leaps in abstract fields without such sensory grounding. They do address this at the end, saying "In abstract domains such as Mathematics or Computer Science, the Sense Experience (E) may be grounded in high-dimensional topology or have other goals such…

That's an interesting analogy. My gut sense is that theoretical mathematics requires a high level of intelligence versus more grounded domains. That may imply that deficiency in grounding can be made up for with intelligence and basically reverse engineering the gaps in grounding from first principles/limited grounding. The ultimate question would then be what is the tradeoffs between grounding and raw intelligence for the same outcome.

Re: Position: LLMs Can't Jump

#17

Post by elar_verole »

jvanderbot wrote: post #2

Came for: "A computer once beat me at chess, but it was no match for me at kick boxing." TFA was actually about leaps of intuition, sadly. One of the experiments I've heard proposed around here is to somehow create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.

I think this can't work because an LLM needs too much data, and before the internet there probably just wasn't enough to get close to what we have now

Re: Position: LLMs Can't Jump

#18

Post by throwaway314155 »

jvanderbot wrote: post #2

Came for: "A computer once beat me at chess, but it was no match for me at kick boxing." TFA was actually about leaps of intuition, sadly. One of the experiments I've heard proposed around here is to somehow create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.

Article was plenty interesting to me.

Re: Position: LLMs Can't Jump

#19

Post by GodelNumbering »

I have been writing a 'paper' [1] on an adjacent topic for months now. At some point, I decided to make it an empirical paper vs position paper. I am still chasing the experiments (when I get some free time waiting for agentic loops)

For this paper specifically, after reading the abstract [2], I felt almost certain that the author would have used Judea Pearl's ladder of causation (https://web.cs.ucla.edu/~kaoru/3-layer-causal-hierarchy.pdf) but they did not. Would have probably been a better argument to make.

[1] paper in quotes because it may never get published (it is over 20 pages atm). the core argument is that lack of native adjacency resolution makes problems harder and sample inefficient, not impossible

[2] "Using Einstein’s formulation of General Relativity as a case study, we demonstrate that LLMs are structurally incapable of creating new foundational axioms, particularly when observational data is scarce. "

Also, the claim that 'LLMs are structurally incapable of creating new foundational axioms' is provably false depending on where you place 'fundamental'.

Re: Position: LLMs Can't Jump

#20

Post by jvanderbot »

elar_verole wrote: post #17
jvanderbot wrote: post #2

Came for: "A computer once beat me at chess, but it was no match for me at kick boxing." TFA was actually about leaps of intuition, sadly. One of the experiments I've heard proposed around here is to somehow create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.

I think this can't work because an LLM needs too much data, and before the internet there probably just wasn't enough to get close to what we have now

Even simpler: Can GPT-2 anticipate and build Gwen/Deepseek? I think the answer is almost trivially "no", so I wonder what changed?
Post reply on HN