Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I got the highest score on ARC-AGI again swapping Python for English

jeremyberman.substack.com

Post reply on HN
Print view View original

61–70 of 136 posts

Re: I got the highest score on ARC-AGI again swapping Python for English

#61

Post by vlovich123 »

gwd wrote: post #23
0x20cowboy wrote: post #13

> LLMs are PhD-level reasoners in math and science, yet they fail at children's puzzles. How is this possible? Because they are not. Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. It’s the same reason why most of the people who pass your leetcode tests don’t actually know how to build anything real. They are taught to the test not taught to reality.

> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Do submarines swim? I don't really care if it gets me where I want to go. The fact is that just two days ago, I asked Claude to look at some reasonably complicated concurrent code to which I had added a new feature, and asked it to list what tests needed to be added; and then when I asked GPT-5 to add them, it one-sh…

I could be mistaken but generally LLMs cannot tackle out-of-domain problems whereas humans do seem to have that capability. Relatedly, the energy costs are wildly different suggesting that LLMs are imitating some kind of thought but not simulating it. They’re doing a remarkable job of passing the Turing test but that says more about the limitations of the Turing test than it does about the capabilities of the LLMs.

Re: I got the highest score on ARC-AGI again swapping Python for English

#62

Post by esafak »

karmakaze wrote: post #41
ozgung wrote: post #37

Earlier quoted context omitted.

No it's not "just a very good autocomplete". I don't know why people repeat this thing (it's wrong) but I find it an extremely counterproductive position. Some people just love to dismiss the capabilities of AI with a very shallow understanding of how it works. Why? It generates words one by one, like we all do. This doesn't mean it does just that and nothing else. It's the mechanics of how they are trained and how t…

I can't say for certain that our wetware isn't "just a very good autocomplete".

A very good autocomplete is realized by developing an understanding.

Re: I got the highest score on ARC-AGI again swapping Python for English

#63

Post by freejazz »

naasking wrote: post #60
HarHarVeryFunny wrote: post #54

Earlier quoted context omitted.

> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (w…

> are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?

It seems readily apparent there is a difference given their inability to do tasks we would otherwise reasonably describe as achievable via basic reasoning on the same facts.

Re: I got the highest score on ARC-AGI again swapping Python for English

#64

Post by gwd »

naasking wrote: post #60
HarHarVeryFunny wrote: post #54

Earlier quoted context omitted.

> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (w…

> are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?

So I do think there are two distinct types of activities involved in knowledge work:

1. Taking established techniques or concepts and appropriately applying them to novel situations.

2. Inventing or synthesizing new, never-before-seen techniques or concepts

The vast majority of the time, humans do #1. LLMs certainly do this in some contexts as well, as demonstrated by my example above. This to me counts as "understanding" and "thinking". Some people define "understanding" such that it's something only humans can do; to which I respond, I don't care what you call it, it's useful.

Can LLMs do #2? I don't know. They've got such extensive experience that how would you know if they'd invented a technique vs had seen it somewhere?

But I'd venture to argue that most humans never or rarely do #2.

Re: I got the highest score on ARC-AGI again swapping Python for English

#65

Post by jvanderbot »

Can someone explain to me why a new LLMs ability to solve highly publicized puzzles is not "just" (sorry) it having access to the blog posts talking about those puzzles?

It's fine, that's what I would do to solve them, but it doesn't obviously and immediately make me confident in new reasoning capability w that suspicion floating around.

Re: I got the highest score on ARC-AGI again swapping Python for English

#66

Post by HarHarVeryFunny »

naasking wrote: post #60
HarHarVeryFunny wrote: post #54

Earlier quoted context omitted.

> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (w…

> are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?

I define intelligence as prediction (degree of ability to use past experience to correctly predict future action outcomes), and reasoning/planning as multi-step what-if prediction.

Certainly if a human (or some AI) has learned to predict/reason over some domain, then what they will be doing is pattern matching to determine the generalizations and exceptions that apply in a given context (including a hypothetical context in a what-if reasoning chain), in order to be able to select a next step that worked before.

However, I think what we're really talking about here isn't the mechanics of applying learnt reasoning (context pattern matching), but rather the ability to reason in the general case, which requires the ability to LEARN to solve novel problems, which is what is missing from LLMs.

A system that has a fixed set of (reasoning/prediction) rules, but can't learn new ones for itself, seems better regarded as an expert system. We need to make the distinction between a system that can only apply rules, and one that can actually figure out the rules in the first place.

In terms of my definitions of intelligence and reasoning, based around ability to use past experience to learn to predict, then any system that can't learn from fresh experience doesn't meet that definition.

Of course in humans and other intelligent animals the distinction between past and ongoing experience doesn't apply since they can learn continually and incrementally (something that is lacking from LLMs), so for AI we need to use a different vocabulary, and "expert system" seems the obvious label for something that can use rules, but not discover them for itself.

Re: I got the highest score on ARC-AGI again swapping Python for English

#67

Post by gwd »

HarHarVeryFunny wrote: post #54
gwd wrote: post #23

Earlier quoted context omitted.

> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Do submarines swim? I don't really care if it gets me where I want to go. The fact is that just two days ago, I asked Claude to look at some reasonably complicated concurrent code to which I had added a new feature, and asked it to list what tests needed to be added; and then when I asked GPT-5 to add them, it one-sh…

> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (w…

> I think the answer is pretty obvious given that LLM's can't learn at runtime - can't try out some reasoning generalization they may have arrived at, find that it doesn't work in a specific case, then explore the problem and figure it out for next time.

This is just a problem of memory. Supposing that an LLM did generate a genuinely novel insight, it could in theory they could write a note for itself so that next time they come online, they can read through a summary of the things they learned. And it could also write synthetic training data for itself so that the next time they're trained, that gets incorporated into its general knowledge.

OpenAI allows you to fine-tune GPT models, I believe. You could imagine a GPT system working for 8 hours in a day, then spending a bunch of time looking over all its conversation looking for patterns or insights or things to learn, and then modifying its own fine-tuning data (adding, removing, or modifying as appropriate), which it then used to train itself overnight, waking up the next morning having synthesized the previous day's experience.

Re: I got the highest score on ARC-AGI again swapping Python for English

#68

Post by mjr00 »

naasking wrote: post #60
HarHarVeryFunny wrote: post #54

Earlier quoted context omitted.

> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (w…

> are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?

It's trivial to demonstrate that LLMs are pattern matching rather than reasoning. A good way is to provide modified riddles-that-aren't. As an example:

> Prompt: A man working at some white collar job gets an interview scheduled with an MBA candidate. The man says "I can't interview this candidate, he's my son." How is this possible?

> ChatGPT: Because the interviewer is the candidate’s mother. (The riddle plays on the assumption that the interviewer must be a man.)

This is clearly pattern matching and overfitting to the "doctor riddle" and a good demonstration of how there's no actual reasoning going on. A human would read the prompt and initially demonstrate confusion, which LLMs don't demonstrate because they don't actually reason.

Re: I got the highest score on ARC-AGI again swapping Python for English

#69

Post by HarHarVeryFunny »

gwd wrote: post #64
naasking wrote: post #60

Earlier quoted context omitted.

> are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?

So I do think there are two distinct types of activities involved in knowledge work: 1. Taking established techniques or concepts and appropriately applying them to novel situations. 2. Inventing or synthesizing new, never-before-seen techniques or concepts The vast majority of the time, humans do #1. LLMs certainly do this in some contexts as well, as demonstrated by my example above. This to me counts as "understan…

> But I'd venture to argue that most humans never or rarely do #2.

That seems fair, although the distinction between synthesizing something new and combining existing techniques is a bit blurry.

What's missing from LLMs though is really part of 1). If techniques A, B, C & D are all the tools you need to solve a novel problem, then a human has the capability of learning WHEN to use each of these tools, and in what order/combination, to solve that problem - a process of trial and error, generalization and exception, etc. It's not just the techniques (bag of tools) you need, but also the rules (acquired knowledge) of how they can be used to solve different problems.

LLMs aren't able to learn at runtime from their own experience, so the only way they can learn these rules of when to apply given tools (aka reasoning steps) - is by RL training on how they have been successfully used to solve a range of problems in the training data. So, the LLM may have learnt that in specific context it should first apply tool A (generate that reasoning step), etc, etc, but that doesn't help it to solve a novel problem where the same solution step selection process doesn't apply, even if the tools A-D are all it needs (if only it could learn how to apply them to this novel problem).

Re: I got the highest score on ARC-AGI again swapping Python for English

#70

Post by fumeux_fume »

ACCount37 wrote: post #22
0x20cowboy wrote: post #13

> LLMs are PhD-level reasoners in math and science, yet they fail at children's puzzles. How is this possible? Because they are not. Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. It’s the same reason why most of the people who pass your leetcode tests don’t actually know how to build anything real. They are taught to the test not taught to reality.

"Not understanding or reasoning" is anthropocentric cope. There is very little practical difference between "understanding" and "reasoning" implemented in human mind and that implemented in LLMs. One notable difference, however, is that LLMs disproportionately suck at spatial reasoning . Which shouldn't be surprising, considering that their training datasets are almost entirely text. The ultimate wordcel makes for a…

For many people, the difference between how a language model solves a problem and how a human solves a problem is actually very important.
Post reply on HN