Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's not just statistics: GPT-4 does reason

jbconsulting.substack.com

Post reply on HN
Print view View original

61–70 of 101 posts

Re: It's not just statistics: GPT-4 does reason

#61

Post by EGreg »

sidlls wrote: post #56
shagie wrote: post #38

The Cabbage, Goat, and Wolf problem intrigued me the other day too... so I did a fantasy world. There aren't incompatible pairs but there are limitations on what can be done. Imagine a universe where there are three types of people: wizards, warriors, and priests. Wizards can open a portal that allows two people to go through at a time, but they cannot go through the portal themselves. Priests can summon people from…

That doesn't seem that impressive. The likelihood of reading text having "sphere" in the same context (i.e. within some small number of tokens) as "roll away" is higher than by random chance, because humans have observed and described this behavior in text. There's no indication that GPT4 understands what "roll away" means in any meaningful way: just that it associates the phrase with the word "sphere". It might have…

For now I say: LLMs have extremely broad knowledge but shallow. Anything you find that is deep is likely scraped from millioms humans recording insights somewhere.

But what I am most interested by is the degree of its symbolic manipulation and abstract reasoning given messy data. How is that not intelligence ?

Re: It's not just statistics: GPT-4 does reason

#62

Post by jbay808 »

eximius wrote: post #59
jbay808 wrote: post #58

Earlier quoted context omitted.

I just can't imagine how a stochastic parrot could repeat back a correctly-sorted list that it hasn't seen in training, without actually implementing a sorting algorithm in the process. It seems (and, by calculation, is) phenomenally unlikely that it would just stochastically happen to pick every single number correctly. When that is combined with the fact that transformers provably can implement proper deterministic…

Again, I think the disagreement is not whether it has learned to approximate a sorting algorithm, but whether that qualifies as reasoning and, if it does, in what sense.

I won't take a hard stance on what counts as "reasoning", which I picked in the title for lack of a better summarizing word; I am open to alternatives. So if you think that making abstractions and implementing a sorting algorithm does not count as reasoning, I will not disagree with that position. Where I am going to take a hard stance is on what does a stochastic parrot cannot do. And a stochastic parrot, defined as "stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning", cannot sort lists of 127 characters.

Re: It's not just statistics: GPT-4 does reason

#63

Post by krainboltgreene »

fasterik wrote: post #25
jxf wrote: post #15

Earlier quoted context omitted.

If a counterexample to a specific claim doesn't disprove the claim, that sometimes suggests the claim is unfalsifiable and therefore suspect.

The claim is that GPT-4 can reason sometimes. Evidence that GPT-4 fails to reason sometimes isn't a counterexample.

The people who made GPT-4 have said it does not reason, please for the love of god drop this nonsense.

Re: It's not just statistics: GPT-4 does reason

#64

Post by flandish »

jbay808 wrote: post #3
dataangel wrote: post #2

The author could have done far simpler tests to find GPT-4 has lots of trouble reasoning. Forget sorting, GPT4 has trouble counting . Repeat a letter N times and ask it how many there are. It breaks before you hit 20. Or try negating multiple times, since more than twice is rare in natural language, and again it will fall over.

Author here. Happy to see this discussion. Absolutely, GPT-4 sometimes has trouble reasoning and doesn't reason perfectly. I'm impressed by its successes, but I agree it's not at the human level yet, and I would not make the claim that it is. Counting is a task that transformers can do, per Weiss.[1] But it's not surprising that transformer networks in general have trouble counting characters -- the tokenizer replace…

It has trouble “reasoning” because that is a human phenomenon. These ML driven LLMs or “AI” systems are, truly, “word calculators.”

They will never achieve “reason” or understand what it means to do so; they are not human.

Sure, with enough input (in the form of LLM) it can predict what a human’s reasoning may look like, but philosophically, that’s a different thing.

Reason is not universal like how math is.

Re: It's not just statistics: GPT-4 does reason

#65

Post by flandish »

xwdv wrote: post #11
fnordpiglet wrote: post #7

Earlier quoted context omitted.

I’d note none of these are reasoning tasks.

No, but if you ask it “Are you sure?” after it gives an answer, then it becomes a reasoning task and it often gives a different wrong answer.

No, it does not become a reasoning task. The human asking “are you sure?” is actually just inputting words into the model.

The model outputs what it predicts a statistically normal output would fit in the context given.

Truly “llm” and these gpt tools are very much large scale “soundex” models.

Fantastic and great.

But not ai or even agi.

Re: It's not just statistics: GPT-4 does reason

#66

Post by flandish »

roywiggins wrote: post #8
kelseyfrog wrote: post #5

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

Okay, what would you call it when a model behaves like it's reasoning? Some models can't behave that way and some can, so we need some language to talk about these capabilities. Insisting that we can't call these capabilities "reasoning" for ontological reasons seems... unlikely to persuade. Maybe we should call human reasoning "reasoning" and what models do "reasoning₂". "reasoning₂" is when a model's output looks l…

I’d call it “meeting spec as defined.”

And that’s the whole problem with this AI / llm / gpt bubble:

Nobody has scientifically or even simply defined the spec, bounds, or even temporal scope on what it “means” to “get to ai.”

Corporations are LOVING that because they can keep profiting off this bubble.

Re: It's not just statistics: GPT-4 does reason

#67

Post by Vanit »

flandish wrote: post #64
jbay808 wrote: post #3

Earlier quoted context omitted.

Author here. Happy to see this discussion. Absolutely, GPT-4 sometimes has trouble reasoning and doesn't reason perfectly. I'm impressed by its successes, but I agree it's not at the human level yet, and I would not make the claim that it is. Counting is a task that transformers can do, per Weiss.[1] But it's not surprising that transformer networks in general have trouble counting characters -- the tokenizer replace…

It has trouble “reasoning” because that is a human phenomenon. These ML driven LLMs or “AI” systems are, truly, “word calculators.” They will never achieve “reason” or understand what it means to do so; they are not human. Sure, with enough input (in the form of LLM) it can predict what a human’s reasoning may look like, but philosophically, that’s a different thing. Reason is not universal like how math is.

The author wrote a thoughtful article attempting to break this down and has humbly popped into the comments to discuss it... Is your direct response really just to cross your arms and say nope? Like, really?

Re: It's not just statistics: GPT-4 does reason

#68

Post by fasterik »

krainboltgreene wrote: post #63
fasterik wrote: post #25

Earlier quoted context omitted.

The claim is that GPT-4 can reason sometimes. Evidence that GPT-4 fails to reason sometimes isn't a counterexample.

The people who made GPT-4 have said it does not reason, please for the love of god drop this nonsense.

I never said that it does. I was pointing out a logical flaw in that person's argument. Also, why are the creators of GPT-4 authorities on what does or doesn't count as reasoning?

Re: It's not just statistics: GPT-4 does reason

#69

Post by dragonwriter »

roywiggins wrote: post #8
kelseyfrog wrote: post #5

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

Okay, what would you call it when a model behaves like it's reasoning? Some models can't behave that way and some can, so we need some language to talk about these capabilities. Insisting that we can't call these capabilities "reasoning" for ontological reasons seems... unlikely to persuade. Maybe we should call human reasoning "reasoning" and what models do "reasoning₂". "reasoning₂" is when a model's output looks l…

> Okay, what would you call it when a model behaves like it's reasoning?

I... wouldn’t. “Behaves like its reasoning” is vague and subjective, and there are a wide variety of un- or distantly-related distinct behavior patterns to which different people would apply that label that may or may not correlate with each other.

I would instead concretely define (sometimes based on encountered examples) concrete terms for specific, objective patterns and capacities of interest, and leave vague quasi-metaphysical labels for philosophizing about AI in the abstract rather than discussions intended to communicate meaningful information about the capacities of real systems.

AI needs more behaviorism, and less appeal to ill-defined intuitions and vague concepts about internal states in humans as metaphorical touchstones.

Re: It's not just statistics: GPT-4 does reason

#70

Post by koonsolo »

flandish wrote: post #64
jbay808 wrote: post #3

Earlier quoted context omitted.

Author here. Happy to see this discussion. Absolutely, GPT-4 sometimes has trouble reasoning and doesn't reason perfectly. I'm impressed by its successes, but I agree it's not at the human level yet, and I would not make the claim that it is. Counting is a task that transformers can do, per Weiss.[1] But it's not surprising that transformer networks in general have trouble counting characters -- the tokenizer replace…

It has trouble “reasoning” because that is a human phenomenon. These ML driven LLMs or “AI” systems are, truly, “word calculators.” They will never achieve “reason” or understand what it means to do so; they are not human. Sure, with enough input (in the form of LLM) it can predict what a human’s reasoning may look like, but philosophically, that’s a different thing. Reason is not universal like how math is.

This is such bullshit. Time and time again this theory has been disproven. "Animals cannot reason", and then oops, sometimes our human brain is holding us back and rats are smarter at the task (https://www.researchgate.net/publication/259652611_More_comp...)

"A computer will never beat a chess master", etc.

Here are the facts for you: our reasoning is done by our brain. Our brain is just a bunch of processes. Those processes can be replicated in a computer. The number of cells and the speed can be improved. And there you have it, a superior reasoning machine.

These "only humans can do X" mostly comes from religion or other superiority bullshit, but in the end humans are not that special, although we seem to like to think so.

Post reply on HN