Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's not just statistics: GPT-4 does reason

jbconsulting.substack.com

Post reply on HN
Print view View original

11–20 of 101 posts

Re: It's not just statistics: GPT-4 does reason

#11

Post by xwdv »

fnordpiglet wrote: post #7
dataangel wrote: post #2

The author could have done far simpler tests to find GPT-4 has lots of trouble reasoning. Forget sorting, GPT4 has trouble counting . Repeat a letter N times and ask it how many there are. It breaks before you hit 20. Or try negating multiple times, since more than twice is rare in natural language, and again it will fall over.

I’d note none of these are reasoning tasks.

No, but if you ask it “Are you sure?” after it gives an answer, then it becomes a reasoning task and it often gives a different wrong answer.

Re: It's not just statistics: GPT-4 does reason

#12

Post by Imnimo »

>Does the n-gram model really need all those parameters to mimic GPT-4? Yes, it does.

I don't understand what this argument is supposed to demonstrate. Obviously you can compress the 8000-gram model that GPT-4 represents - GPT-4's weights are proof!

Re: It's not just statistics: GPT-4 does reason

#13

Post by [deleted] »

roywiggins wrote: post #8
kelseyfrog wrote: post #5

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

Okay, what would you call it when a model behaves like it's reasoning? Some models can't behave that way and some can, so we need some language to talk about these capabilities. Insisting that we can't call these capabilities "reasoning" for ontological reasons seems... unlikely to persuade. Maybe we should call human reasoning "reasoning" and what models do "reasoning₂". "reasoning₂" is when a model's output looks l…

[deleted]

Re: It's not just statistics: GPT-4 does reason

#14

Post by jojobas »

kelseyfrog wrote: post #5

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

Human brain output is just as much a product of the weights. What of it?

Re: It's not just statistics: GPT-4 does reason

#15

Post by jxf »

xyzzy123 wrote: post #4
dataangel wrote: post #2

The author could have done far simpler tests to find GPT-4 has lots of trouble reasoning. Forget sorting, GPT4 has trouble counting . Repeat a letter N times and ask it how many there are. It breaks before you hit 20. Or try negating multiple times, since more than twice is rare in natural language, and again it will fall over.

If someone can show GPT-4 is "reasoning" (for some meaningful definition of that) in specific scenarios, surely counter-examples do not disprove this.

If a counterexample to a specific claim doesn't disprove the claim, that sometimes suggests the claim is unfalsifiable and therefore suspect.

Re: It's not just statistics: GPT-4 does reason

#16

Post by svachalek »

lmm wrote: post #10
kelseyfrog wrote: post #5

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

> Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Proof? Human reasoning somehow manages to retain its metaphysical reasoning-ness despite being processed as a bunch of mere electrical signals in the brain.

No true human reasons with mere electrical signals!

Re: It's not just statistics: GPT-4 does reason

#17

Post by jbay808 »

Imnimo wrote: post #12

>Does the n-gram model really need all those parameters to mimic GPT-4? Yes, it does. I don't understand what this argument is supposed to demonstrate. Obviously you can compress the 8000-gram model that GPT-4 represents - GPT-4's weights are proof!

That's right, but if you did that compression, it wouldn't be an n-gram anymore. What I'm attempting to get across is that you could model GPT-4 as an equivalent 8000-gram in an abstract sense, but that's not a good mental picture for how it actually functions. Internally, GPT-4 is no more an 8000-gram than Stockfish is a giant lookup table of chess positions. GPT-4 is learning RASP programs, not statistical text correlations.

Re: It's not just statistics: GPT-4 does reason

#18

Post by IAmNotACellist »

kelseyfrog wrote: post #5

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

Nonsense. The contrary is philosophically arguable: optimization is how reason comes to exist, as goal-oriented reinforcement means that an initially stochastic state loses entropy as it becomes ordered in such a manner (perhaps unknown) that its outputs more closely align with that goal.

Re: It's not just statistics: GPT-4 does reason

#19

Post by pixl97 »

xwdv wrote: post #11
fnordpiglet wrote: post #7

Earlier quoted context omitted.

I’d note none of these are reasoning tasks.

No, but if you ask it “Are you sure?” after it gives an answer, then it becomes a reasoning task and it often gives a different wrong answer.

I mean, in training children we give them reasoning tasks they commonly get wrong. I don't think we say they are incapable of reasoning because they get wrong answers commonly?

This is why we see improvement in GPT when chain of thought/tree of thought is used with reasoning for each step. That can't correct every failure mode, but it increases the likelihood you'll receive a more correct answer.

Post reply on HN