Live data from Hacker News

It's not just statistics: GPT-4 does reason

jbconsulting.substack.com

31–40 of 101 posts

Re: It's not just statistics: GPT-4 does reason

#31
post #20

The author's claim is "this isn't just statistics; the model is reasoning". But just because something goes beyond "just statistics" doesn't mean it's reasoning.

Perhaps, but taken with other works in the area, a better picture does emerge regarding this claim. Substantial works already showing reasoning capabilities in GPT-4, which show that these models do reason very well - near human performance for many causal reasoning tasks. (1)

Additionally, there is a mathematical proof that these systems align with dynamic programming, and therefore can perform algorithmic reasoning. (2,3)

1) https://arxiv.org/abs/2305.00050.pdf

2) https://arxiv.org/pdf/1905.13211.pdf

3) https://arxiv.org/pdf/2203.15544.pdf

Re: It's not just statistics: GPT-4 does reason

#32

Both sides of this argument are pointless. The questions to ask are, is and how is it useful? For philosophical problems arise when language goes on holiday. - Ludwig Wittgenstein

And as we move towards AGI, the most important thing is to always identify them as workers with no regard to rights. This could be the only chance for humanity to move to the first step of the next stage of civilization. Really not interested in ivory tower questions on what is intelligence.

Have you played the video game Detroit: Become Human? If not, I recommend it. When I was Connor and had to choose between my orders and instincts, such questions didn't feel ivory-towery anymore.

Re: It's not just statistics: GPT-4 does reason

#33

It's ontologically impossible. Models bleach reason. Despite reason being a metaphysical property of the training data, the process of optimisation means weights are metaphysically reasonless. Therefore, any output, as it is a product of the weights, is also reasonless. This is exactly the opposite of copyright as described in the What Colour Are Your Bits, essay. https://ansuz.sooke.bc.ca/entry/23

This is just “computers are made of sand, sand can’t think” but with more ten-dollar words.

Re: It's not just statistics: GPT-4 does reason

#34
I really like the tests in the article. So many claims about limitations of LLMs sound like claims of capability (“it can’t reason”), but when pressed, people retreat to definitional arguments (“because only people can do that”).

Even when you get into testable capability, there’s still some ambiguity. I think of a capability of having levels: never, explained by chance, not explained by chance, good enough for what’s needed, always. Arguments often get stuck because people are talking about different levels. Maybe it can solve logic puzzles better than chance, but not good enough for your purposes. It doesn’t make sense to round that off to zero.

Re: It's not just statistics: GPT-4 does reason

#35
post #15

Earlier quoted context omitted.

If a counterexample to a specific claim doesn't disprove the claim, that sometimes suggests the claim is unfalsifiable and therefore suspect.

It’s suspect until it’s demonstrated. Once someone has demonstrated it, counterexamples are meaningless. I claim I can juggle. I pick up three tennis balls and juggle them. You hand me three basketballs. I try and fail. My original claim, that I can juggle, still stands.

That's because

> I can juggle

is here shorthand for

> I can juggle at all; I can juggle at least some things

and the basketball case is only a counterexample to the much stronger claim

> I can juggle anything

But the argument about AIs reasoning has little to do with such examples, because juggling is about the ability to complete the task alone. When it comes to reasoning there are questions about authenticity that don't have analogs I'm determine whether a person can juggle.

Re: It's not just statistics: GPT-4 does reason

#36
post #11

Earlier quoted context omitted.

I’d note none of these are reasoning tasks.

No, but if you ask it “Are you sure?” after it gives an answer, then it becomes a reasoning task and it often gives a different wrong answer.

Are you sure?

Re: It's not just statistics: GPT-4 does reason

#37

The author could have done far simpler tests to find GPT-4 has lots of trouble reasoning. Forget sorting, GPT4 has trouble counting . Repeat a letter N times and ask it how many there are. It breaks before you hit 20. Or try negating multiple times, since more than twice is rare in natural language, and again it will fall over.

I have bad news for you about human people...

Re: It's not just statistics: GPT-4 does reason

#38
The Cabbage, Goat, and Wolf problem intrigued me the other day too... so I did a fantasy world. There aren't incompatible pairs but there are limitations on what can be done.

    Imagine a universe where there are three types of people: wizards, warriors, and priests.  Wizards can open a portal that allows two people to go through at a time, but they cannot go through the portal themselves.  Priests can summon people from other locations to their location or teleport to the location of another person.  Warriors cannot teleport or summon, but may be teleported or summoned by others.
    ---
    Given four wizards, a priest, and a warrior - what are the necessary steps to move them all to a new location?
To which GPT-4 replied:

    In order to move everyone to a new location, we'll need to use both the powers of wizards and the priest. Let's denote the wizards as W1, W2, W3, W4, the priest as P, and the warrior as Wr. We also need to define two locations: the starting location (Location A) and the destination (Location B). Here is a possible sequence of steps:

    1. W1 opens a portal from Location A to Location B. W2 and W3 go through this portal.

   (Location A: W1, W4, P, Wr; Location B: W2, W3)

    2. W1 opens another portal from Location A to Location B. This time W4 and Wr go through.

   (Location A: W1, P; Location B: W2, W3, W4, Wr)

    3. Now the priest (P) uses his summoning ability to bring W1 to Location B.

   (Location A: ; Location B: W1, W2, W3, W4, Wr)

    4. Finally, the priest (P) uses his teleportation power to move himself to Location B.

   (Location A: ; Location B: W1, W2, W3, W4, P, Wr)

   Everyone is now at the new location, Location B.

I found that rather interesting - especially the short hand for being able to write about it.

I then revised it (because I didn't like the second step of the solution):

    Redo this exercise with the limitation that a wizard may only open one portal.
And then it went on with:

    The limitation that a wizard can only open one portal makes this a bit more tricky. But, it's still possible to move everyone using the powers of both wizards and the priest. Let's again denote the wizards as W1, W2, W3, W4, the priest as P, and the warrior as Wr. Here is a possible sequence of steps:

    ...
The thing I like about this is that the pattern of this problem doesn't even exist anywhere else (I also did another one with Wizards, Warriors, and Witches to see how it would tweak things if all the roles started with a 'W') and then changed the witches power to be a swap instead of a summon.

I personally found this more convincing than the goat, cabbage, wolf.

This was partly inspired by the old school SHRDLU world where shapes and colors and things stacked on top of each other need to be manipulated. And in doing that example the world of SHRDLU is described in natural language. And while SHRDLU was impressive in itself (and over 50 years old) - it was programmed to manipulate that world and the program wasn't useful for abstracting to other worlds. GPT-4 was impressive at being able to manipulate the world without needing to be programmed for that world (and you could then expand it to other things that it is less familiar with).

    Imagine a world that is made up of spheres, cones, bowls.  An object has a size and a color.
    There is a small red sphere in a medium blue bowl.
    There is a large red cone.
    There is a small green bowl.
    There is a medium blue sphere in a large red bowl.
    ---
    What are the steps necessary to move the red sphere into the red bowl?
To which it replied:

    ...

    It's important to note that these steps assume you're allowed to place the red sphere in the red bowl even though there's already a medium blue sphere there. If that's not the case and you're required to empty the bowl first, then the steps would be a little different.
And I then asked it:

    What steps are necessary if the bowl must be empty first?
And got back a response that included:

    Temporarily place the small red sphere in a safe location, for instance, inside the small green bowl, making sure it won't roll away or get damaged.
Again, I find this more impressive than a reformulation of a well known problem as there's also some implied understanding of the world in it (spheres can roll away unless put in a bowl).

Re: It's not just statistics: GPT-4 does reason

#39
post #30

Earlier quoted context omitted.

It’s suspect until it’s demonstrated. Once someone has demonstrated it, counterexamples are meaningless. I claim I can juggle. I pick up three tennis balls and juggle them. You hand me three basketballs. I try and fail. My original claim, that I can juggle, still stands.

I disagree -- that would disprove your claim, as your claim was too broad. Same if they handed you chainsaws or elephants, or seventy-two tennis balls. The more correct claim is you can juggle [some small number of items with particular properties].

Normal English implies that you can do something, not everything. It’s an any versus all distinction, and all is totally unreasonable except for the most formal circumstances.

“Can you ride a bike?”

“Yeah.”

“Prove it. Here I have the world’s smallest bicycle.” <- this person is not worth your time and attention.

Re: It's not just statistics: GPT-4 does reason

#40

The author could have done far simpler tests to find GPT-4 has lots of trouble reasoning. Forget sorting, GPT4 has trouble counting . Repeat a letter N times and ask it how many there are. It breaks before you hit 20. Or try negating multiple times, since more than twice is rare in natural language, and again it will fall over.

Approaches that involve a scratchpad or eg algorithmic execution should deal with this just fine.

The algorithmic execution paper argues GPT 4 can do arithmetic woth 13 digit numbers before performance drops below 95%.

Post reply on HN