Well it depends on what you mean patched, but I'll be generous and say that it gets the right answer and there are no logical errors. I used to use
Which weighs more, a kilogram of bricks or a pound of feathers
and that worked until the second or third update on GPT 3.5 (when everyone was like "but did you try 3.5?)
Here's a river crossing puzzle with the current GPT-4o that "works"
----------------------------------------------------
> I have a duck, a goose, and a pig that I need to take across a river. If I live the duck and goose alone, the duck will quack. If I leave the pig with the goose the pig will oink. I can only fit two animals in the boat at a time. What is the quickest way to get all animals across the river?
To solve this puzzle, we need to find a way to transport the animals across the river without triggering the quacking or oinking conditions:
1. **First Trip:** Take the duck and the goose across the river. Leave the goose on the other side and take the duck back with you.
2. **Second Trip:** Take the pig across the river. Leave the pig on the other side and bring the goose back with you.
3. **Third Trip:** Take the duck and the goose across the river again.
Now all three animals are across the river, and at no point were the duck and goose left alone, nor were the pig and goose left together. This method ensures no quacking or oinking occurs.
----------------------------------------------------
Idk why it decided to add the extra constraint of keeping the animals quiet but hey, it doesn't really matter.
As far as I know, I'm the first one that came up with this variant and strangely it has been consistently answered correctly when 4o came out (after my post got repeated by a few bigger names) so I thought it was patched, but just now I wasn't able to reproduce a success with various checkpoints and system prompts, so I guess it got undone?
----------------------------------------------------
> I have a fox, a goose, and a bag of corn that I need to transport across a river. All three can fit in the boat. If you leave the fox alone with the goose, it'll eat the goose. If you leave the goose alone with the corn, it'll eat the corn. Describe how to get all items safely across the river.
To solve this classic puzzle, follow these steps:
1. **Take the Goose Across First:** Start by taking the goose across the river and leave it on the other side.
2. **Return and Take the Fox Across:** Go back to the original side and take the fox across the river.
3. **Bring the Goose Back:** When you reach the other side, leave the fox there and take the goose back with you to the original side.
4. **Take the Corn Across:** Leave the goose on the original side and take the bag of corn across the river.
5. **Return Alone:** Leave the bag of corn with the fox on the other side and return alone to the original side.
6. **Take the Goose Across Again:** Finally, take the goose across the river one last time.
Now, all three items—the fox, the goose, and the corn—are safely across the river without any being eaten.
----------------------------------------------------
claude-3-5-sonnet-20240620 and claude-3-opus-20240229 both act like it is the normal puzzle.
It doesn't matter if you say "All three can fit in the boat with the farmer." Nor if you ask it to explain its logic step-by-step.
The only humans I've seen struggle with this variant ask me to repeat the question several times before saying something similar to "I don't understand the trick. You just take all the animals across at once?" But the trick is there's no trick and I accept that as an answer lol.
FWIW, I've had several LLMs explain to me that all animals can fit in the boat and then proceed to do the normal solution to the puzzle. I've also seen it put all animals into the boat and then only drop one off and travel back and forth, or sometimes animals teleporting or even the framer. Sometimes the farmer is just going back and forth. That happens a lot when I use a variant where animals will follow the farmer and can swim.
Very often I see the better models great at getting the right answers but fail (or even worse) when explaining that answer. I don't think this makes the models useless, but I feel that their highest utility is mostly due to Google's ever decreasing quality. But what can you do?