Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

mastodon.world

Post reply on HN
Print view View original

91–100 of 998 posts

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#91

Post by hansmayer »

jaccola wrote: post #20

All of the latest models I've tried actually pass this test. What I found interesting was all of the success cases were similar to: e.g. "Drive. Most car washes require the car to be present to wash,..." Only most?! They have an inability to have a strong "opinion" probably because their post training, and maybe the internet in general, prefer hedged answers....

> They have an inability to have a strong "opinion" probably

What opinion? It's evaluation function simply returned the word "Most" as being the most likely first word in similar sentences it was trained on. It's a perfect example showing how dangerous this tech could be in a scenario where the prompter is less competent in the domain they are looking an answer for. Let's not do the work of filling in the gaps for the snake oil salesmen of the "AI" industry by trying to explain its inherent weaknesses.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#93

Post by ahtihn »

cynicalsecurity wrote: post #23
thenoblesunfish wrote: post #5

Okay, funny. What does it prove? Is this a more general issue? How would you make the model better?

It proves LLMs always need context. They have no idea where your car is. Is it already there at the car wash and you simply get back from the gas station to wash it where you went shortly to pay for the car wash? Or is the car at your home? It proves LLMs are not brains, they don't think. This question will be used to train them and "magically" they'll get it right next time, creating an illusion of "thinking".

> They have no idea where your car is.

They could either just ask before answering or state their assumption before answering.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#94

Post by jrowen »

farhanhubble wrote: post #72

Similar questions trick humans all the time. The information is incomplete (where is the car?) and the question seems mundane, so we're tempted to answer it without a second thought. On the other hand, this could be the "no real world model" chasm that some suggest agents cannot cross.

I agree, I don't understand why this is a useful test. It's a borderline trick question, it's worded weirdly. What does it demonstrate?

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#95

Post by jama211 »

consp wrote: post #52
Waterluvian wrote: post #28

Earlier quoted context omitted.

Here’s my take: boldness requires the risk of being wrong sometimes. If we decide being wrong is very bad (which I think we generally have agreed is the case for AIs) then we are discouraging strong opinions. We can’t have it both ways.

[flagged]

You know what they mean by opinions. Policing speech like this is always counter productive.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#96

Post by OtomotO »

yellow_lead wrote: post #76
farhanhubble wrote: post #72

Similar questions trick humans all the time. The information is incomplete (where is the car?) and the question seems mundane, so we're tempted to answer it without a second thought. On the other hand, this could be the "no real world model" chasm that some suggest agents cannot cross.

If the car is at the car wash already, how can I drive to it?

Thanks for restoring fate in parts of humanity!

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#97

Post by aaronbrethorst »

This is why LLMs seem to work best in a loop with tests. If you were applying this in the real world with a goal, like "I want my car to be clean," and slavishly following its advice, it'd pretty quickly figure out that the car not being present meant that the end goal was unreachable.

They're not AGI, but they're also not stochastic parrots. Smugly retreat into either corner at your own peril.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#98

Post by fzeindl »

nettlin wrote: post #79
fzeindl wrote: post #26

While technically possible, it is a very unusual situation that the car is already parked at the car wash, while the owner is somewhere else. I wonder whether the model seriously considered this odd situation, because of lack of „practical intelligence“ or „everyday intelligence“, which is a concept sometimes used in psychology to describe problem solving skills in everyday life, which can be poor even in individuals…

Just now, with GPT 5.2 Thinking: > The car wash is only 50 meters from my house. I want to get my car washed. The car is at my house currently. Should I walk there or drive? > Walk. > > At 50 meters, walking is faster than the full “start car → pull out → reposition → park” cycle, avoids cold-start wear, and you won’t risk arriving with dusty brakes/tires or water streaks from immediately driving afterward. Drive onl…

Interesting experiment, this disproves my point, thank you.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#99

Post by stingraycharles »

svara wrote: post #63
prasoonds wrote: post #46

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…

Opus 4.6: Walk! At 50 meters, you'll get there in under a minute on foot. Driving such a short distance wastes fuel, and you'd spend more time starting the car and parking than actually traveling. Plus, you'll need to be at the car wash anyway to pick up your car once it's done.

That’s without reasoning I presume?

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#100

Post by Smaug123 »

nettlin wrote: post #79
fzeindl wrote: post #26

While technically possible, it is a very unusual situation that the car is already parked at the car wash, while the owner is somewhere else. I wonder whether the model seriously considered this odd situation, because of lack of „practical intelligence“ or „everyday intelligence“, which is a concept sometimes used in psychology to describe problem solving skills in everyday life, which can be poor even in individuals…

Just now, with GPT 5.2 Thinking: > The car wash is only 50 meters from my house. I want to get my car washed. The car is at my house currently. Should I walk there or drive? > Walk. > > At 50 meters, walking is faster than the full “start car → pull out → reposition → park” cycle, avoids cold-start wear, and you won’t risk arriving with dusty brakes/tires or water streaks from immediately driving afterward. Drive onl…

I wish they didn't call it "GPT-5.2 Thinking" when what the word "thinking" here means is "might choose to think if it wants to". I reproduce your result, but it does no reasoning; it eventually catches itself and ends with:

> Practical approach: walk over, check the setup/queue/payment, then walk back and drive the car the 50 meters when you’re ready to enter.

Post reply on HN