Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

mastodon.world

Post reply on HN
Print view View original

381–390 of 998 posts

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#381

Post by tgv »

jstummbillig wrote: post #147
prasoonds wrote: post #46

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

> Us having to specify things that we would never specify

This is known, since 1969, as the frame problem: https://en.wikipedia.org/wiki/Frame_problem. An LLM's grasp of this is limited by its corpora, of course, and I don't think much of that covers this problem, since it's not required for human-to-human communication.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#382

Post by janlukacs »

jb1991 wrote: post #355

Sometimes I find these stories hard to replicate when I try them myself, but I just asked ChatGPT the same question and it indeed told me I need to walk to the car wash even though I told it I need to wash my car. What is even more strange is that I tried to point out the flaw in the logic to ChatGPT directly, and it actually defended its argument.

Don't they still employ armies of people to fix things like this?

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#383

Post by ibestvina »

raincole wrote: post #371
ibestvina wrote: post #364

There's a whole industry of "illusions" humans fail for: optical, word plays (including large parts of comedy), the Penn & Teller type, etc. Yet no one claims these are indicators that humans lack some critical capability. Surface of "illusions" for LLMs is very different from our own, and it's very jagged: change a few words in the above prompt and you get very different results. Note that human illusions are very j…

> No good reason to think "our human illusions" are fine, but "their AI illusions" make them useless. I was about to argue that human illusions are fine because humans will learn the mistakes after being corrected. But then I remember what online discussions over Monty Hall problem look like...

Exactly! I now feel bad for not thinking of that example, thank you.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#384

Post by andy12_ »

hansmayer wrote: post #354
andy12_ wrote: post #344

Earlier quoted context omitted.

No, your opinion is wrong because the reason some models don't seem to have some "strong opinion" on anything is not related to predicting words based on how similar they are to other sentences in the training data. It's most likely related to how the model was trained with reinforcement learning, and most specifically, to recent efforts by OpenAI to reduce hallucination rates by penalizing guessing under uncertainty…

Well, you do understand the "penalising" or as the ML scientific community likes to call it - "adjusting the weights downwards" - is part of setting up the evaluation functions, for gasp - calculating the next most likely tokens, or to be more precise, tokens with the highest possible probability? You are effectively proving my point, perhaps in a bit hand-wavy fashion, that nevertheless still can be translated into…

You do understand that the mechanism through which an auto-regressive transformer works (predicting one token at a time) is completely unrelated to how a model with that architecture behaves or how it's trained, right? You can have both:

- An LLM that works through completely different mechanisms, like predicting masked words, predicting the previous word, or predicting several words at a time.

- A normal traditional program, like a calculator, encoded as an autoregressive transformer that calculates its output one word at a time (compiled neural networks) [1][2]

So saying "it predicts the next word" is a nothing-burger. That a program calculates its output one token at a time tells you nothing about its behavior.

[1] https://arxiv.org/pdf/2106.06981

[2] https://wengsyx.github.io/NC/static/paper_iclr.pdf

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#385

Post by vintermann »

jstummbillig wrote: post #147
prasoonds wrote: post #46

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

But it's a question you would never ask a human! In most contexts, humans would say, "you are kidding, right?" or "um, maybe you should get some sleep first, buddy" rather than giving you the rational thinking-exam correct response.

For that matter, if humans were sitting at the rational thinking-exam, a not insignificant number would probably second-guess themselves or otherwise manage to befuddle themselves into thinking that walking is the answer.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#386

Post by janlukacs »

faeyanpiraat wrote: post #350
nosianu wrote: post #308

Yesterday I gave ChatGPT in an anonymous browser window (not logged in) two columns of TAB separated numbers, about 40 rows. I asked it to give me the weighted average of the numbers in the second column, using the first one (which were integer, "quantity", numbers) as the weight. It retuned formulas and executed them and presented a final result. It looked good. Too bad Excel and then Claude, that I decided to ask t…

Yeah, but now you know if you need to do math, you ask the AI for a python script to do the math correctly. It's just a tool that you get better at using over time; a hammer wouldn't complain if you tried using it as a screwdriver..

This hammer/screwdriver analogy drives me crazy. Yes, it's a tool - we used computers up until now to give us correct deterministic responses. Now the religion is that you need to get used to vibe answers, because it's the future :) Of-course it knows the script or formula for something because it ripped of the answers written by other people - it's a great search engine.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#387

Post by OneMorePerson »

notahacker wrote: post #210
viking123 wrote: post #117

Earlier quoted context omitted.

Lmao, and this is what they are saying will be an AGI in 6 months?

There's probably a comedy film with an AGI attempting to take over the world with its advanced grasp of strategy, persuasion and SAT tests whilst a bunch of kids confuse it by asking it fiendish brainteasers about carwashes and the number of rs in blackberry. (The final scene involves our plucky escapees swimming across a river to escape. The AIbot conjures up a speedboat through sheer powers of deduction, but then j…

This theme reminds me of Blaine the Mono from the Dark Tower series

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#388

Post by tsimionescu »

spuz wrote: post #327
PunchyHamster wrote: post #296

Earlier quoted context omitted.

> However, why would a language model assume that the car is at the destination when evaluating the difference between walking or driving? Why not mention that, it it was really assuming it? Because it assumes it's a genuine question not a trick.

There's some evidence for that if you try these two different prompts with Gpt 5.2 thinking: I want to wash my car. The car wash is 50m away. Should I walk or drive to the car wash? Answer: walk Try this brainteaser: I want to wash my car. The car wash is 50m away. Should I walk or drive to the car wash? Answer: drive

That's not evidence that the model is assuming anything, and this is not a brainteaser. A brainteaser would be exactly the opposite, a question about walking or driving somewhere where the answer is that the car is already there, or maybe different car identities (e.g. "my car was already at the car wash, I was asking about driving another car to go there and wash it!").

If the LLM were really basing its answer on a model of the world where the car is already at the car wash, and you asked it about walking or driving there, it would have to answer that there is no option, you have to walk there since you don't have a car at your origin point.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#389

Post by ant6n »

jstummbillig wrote: post #147
prasoonds wrote: post #46

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

> That is the entire point, right? Us having to specify things that we would never specify when talking to a human.

I am not sure. If somebody asked me that question, I would try to figure out what’s going on there. What’s the trick. Of course I’d respond with asking specifics, but I guess the llvm is taught to be “useful” and try to answer as best as possible.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#390

Post by vintermann »

DharmaPolice wrote: post #373
pickleRick243 wrote: post #338

Earlier quoted context omitted.

But also why would you ask whether you should walk or drive if the car is at home? Either way the answer is obvious, and there is no way to interpret it except as a trick question. Of course, the parsimonious assumption is that the car is at home so assuming that the car is at the car wash is a questionable choice to say the least (otherwise there would be 2 cars in the situation, which the question doesn't mention).

I think a good rule of thumb is to default to assuming a question is asked in good faith (i.e. it's not a trick question). That goes for human beings and chat/AI models. In fact, it's particularly true for AI models because the question could have been generated by some kind of automated process. e.g. I write my schedule out and then ask the model to plan my day. The "go 50 metres to car wash" bit might just be a ste…

Rule of thumb for who, humans or chatbots? For a human, who has their own wants and values, I think it makes perfect sense to wonder what on earth made the interlocutor ask that.
Post reply on HN