[dead]
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
991–998 of 998 posts
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#992Earlier quoted context omitted.
> This is literally the Gell-Mann Amnesia Effect in action. Absolutely! But there is some nuance, here. The failure mode is for an ambiguous question, which is an open research topic. There is no objectively correct answer to "Should I walk or drive?" given the provided constraints. Because handling ambiguities is a problem that researchers are actively working on, I have confidence that models will improve on these…
Producing fewer "Compiler errors" and more "broken code errors" is a fundamental failure. The cost of detecting compiler errors is lower than detecting broken code. If the cost of detecting and fixing broken code increases at the same rate as LLMs "improve" then their net benefit will remain fixed. I asked my five year old the above "brain teaser" and he got it right. I did a follow up of what should he wash at a car…
I agree it is a fundamental failure of the current state of models. I believe it is solvable. The nuance is just that "solving" the problem might not look like what we think of as a solution. Hence the asymptote.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#993Earlier quoted context omitted.
I mentioned this in another thread, but this is genuinely demonstrating a known issue with ambiguous prompts. You might be inclined to say, "a human would always interpret the question as having the car nearby the speaker, 50m away from the carwash." But this is objectively untrue. There are people in this comments section and on the Mastodon thread that found the question to be somewhat confusing. In other words, th…
It's not necessarily about that humans can't mistake the question too, but just that overall LLMs seem to have far less ability to correctly understand a prompt than the average human. And that the "intelligence" shown in its understanding of the prompt seems to be far less than its "intelligence" in its answers. So it feels like a big area of limitation or a big bottleneck towards getting a good answer.
I think we're also miscommunicating, so this isn't really a surprise.
It is not clear to me why you've quoted "intelligence" or why you separated understanding and answers as having independent intelligences.
But yes, I agree there are limitations. Much like those above that are being researched.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#994They've kind of patched this for direct questions, but distractions can still confuse it into nonsense.
("drive it over when it's ready to be picked up"??!)
e.g.
Welcome to Opus 4.6
Dude, should I do a walk to or drive on up to the carwash - it's only a block from my house, yoyoyo and my car's yellow (no poodle anymore tho)
● If it's only a block away, walk. You're going there to get the car washed, so drive it over when it's ready to be picked up, or just drive it one block -- either way works. But driving one block to wait around seems unnecessary when you could walk over, get things set up, and then go grab the car.
That said, the practical answer: drive it. The car needs to be there to get washed. Just drive the one block.Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#995Earlier quoted context omitted.
Because, this fundamental premise demonstrates that LLMs can't really think logically like we do and they are far from replacing actual humans, let alone senior software engineers.
So if you took one of the greatest software engineers ever and made it so he was unable to answer this nonsensical and pointless question, he would be a lesser engineer because of it?
The fundamental of software engineering is built on basic logic. If an engineer made a mistake in a basic logical problem, yes, that does make them less of a software engineer. It's the difference between memorization and actual problem solving. What's funny is I'm actually pro-AI when it comes to software engineering, but I'm of the belief that AI will and should complement humans, atleast with the current state of models, not actually try to replace them like all the vibe coders love to exaggerate. In fact, I'm building an AI tool around the whole concept - with humans in the loop, not outside of it.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#996My favorite trick question so far is: You are in a room with three switches and three lightbulbs. Each switch turns on one lightbulb. How to determine which switch turns on which lightbulb? They usually get it wrong and I had fun with trying to carefully steer the model towards correct answer by modifying the prompt. Gemni 3 on Fast right now gives the funniest reaction. It starts with the answer to classic puzzle (n…
Nice! Gemini 3.1 Pro Preview gets it correctly and gives the correct solution at the beginning of the answer, but also considers possibility that I might have the classic riddle in mind and gives solution to that in the second part of the answer.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#997I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…
What is the version used by the free chatgpt now? ( https://chatgpt.com/ ) > Since the car wash is only 50 meters away (about 55 yards), you should walk. > Here’s why: > - It’ll take less than a minute. > - No fuel wasted. > - Better for the environment. > - You avoid the irony of driving your dirty car 50 meters just to wash it. the last bullet point is amusing, it understands you intend to wash the car you drive bu…
As long as there is a forum as technical as this where LLM performance commentary uses the word "it understands" irony is still alive.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#998Earlier quoted context omitted.
Absolutely. There is a preflight guardrail that steers specific words, phrases, concepts with tweaked output.
I've speculated about this myself, but haven't heard anyone actually discuss it or reveal/leak this is the case. Do you have a source for this?
I implement them.