He left out Grok, who actually replies correctly :thinking:
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
241–250 of 998 posts
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#242Earlier quoted context omitted.
Opus 4.6: Walk! At 50 meters, you'll get there in under a minute on foot. Driving such a short distance wastes fuel, and you'd spend more time starting the car and parking than actually traveling. Plus, you'll need to be at the car wash anyway to pick up your car once it's done.
Lmao, and this is what they are saying will be an AGI in 6 months?
Yes, get ready to lose your job and cash your UBI check! It's over.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#243LLM failures go viral because they trigger a "Schadenfreude" response to automation anxiety. If the oracle can't do basic logic, our jobs feel safe for another quarter. Wrong.
I agree that this is more of a social media effect than an LLM effect. But I'll add that this failure mode is very repeatable, which is a condition for its virality. A lot of people can reproduce the failure, even if it isn't 100% reproducible, even better for virality, if 50% can reproduce it and 50% can't, it feeds off even more into the polarizing "white dress blue dress" effect.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#244This is the voice model, which doesn’t have any «thinking» or «reasoning» phase. It’s a useful model for questions that aren’t intended to trick the model. I’ve used it for live translation with great success. It tends to start ignoring the original instructions after 20 min, so you have to start a new conversation if you don’t want it to meddle in the conversation instead of just transferring.
The text-only model with reasoning (both of opus 4.6, gpt 5.2) can be tricked with this question. Note: you might have to try it multiple times as they are not deterministic. But I managed to get a failing result right away on both.
Also note, some model may decide to do a web search, in which case they just likely find this "bug".
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#245Earlier quoted context omitted.
That's not what I got. Opus 4.6 (not Extended Thinking): Drive. You'll need the car at the car wash.
Am I the only one who thinks these people are monkey patching embarrassments as they go? I remember the r in strawberry thing they suddenly were able to solve, while then failing on raspberry.
They should make Opus Extended Extended that routes it to actual person in a low cost country.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#246Earlier quoted context omitted.
> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…
The question is so outlandish that it is something that nobody would ever ask another human. But if someone did, then they'd reasonably expect to get a response consisting 100% of snark. But the specificity required for a machine to deliver an apt and snark-free answer is -- somehow -- even more outlandish? I'm not sure that I see it quite that way.
But the number of outlandish requests in business logic is countless.
Like... In most accounting things, once end-dated and confirmed, a record should cascade that end-date to children and should not be able to repeat the process... Unless you have some data-cleaning validation bypass. Then you can repeat the process as much as you like. And maybe not cascade to children.
There are more exceptions, than there are rules, the moment you get any international pipeline involved.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#247LLM failures go viral because they trigger a "Schadenfreude" response to automation anxiety. If the oracle can't do basic logic, our jobs feel safe for another quarter. Wrong.
I'd say it's moreso that it's a startlingly clear rebuttal to the tired refrain of, "Models today are nothing like they were X months ago!" When actually, yes, they still fucking blow.
So rather than patiently explain to yet another AI hypeman exactly how models are and aren't useful in any given workflow, and the types of subtle reasoning errors that lead to poor quality outputs misaligned with long-term value adds, only to invariably get blamed for user incompetence or told to wait Y more months, we can instead just point to this very concise example of AI incompetence to demonstrate our frustrations.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#248I challenged Gemini to answer this too, but also got the correct answer. What came to my mind was: couldn't all LLM vendors easily fund teams that only track these interesting edge cases and quickly deploy filters for these questions, selectively routing to more expensive models? Isn't that how they probably game benchmarks too?
Yes that's potentially why it's already fixed now in some models, since it's about a week after this actually went viral on r/localllama originally. I wouldn't be surprised if most vendors run some kind of swappable lora for quick fixes at this point. It's an endless whac-a-mole of edge cases that show that most LLMs generalize to a much lesser extent than what investors would like people to believe.
Like, this is not an architectural problem unlike the strawberry nonsense, it's some dumb kind of overfitting to a standard "walking is better" answer.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#249Out of all conceptual mistakes people make about LLMs, one that needs to die very fast is to assume that you can test what it "knows" by asking a question. This whole thread is people asking different models a question one time and reporting a particular answer, which is the mental model you would use for whether a person knows something or not.
The other funny thing is thinking that the answer the llm produces is wrong. It is not, it is entirely correct. The question: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? The question is non-sensical. If the reason you want to go to the car wash is to help your buddy Joe wash his car you SHOULD walk. Nothing in the question reveals the reason for why you want to go to the car wash,…
>I want to wash MY car
>you want to go to the car wash is to help your buddy Joe wash HIS car
nope, question is pretty clear, however I will grant that it's only a question that would come up when "testing" the AI rather than a question that might genuinely arise.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#250Out of all conceptual mistakes people make about LLMs, one that needs to die very fast is to assume that you can test what it "knows" by asking a question. This whole thread is people asking different models a question one time and reporting a particular answer, which is the mental model you would use for whether a person knows something or not.
The other funny thing is thinking that the answer the llm produces is wrong. It is not, it is entirely correct. The question: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? The question is non-sensical. If the reason you want to go to the car wash is to help your buddy Joe wash his car you SHOULD walk. Nothing in the question reveals the reason for why you want to go to the car wash,…
It explicitly says you want to wash your car in the statement. Maybe it's not just LLMs struggling with a fairly basic question...