Remember: models don't think.
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
631–640 of 998 posts
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#632Similar "broken" common-sense reasoning also occurs in medical edge-case reasoning ( https://www.nature.com/articles/s41598-025-22940-0 ); e.g. LLMs (o1) gets the following type of question wrong: A 4-year-old boy born without a left arm, who had a right arm below elbow amputation one month ago, presents to your ED with broken legs after a motor vehicle accident. His blood pressure from his right arm is 55/30, and wa…
I'm not a doctor, but am amazed that we've apparently reached the situation where we need to use these kinds of complex edge cases in order to hit the limit of the AI's capability; and this is with o1, released over a year ago, essentially 3 generations behind the current state of the art. Sorry for gushing, but I'm amazed that the AI got so far just from "book learning", without never stepping into a hospital, or ev…
These failure modes are not AI’s edge cases at the limit of its capabilities. Rather they demonstrate a certain category of issues with generalization (and “common sense”) as evidenced by the models’ failure upon slight irrelevant changes in the input. In fact this is nothing new, and has been one of LLMs fundamental characteristics since their inception.
As for your suggestion on learning from simulations, it sounds interesting, indeed, for expanding both pre and post training but still that wouldn’t address this problem, only hides the shortcomings better.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#633Earlier quoted context omitted.
A structured language without ambiguity is not, in general, how people think or express themselves. In order for a model to be good at interfacing with humans, it needs to adapt to our quirks. Convincing all of human history and psychology to reorganize itself in order to better service ai cannot possibly be a real solution. Unfortunately, the solution is likely going to be further interconnectivity, so the model can…
>Convincing all of human history and psychology to reorganize itself in order to better service ai cannot possibly be a real solution. I think there's a substantial subset of tech companies and honestly tech people who disagree. Not openly, but in the sense of 'the purpose of a system is what it does'.
I agree but it feels like a type-of-mind thing. Some people gravitate toward clean determinism but others toward chaotic and messy. The former requires meticulous linear thinking and the latter uses the brain’s Bayesian inference.
Writing code is very much “you get what you write” but AI is like “maintain a probabilistic mental model of the possible output”. My brain honestly prefers the latter (in general) but I feel a lot of engineers I’ve met seem to stray towards clean determinism.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#634This is the voice model, which doesn’t have any «thinking» or «reasoning» phase. It’s a useful model for questions that aren’t intended to trick the model. I’ve used it for live translation with great success. It tends to start ignoring the original instructions after 20 min, so you have to start a new conversation if you don’t want it to meddle in the conversation instead of just transferring.
LLM's don't 'think' nor do they 'reason'.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#635I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…
> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…
> we can assume similar issues arise in more complex cases
I would assume similar issues are more rare in longer, more complex prompts.
This prompt is ambiguous about the position of the car because it's so short. If it were longer and more complex, there could be more signals about the position of the car and what you're trying to do.
I must confess the prompt confuses me too, because it's obvious you take the car to the car wash, so why are you even asking?
Maybe the dirty car is already at the car wash but you aren't for some reason, and you're asking if you should drive another car there?
If the prompt was longer with more detail, I could infer what you're really trying to do, why you're even asking, and give a better answer.
I find LLMs generally do better on real-world problems if I prompt with multiple paragraphs instead of an ambiguous sentence fragment.
LLMs can help build the prompt before answering it.
And my mind works the same way.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#636Companies are making decisions based on these things. It's mind-boggling.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#637IMO this is a great example of how we're often asking loaded questions without realizing it. IMO it's the same when we're asking: "Should I implement X from scratch, or import a small library to do it?" vs "Should I add feature X to the codebase or bring in another dependency for it?" In the first question, it sounds like a good idea to not reinvent the wheel, in the second it sounds bad to have dependency.
I totally agree! Interacting with LLMs at work for the past 8 months has really shaped how I communicate with them (and people! in a weird way).
The solution I've found for "un-loading" questions is similar to the one that works for people: build out more context where it's missing. Wax about specifically where the feature will sit and how it'll work, force it to enumerate and research specific libraries and put these explorations into distinct documents. Synthesize and analyze those documents. Fill in any still-extant knowledge gaps. Only then make a judgement call.
As human engineers, we all had to do this at some point in our careers (building up context, memory, points of reference and experience) so we can now mostly rely on instinct. The models don't have the same kind of advantage, so you have to help them simulate that growth in a single context window.
Their snap/low-context judgements are really variable, generalizing, and often poor. But their "concretely-informed" (even when that concrete information is obtained by prompting) judgements are actually impressively-solid. Sometimes I'll ask an inversely-loaded question after loading up all the concrete evidence just to pressure-test their reasoning, and it will usually push back and defend the "right" solution, which is pretty impressive!
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#638Earlier quoted context omitted.
Developers are - on average - terrible at this. If they weren't, TPMs, Product Managers, CTOs, none of them would need to exist. It's not specific to software, it's the entire World of business. Most knowledge work is translation from one domain/perspective to another. Not even knowledge work, actually. I've been reading some works by Adler[0] recently, and he makes a strong case for "meaning" only having a sense to…
> Developers are - on average - terrible at this. If they weren't, TPMs, Product Managers, CTOs, none of them would need to exist. This is not really true, in fact products become worse the farther away from the problem a developer is kept. Best products I worked with and on (early in my career, before getting digested by big tech) had developers working closely with the users of the software. The worst were things l…
Yet IDEs are some of the worst things in the world. From EMacs to Eclipse to XCode, they are almost all bad - yet they are written by devs for devs.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#639Similar "broken" common-sense reasoning also occurs in medical edge-case reasoning ( https://www.nature.com/articles/s41598-025-22940-0 ); e.g. LLMs (o1) gets the following type of question wrong: A 4-year-old boy born without a left arm, who had a right arm below elbow amputation one month ago, presents to your ED with broken legs after a motor vehicle accident. His blood pressure from his right arm is 55/30, and wa…
I put this into Grok and it got the right answer on quick mode. I did not give multiple choice though. The real solution is to have 4 AI answer and let the human decide. If all 4 say the same thing, easy. If there is disagreement, further analysis is needed.
The issue with "adversarial" questions like the blood pressure one (which is open-sourced and published 1 year ago) is that they are eventually are ingested into model training data.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#640Earlier quoted context omitted.
You are right about the motivation behind the glee but it actually has a kernel of truth in it: With making such elementary mistakes, this thing isn't going to be autonomous anytime soon. Such elementary mistakes can be made by humans under influence of a substance or with some mental issues. It's pretty much the kind of people you wouldn't trust with a vehicle or anything important. IMHO all entry level clerical job…
I think the 'elementary mistakes' in humans are far more common than confined to the mentally ill or intoxicated. There are entire shows/YT channels dedicated to grabbing a random person on the street and asking them a series of simple questions. Often, these questions are pure-fact (who is the current US Vice President), but for some, the idea is that a young child can answer the questions better than an 'average' a…
The LLM is probably also not going to launch into a rant about how they incorporate religious and racial beliefs into their life when asked about current heads of state. You ask the LLM about a solar configuration, and I think it must be exceptionally rare to have it instead tell you about its feelings on politics.
We had a big winter storm a few weeks ago, right when I received a large solar panel to review. I sent my grandpa a picture of the solar panel on its ground mount, covered in snow, noting I just got it today and it wasn't working well (he's very MAGA-y, so I figured the joke would land well). I received a straight-faced reply on how PV panels work, noting they require direct sunlight and that direct sunlight through heavy snow doesn't count; they don't tell you this when they sell these things, he says. I decided to chalk this up to being out-deadpanned and did not reply "thanks, ChatGPT."