Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

mastodon.world

Post reply on HN
Print view View original

741–750 of 998 posts

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#741

Post by sensanaty »

dirkc wrote: post #512
jstummbillig wrote: post #147

Earlier quoted context omitted.

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

> That is the entire point, right? Us having to specify things that we would never specify when talking to a human. Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn. But this will only happen after the last programmer has died and no-one will remember pro…

> Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn.

We've truly gone full circle here, except now our programming languages have a random chance for an operator to do the opposite of what the operator does at all other times!

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#742

Post by gs17 »

benterix wrote: post #493
rahidz wrote: post #462

Earlier quoted context omitted.

For GPT at least, a lot of it is because "DO NOT ASK A CLARIFYING QUESTION OR ASK FOR CONFIRMATION" is in the system prompt. Twice. https://github.com/Wyattwalls/system_prompts/blob/main/OpenA...

Out of curiosity: when you add custom instructions client-side, does it change this behavior?

It changes some behavior, but there's some things that are frustratingly difficult to override. The GPT-5 version of ChatGPT really likes to add a bunch of suggestions for next steps at the end of every message (e.g. "if you'd like, I can recommend distances where it would be better to walk to the car wash and ones where it would be better to drive, let me know what kind of car you have and how far you're comfortable walking") and really loves bringing up resolved topics repeatedly (e.g. if you followed up the car wash question with a gas station question, every message will talk about the car wash again, often confusing the topics). Custom instructions haven't been able to correct these so far for me.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#743

Post by jodrellblank »

rahidz wrote: post #462
nicbou wrote: post #321

Earlier quoted context omitted.

I get that issue constantly. I somehow can't get any LLM to ask me clarifying questions before spitting out a wall of text with incorrect assumptions. I find it particularly frustrating.

For GPT at least, a lot of it is because "DO NOT ASK A CLARIFYING QUESTION OR ASK FOR CONFIRMATION" is in the system prompt. Twice. https://github.com/Wyattwalls/system_prompts/blob/main/OpenA...

Are these actual (leaked?) system prompts, or are they just "I asked it what its system prompt is and here's the stuff it made up:" ?

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#744

Post by hansmayer »

andy12_ wrote: post #722
hansmayer wrote: post #574

Earlier quoted context omitted.

> So saying "it predicts the next word" is a nothing-burger. That a program calculates its output one token at a time tells you nothing about its behavior. Well it does - it tells me it is utterly un-reliable, because it does not understand anything. It just merely goes on, shitting out a nice pile of tokens that placed one after another kind of look like coherent sentences but make no sense, like "you should absolut…

No, you can't know that the output of a program is unreliable just from the fact that it outputs one words at a time. I already told you that you can perfectly compile a normal program, like a calculator, into the weights of an autoregressive transformer (this comes from works like RASP, ALTA, tracr, etc). And with this I don't mean it in the sense of "approximating the output of a calculator with 99.999% accuracy",…

> No, you can't know that the output of a program is unreliable just from the fact that it outputs one words at a time

Yes I can, and it shows everytime the "smart" LLMs suggest us to take a walk to the carwash or suggests 1.9 < 1.11 etc...

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#745

Post by kshacker »

visarga wrote: post #737

Never ask an important question just once. Ask it in many ways, and on multiple models. If they don't agree at least you know you can't rely on these answers. For important questions I run 3-4 Deep Research reports (Claude, ChatGPT, Gemini, Perplexity) and then comparative analysis at the end.

Is that how you talk to friends too?

Somewhere the goalpost changed from natural language to prompt engineering and failsafes and audit trails.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#746

Post by marc_g »

So I'm not sure if anyone has tried this in the over 700 comments here, so apologies if it's been double-posted, but the rationale from ChatGPT almost makes me understand where it's coming from when you ask it to create an image of what it's thinking.

Here's the image: https://imgur.com/a/kQmo0jY

Here's the chat: https://chatgpt.com/share/69935336-6438-8002-995d-f26989d59a...

Still not really sure why you would need to get the water from the carwash next door, but maybe the soap quality is better?

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#747

Post by oatmeal1 »

prasoonds wrote: post #46

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…

If the model assumed the car to be cleaned was already at the car wash, it should identify the reason to walk is that if you drive, the extra car you take to get to the car wash will now be stuck there requiring redundant walking.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#748

Post by mikkupikku »

steveBK123 wrote: post #543
Gabrys1 wrote: post #524

Earlier quoted context omitted.

I would say, the proper response to this question is not "walk, blablablah" but rather "What do you mean? You need to drive your car to have it washed. Did I miss anything?"

Yes, this is what irks me about all the chatbots, and the chat interface as a whole. It is a chat-like UX without a chat-like experience. Like you are talking to a loquacious autist about their favorite topic every time. Just ask me a clarifying question before going into your huge pitch. Chats are a back & forth. You don’t need to give me a response 10x longer than my initial question. Etc

> Like you are talking to a loquacious autist about their favorite topic every time

That's the best part.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#750

Post by gorjusborg »

dirkc wrote: post #512
jstummbillig wrote: post #147

Earlier quoted context omitted.

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

> That is the entire point, right? Us having to specify things that we would never specify when talking to a human. Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn. But this will only happen after the last programmer has died and no-one will remember pro…

> But this will only happen after the last programmer has died and no-one will remember programming languages, compilers, etc.

If we're 'lucky' there will still be some 'priests' around like in the Foundation novels. They don't understand how anything works either, but can keep things running by following the required rituals.

Post reply on HN