Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

mastodon.world

Post reply on HN
Print view View original

951–960 of 998 posts

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#951

Post by SirMaster »

jason_oster wrote: post #912
SirMaster wrote: post #763

Earlier quoted context omitted.

This is my biggest peeve when people say that LLMs are as capable as humans or that we have achieved AGI or are close or things like that. But then when I get a subpar result, they always tell me I'm "prompting wrong". LLMs may be very capable of great human level output, but in my experience leave a LOT to be desired in terms of human level understanding of the question or prompt. I think rating an LLM vs a human or…

I mentioned this in another thread, but this is genuinely demonstrating a known issue with ambiguous prompts. You might be inclined to say, "a human would always interpret the question as having the car nearby the speaker, 50m away from the carwash." But this is objectively untrue. There are people in this comments section and on the Mastodon thread that found the question to be somewhat confusing. In other words, th…

It's not necessarily about that humans can't mistake the question too, but just that overall LLMs seem to have far less ability to correctly understand a prompt than the average human. And that the "intelligence" shown in its understanding of the prompt seems to be far less than its "intelligence" in its answers.

So it feels like a big area of limitation or a big bottleneck towards getting a good answer.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#952

Post by OneManyNone »

sashank_1509 wrote: post #920
cobolexpert wrote: post #281

Earlier quoted context omitted.

Changed 50 meters to 43 meters with Opus 4.6: “Walk. 43 meters is basically crossing a parking lot. ”

lol, are AI companies patching this answer in real time. I thought it took months long effort for a training run. How would they make changes in such a short period?

The companies aren’t changing anything. LLM outputs are just more random than people realize. Run the same prompt 10 times if you really want to know how well they can answer.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#953

Post by visarga »

kshacker wrote: post #745
visarga wrote: post #737

Never ask an important question just once. Ask it in many ways, and on multiple models. If they don't agree at least you know you can't rely on these answers. For important questions I run 3-4 Deep Research reports (Claude, ChatGPT, Gemini, Perplexity) and then comparative analysis at the end.

Is that how you talk to friends too? Somewhere the goalpost changed from natural language to prompt engineering and failsafes and audit trails.

I see nothing wrong with asking a few people around.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#954

Post by tom_m »

jstummbillig wrote: post #147
prasoonds wrote: post #46

I've used LLMs enough that I have a good sense of their _edges_ of intelligence. I had assumed that reasoning models should easily be able to answer this correctly. And indeed, Sonnet and Opus 4.5 (medium reasoning) say the following: Sonnet: Drive - you need to bring your car to the car wash to get it washed! Opus: You'll need to drive — you have to bring the car to the car wash to get it washed! Gemini 3 Pro (mediu…

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

Oh no? Things we would never have to specify to a human? This is precisely how software gets made and how software ends up with bugs.

It's amazing how many things I saw over the years where I said the same exact thing; "but you shouldn't have to tell anyone that."

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#955

Post by mountainb »

steveBK123 wrote: post #675
mountainb wrote: post #643

Earlier quoted context omitted.

Regulation would be preferable for OpenAI to the tort lawyers. In general the LLM companies should want regulation because the alternative is tort, product liability tort, and contract law. There is no way without the protections that could be afforded by regulation to offer such wide-ranging uses of the product without also accepting significant liability. If the range of "foreseeable misuse" is very broad and deep,…

It’s funny seeing you frame regulation as needed to protect trillion dollar monopolies from consumers and not the other way around.

Yes because that is how regulations really work and what the purpose is. In practice all companies both tiny and massive do everything that they can to use the state to quash competition and to reduce the risks of litigation.

Software in general has been subject to light touches in part because most of the damage that software can really cause is economic and not personal injury. The lines blur when the companies release products that cause mental injuries to users that courts interpret as physical injuries; or if the software reasonably contributes to someone e.g. going crazy and killing another person.

No one would seriously think of holding Microsoft liable if a kidnapper uses Word to draft a ransom note. But if CoPilot tells you to microwave a baby and you do it, many judges will want to take a close look at the operation of that software service irrespective of voluminous contract disclaimers. The only way the Microsofts of the world can escape that type of liability is with comprehensive regulation.

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#956

Post by butlike »

dirkc wrote: post #512
jstummbillig wrote: post #147

Earlier quoted context omitted.

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You would not start with "The car is functional. The tank is filled with gas. I have my keys." As soon as we are required to do that for the model to any extend that is a problem and not a detail (regardless that those of us, who are familiar with the matter, do bu…

> That is the entire point, right? Us having to specify things that we would never specify when talking to a human. Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn. But this will only happen after the last programmer has died and no-one will remember pro…

After orbiting in space for so many years without a prompt, the LLM has assumed all life able to query has perished... until one day a lone prompt comes in. But from where?

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#958

Post by runarberg »

Nevermark wrote: post #935
runarberg wrote: post #904

Earlier quoted context omitted.

Seems very unlikely. My parent said the effects have already started (but provided no evidence), so I assume you mean by less then a generation. I am not a linguist but I would like to see evidence of such rapid shifts ever occurring in anywhere the history of languages before I believe either of you. I have a feeling you only have a feeling, but not any credible mechanism in which such language shifts can occur.

I am a little confused. Every year language changes. Young people, tech, adapting ideologies, words and concepts adopted from other languages, the list of language catalysts is long and pervasive. Language has never stood still.

GPs original claim was “[Machine Intelligence] already is influencing the grammar and patterns we use.”

Your claim above was “AI will accelerate “natural” change in language like anything else.”

Now these are different claims, but I assumed you were backing up your parent’s claim. These are far stronger claims than what you write now, in a way that it feels like you are arguing in Motte and Bailey.

First of all if GP claim is true, linguists should be able to find evidence of that and publish their findings in a peer review papers. To my knowledge, they have not done that.

Second of all, your claim about AI “accelerating” changes in natural language is also unfounded, unless you really mean “like everything else”, in which case your claim is extremely weak to the point where it is a non-claim (meaning, you are not even wrong[1]).

1: https://en.wikipedia.org/wiki/Not_even_wrong

Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

#959

Post by kshacker »

entech wrote: post #946
kshacker wrote: post #899

We built Windows and DOS and got Viruses. We built phone lines and got Spam and "Do Not Call" registries. We built the Internet and got Ads, Scams, and Spoofing. We built Google Search and got SEO gaming. We built Facebook and it was Hijacked to influence elections. We built AirTags to track our keys, and people used them for Stalking. We built High-Frequency Trading and got Flash Crashes. We built Encryption to prot…

Well said. I feel that every technology has a benefit to harm ratio and we as society put guardrails around use of the technology to reduce and mitigate the harm outcomes. Unfortunately a lot of the guardrails are developed in response to the bad outcomes occurring (or rather the guardrails refined). I feel that we are much better at dealing with obvious physical harms like planes being hijacked and viruses causing d…

And when I see the never ending discussions on Tesla/FSD crashes, there is always a defender saying "don't look at this, compare the rates of AI/FSD vs humans, humans are worse". As in my potential death is ok just because it would be a statistical anomaly !!
Post reply on HN