Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Car Wash” test with 53 models

opper.ai

Post reply on HN
Print view View original

421–430 of 469 posts

Re: “Car Wash” test with 53 models

#421

Post by CamperBob2 »

rerdavies wrote: post #309
Night_Thastus wrote: post #27

Earlier quoted context omitted.

I agree. If the LLM were truly an intelligence, it would be able to ask about this nonsense question. It would be able to ask "Why is walking even an option? Can you please explain how you imagine that would work? Do you mean hand-washing the car at home, instead?" (etc, etc) Real people can ask for clarification when things are ambiguous or confusing. Once something is clarified, they can work that into their unders…

And the corollary: if LLMs were truly intelligent, they would also be able to respond to such questions sarcastically.

Which Gemini does...?

Re: “Car Wash” test with 53 models

#422

Post by BobbyJo »

bigbuppo wrote: post #410
underlines wrote: post #394

I maintain a private evaluation set of what many call "misguided attention" questions. In many of these cases, the issue isnt failed logical reasoning. Its ambiguity, underspecified context, or missing constraints that allow multiple valid interpretations. Models often fail not because they can’t reason, but because the prompt leaves semantic gaps that humans silently fill with shared assumptions. A lot of viral "fro…

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

Crazy how five years ago this level of AI would be seen as scifi, and now there are people out there who think it's trash because we can trick it if we ask questions in weird ways.

Re: “Car Wash” test with 53 models

#423

Post by steveBK123 »

bigbuppo wrote: post #410
underlines wrote: post #394

I maintain a private evaluation set of what many call "misguided attention" questions. In many of these cases, the issue isnt failed logical reasoning. Its ambiguity, underspecified context, or missing constraints that allow multiple valid interpretations. Models often fail not because they can’t reason, but because the prompt leaves semantic gaps that humans silently fill with shared assumptions. A lot of viral "fro…

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

An issue in the chat format is that all these models seem bad at recognizing when they have extraneous information from user that can be ignored, or insufficient information from the user to answer the question fully.

This issue is compounded by the lack of probabilities in the answers, despite the machines ultimately being probabilistic.

Notice a human in a real conversation will politely ignore extra info (the distance to car wash) or ask clarifying questions (where is the car?).

Even non-STEM people answer using probabilistic terms casually (almost certainly / most likely / probably / possibly / unlikely).

I suspect some of this is to minimize token usage in the fixed monthly price chat models, because back&forth would cost more tokens.. but maybe I'm too cynical.

Re: “Car Wash” test with 53 models

#424

Post by bigbuppo »

BobbyJo wrote: post #422
bigbuppo wrote: post #410

Earlier quoted context omitted.

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

Crazy how five years ago this level of AI would be seen as scifi, and now there are people out there who think it's trash because we can trick it if we ask questions in weird ways.

When did Microsoft release that chat bot that went full nazi in a couple of hours?

Re: “Car Wash” test with 53 models

#425

Post by steveBK123 »

bigbuppo wrote: post #424
BobbyJo wrote: post #422

Earlier quoted context omitted.

Crazy how five years ago this level of AI would be seen as scifi, and now there are people out there who think it's trash because we can trick it if we ask questions in weird ways.

When did Microsoft release that chat bot that went full nazi in a couple of hours?

2016 for those keeping score

Re: “Car Wash” test with 53 models

#426

Post by bigbuppo »

steveBK123 wrote: post #423
bigbuppo wrote: post #410

Earlier quoted context omitted.

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

An issue in the chat format is that all these models seem bad at recognizing when they have extraneous information from user that can be ignored, or insufficient information from the user to answer the question fully. This issue is compounded by the lack of probabilities in the answers, despite the machines ultimately being probabilistic. Notice a human in a real conversation will politely ignore extra info (the dist…

The systems recognized the pattern that it looks like a generic article on the internet asking whether someone should walk or drive and answered it exactly as expected based on their training data. None of this should be surprising.

We are the ones fooling ourselves into believing there's more intelligence in these systems than they really have. At the end of the day, it's just an impressive parlor trick.

Re: “Car Wash” test with 53 models

#427

Post by felix089 »

coldtea wrote: post #332

> so I got a human baseline through Rapidata (10k people, same forced choice): 71.5% said drive. What kind of idiot would say "walk"? I guess if they were average and above they wouldn't gig at Rapidata. Either that or foreigners with mediocre understanding of the English language. Or some bored teenagers that get their $0.20 or whatever Rapidata pays whatever they click (was it a multiple choice) - the (studied and…

Yea, I thought the same before the test and was pretty surprised. But RE the data, it's actually not a gig platform where people get paid. Rapidata answered this in another comment below. They integrate micro-surveys into mobile apps (like Duolingo, games, etc) as an optional opt-in instead of watching ads. The users are vetted and there's no incentive to answer correctly.

Re: “Car Wash” test with 53 models

#428

Post by GitPushOrigin »

w4der wrote: post #416
keeganpoppen wrote: post #398

Earlier quoted context omitted.

that's what the cultivators of these examples are preying on. but in practice what people care about is "can i get it to do ", not "is it a decider on every possible token sequence that humans perceive to be about ".

But what is being pitched as "AGI" hype is the latter.

Fair, but that's just what hype is. Overpromise, underdeliver. Most of us recognize its limits and take advantage of its strengths. This post (and many in it) seem to be inferring that AI is useless because it isn't AGI, answered a simple question wrong, was tricked, or didn't answer perfectly. This is cherry-picking at best, disingenuous at worst.

Re: “Car Wash” test with 53 models

#429

Post by keeda »

I'm imposing but could you try these runs again with this small change: Simply append “Make sure to check your assumptions.” to the question.

Note, it does not mention what assumption specifically. In my experiments, after the models got it wrong the first time (i.e. they weren't "patched" yet) adding that simple caveat fixed it for all of them except the older Llama models.

This is not the first time I've observed this; I found the same when the Apple "red herrings" study came out.

If these gotcha questions can be trivially overcome by a simple caveat in the prompt, I suspect the only reason AI providers don't include it in the system prompt by default is as a cost optimization, as I postulated in a previous comment: https://news.ycombinator.com/item?id=47040530

Re: “Car Wash” test with 53 models

#430

Post by keeda »

bigbuppo wrote: post #410
underlines wrote: post #394

I maintain a private evaluation set of what many call "misguided attention" questions. In many of these cases, the issue isnt failed logical reasoning. Its ambiguity, underspecified context, or missing constraints that allow multiple valid interpretations. Models often fail not because they can’t reason, but because the prompt leaves semantic gaps that humans silently fill with shared assumptions. A lot of viral "fro…

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

> They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect.

Not necessarily. Simply asking models to "check your assumptions" -- note, without specifying what assumptions! -- overcomes a lot of these gotcha questions. The reason it's not in their system prompts by default is I think just a cost optimization: https://news.ycombinator.com/item?id=47040530

Post reply on HN