Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Car Wash” test with 53 models

opper.ai

Post reply on HN
Print view View original

431–440 of 469 posts

Re: “Car Wash” test with 53 models

#431

Post by keeda »

ForHackernews wrote: post #397
underlines wrote: post #394

I maintain a private evaluation set of what many call "misguided attention" questions. In many of these cases, the issue isnt failed logical reasoning. Its ambiguity, underspecified context, or missing constraints that allow multiple valid interpretations. Models often fail not because they can’t reason, but because the prompt leaves semantic gaps that humans silently fill with shared assumptions. A lot of viral "fro…

Some might argue "sensitivity to framing and distributional priors" is a fancy way to say "absence of reasoning capability".

Actually, no those are separate things, and you can see for yourself with this simple experiment. Just adding "Check your assumptions" to the prompt -- note, without specifying what assumptions! -- leads them to reason to the right answer.

You can try it with the free version of ChatGPT yourself (remember to ask the original question in a separate session to verify it hasn't been "patched" yet.)

Re: “Car Wash” test with 53 models

#432

Post by joquarky »

keeda wrote: post #429

I'm imposing but could you try these runs again with this small change: Simply append “Make sure to check your assumptions.” to the question. Note, it does not mention what assumption specifically. In my experiments, after the models got it wrong the first time (i.e. they weren't "patched" yet) adding that simple caveat fixed it for all of them except the older Llama models. This is not the first time I've observed t…

In my experience, asking "what did we forget?" after Claude/Codex finishes a task usually results in a few extra tweaks that are beneficial.

Re: “Car Wash” test with 53 models

#433

Post by davorak »

BobbyJo wrote: post #422
bigbuppo wrote: post #410

Earlier quoted context omitted.

Well, that means the AI is garbage. They'll eventually train it to answer this specific question, and then it will perform worse in some other aspect. Wash, rinse, repeat, and eventually they'll claim the new frontier model is the best yet on carwash tests.

Crazy how five years ago this level of AI would be seen as scifi, and now there are people out there who think it's trash because we can trick it if we ask questions in weird ways.

I think the level of ai we have is amazing.

> there are people out there who think it's trash because we can trick it if we ask questions in weird ways.

Some of this sentiment comes form wanting AI to be predictable and for me stumbling into questions that the current models interpret oddly is not uncommon. There are a bunch of rules of thumbs that can be used to help when you run into a cases like this but no guarantee that they will work, or that the problem will remain solved after a model update, or across models.

Re: “Car Wash” test with 53 models

#434

Post by oenton »

bityard wrote: post #366

It's not hard to come up with questions designed to fool or puzzle the listener. We call them riddles. The fact that it fools some percentage of LLMs (and people) should not be surprising. What is surprising (to me) is how this continues to be a meme. ("I tried to trick an LLM and I did" is not exactly a noteworthy achievement at this stage in AI technology.)

> "I tried to trick an LLM and I did" is not exactly a noteworthy achievement at this stage in AI technology.

I agree it’s not surprising and I would also agree it’s not noteworthy, if the CEO of OpenAI wasn’t still making public statements like this:

People talk about how much energy it takes to train an AI model … But it also takes a lot of energy to train a human. It takes like 20 years of life and all of the food you eat during that time before you *get smart*.

Re: “Car Wash” test with 53 models

#435

Post by steveBK123 »

bigbuppo wrote: post #426
steveBK123 wrote: post #423

Earlier quoted context omitted.

An issue in the chat format is that all these models seem bad at recognizing when they have extraneous information from user that can be ignored, or insufficient information from the user to answer the question fully. This issue is compounded by the lack of probabilities in the answers, despite the machines ultimately being probabilistic. Notice a human in a real conversation will politely ignore extra info (the dist…

The systems recognized the pattern that it looks like a generic article on the internet asking whether someone should walk or drive and answered it exactly as expected based on their training data. None of this should be surprising. We are the ones fooling ourselves into believing there's more intelligence in these systems than they really have. At the end of the day, it's just an impressive parlor trick.

In that sense the google AI summary search results are a better UX for this type experience

Re: “Car Wash” test with 53 models

#436

Post by bigbuppo »

davorak wrote: post #433
BobbyJo wrote: post #422

Earlier quoted context omitted.

Crazy how five years ago this level of AI would be seen as scifi, and now there are people out there who think it's trash because we can trick it if we ask questions in weird ways.

I think the level of ai we have is amazing. > there are people out there who think it's trash because we can trick it if we ask questions in weird ways. Some of this sentiment comes form wanting AI to be predictable and for me stumbling into questions that the current models interpret oddly is not uncommon. There are a bunch of rules of thumbs that can be used to help when you run into a cases like this but no guaran…

There are a lot of rules of thumb you can follow to avoid getting bitten by a rattlesnake, but the easiest way is to just not pick up random snakes. I don't know where I'm going with this, but I am going for a walk.

Re: “Car Wash” test with 53 models

#437

Post by sparky_z »

malfist wrote: post #399
Maxion wrote: post #388

Earlier quoted context omitted.

You might own multiple cars, you might be borrowing someone elses and so forth.

That still doesn't make sense. I'm going to use another car, or borrow a car to drive to a carwash where my car I want to wash is and then....I guess leave it there? Or leave the car I came in? This isn't a viable out for explaining why AI can't "reason" through this.

But why would they reason through it in that way? You haven't asked them to listen carefully and find the secret reason you're a dumb-ass in order to prove how smart they are. If they default to that mode on every query, that would just make them insufferable conversational partners, which is not the training goal.

Let me put it this way. If you were to prefix the prompts they used with "This is an IQ test: ", I wouldn't be surprised if most of the the models did much better. That would give them the context that the humans reading this article already have.

Re: “Car Wash” test with 53 models

#438

Post by andai »

XCSme wrote: post #351
andai wrote: post #347

Earlier quoted context omitted.

In your benchmark, GPT 5 Nano is basically tied with Opus?

Yes. Opus could do a lot better, but fails a lot because it doesn't respect the given formatting instructions/output format. I could modify the tests to emphasize the requirements, but then, what's the point of a test. In real life, we expect the AI to do something if we ask it, especially for agentic use-case or in n8n, because if the output is slightly wrong, the entire workflow fails.

Interesting. This has to do with the "instruction following" aspect, right? I saw that GPT models do a lot higher than Claude on those benchmarks.

I haven't done my own tests, but I did notice a lot of models are very low there. You'll give them specific instructions and they'll ignore them and just pattern match to whatever was the format they saw most commonly during training.

Re: “Car Wash” test with 53 models

#439

Post by XCSme »

andai wrote: post #438
XCSme wrote: post #351

Earlier quoted context omitted.

Yes. Opus could do a lot better, but fails a lot because it doesn't respect the given formatting instructions/output format. I could modify the tests to emphasize the requirements, but then, what's the point of a test. In real life, we expect the AI to do something if we ask it, especially for agentic use-case or in n8n, because if the output is slightly wrong, the entire workflow fails.

Interesting. This has to do with the "instruction following" aspect, right? I saw that GPT models do a lot higher than Claude on those benchmarks. I haven't done my own tests, but I did notice a lot of models are very low there. You'll give them specific instructions and they'll ignore them and just pattern match to whatever was the format they saw most commonly during training.

Yup, for example I tell Claude to return ONLY the answer as "LEFT" or "RIGHT".

And it outputs:

**RIGHT**

With markdown bold formatting... This is probably fine in a chat app, but when you use this in a workflow, it will break the workflow if you then have an if check like if(response === 'RIGHT')...

Re: “Car Wash” test with 53 models

#440

Post by jiggawatts »

1718627440 wrote: post #334
jiggawatts wrote: post #266

Earlier quoted context omitted.

This reminds me of a Uni exam that was soooo broken that answering “correctly” entailed guessing how exactly the professor designing the questions misunderstood the topic of his own lectures.

Are you not allowed to ask the professor questions? We are, and it is not to seldom that the professor then walks to the black board and updates the question.

It was an examination with 300 students in a giant hall, overseen by university staff, not the individual professors.

So many people complained that they did eventually fetch him to come and clarify (correct) the questions.

I didn’t have the patience to wait for him to turn up, so I simply provided a matrix of solutions for every possible combination of potential original intent… with note next to it saying that anything other than a 100% mark will be met with official complaints about his lack of due diligence.

Post reply on HN