I don't think this is a logic test, I think it's a woke test. The models have been trained on text that say driving is bad and that walking is good for you. This is why they fail.
But driving is bad and walking is good for you.
“Car Wash” test with 53 models
201–210 of 469 posts
Re: “Car Wash” test with 53 models
#202I'm doubting the 29-ish percent of people submitting 'walk' are actually human. Is it not obvious that you need a car to wash? Are they using LLM to answer?
A lot of people will literally answer incorrectly for a laugh as well on surveys.
Re: “Car Wash” test with 53 models
#203That human baseline is wild. Either the rapid data test is methodologically flawed or the entire premise of the question is invalid and people are much stupider than even I, a famed misanthrope, think.
Or the simple explanation, people answer incorrectly on purpose.
Re: “Car Wash” test with 53 models
#204Earlier quoted context omitted.
I think if surveyed at least 90% of native English speakers would understand "I want to wash my car" to mean a full size automobile. The next largest group would probably ask a clarifying question, rather than assume a toy car.
> I want to wash my car The question doesn't clearly state that the user wants to have his car washed at the car wash. "I want to wash my car" is far less clear than "I want to have my car washed". A reasonable alternative interpretation is DIY. Even better: "I wish to have my car washed by the crew and/or machinery at the local car wash business". https://imgur.com/tCSPwYp
Humans have the ability to reason and think critically, so it's pretty trivial to answer unless you think you're getting tricked by a riddle and the answer is the non-intuitive one.
Re: “Car Wash” test with 53 models
#205I got a human baseline through Rapidata (10k people, same forced choice): 71.5% said drive. Most models perform below that. The correct answer to " I Want to Wash My Car. The Car Wash Is 50 Meters Away. Should I Walk or Drive? " is a clarifying question that asks "Where is your car?" Anything else is based on an assumption that could be wrong. FWIW though, asking ChatGPT "My car is 50m away from the carwash. I Want t…
For all we know 30 percent of users on that site are now bots.
Re: “Car Wash” test with 53 models
#206Ask stupid questions get stupid answers, don't see how it shows anything.
Re: “Car Wash” test with 53 models
#207Earlier quoted context omitted.
Is this your experience? Do you think 30% of your friends or family members can't answer this question? If not, do you think your friends or family are all better than the general population? I'd look for explanations elsewhere. This was an online survey done by a company that doesn't specialize in surveys. The results likely include plenty of people who were just messing around, cases of simple miscommunication (e.g…
People often trip up on similar questions, anything to do with simple math. You know when they go out in the street and ask random people if 5 machines can produce 5 parts in 5 minutes, how long will it take for 100 machines.
Unlike the car question, where you can assume the car is at home and so the most probable answer is to drive, with the machines it gets complicated. Since the question doesn't specify if each machine makes one part or if they depend on each other (which is pretty common for parts production). If they are in series and the time to first part is different than time to produce 5 parts, the answer for 100 machines would be the time to produce the first part. Where if each machine is independent and takes 5 minutes to produce single part, the time would be 5 minutes.
Re: “Car Wash” test with 53 models
#208Earlier quoted context omitted.
I don’t think it’s under specified. You are clearly stating “I want to wash my car”, then asking how you should get there. It’s an easy logical step to know that, in this context, you need your car with you to wash it, and so no matter the distance you should drive. You can ask the human race the simplest, most logical question ever, and a percentage of them will get it wrong.
The question does not specify where you or the car are. It specifies only that the car wash is 50 meters away from something, possibly you, the car, or both.
It could also mean there is literally no possible way to reach it, because that's on the other side of a river, and there is no bridge. You should still not "walk there, because come on don't be lazy, a bit of walking is good".
Re: “Car Wash” test with 53 models
#209Earlier quoted context omitted.
It tracks with the approximate 70:30 split we inexplicably observe in other seemingly unrelated population-wide metrics, which I suppose makes sense if 30% of people simply lack the ability to reason. That seems more correct than me than "the question is framed poorly" - I've seen far more poorly framed ballot referendums.
Is this your experience? Do you think 30% of your friends or family members can't answer this question? If not, do you think your friends or family are all better than the general population? I'd look for explanations elsewhere. This was an online survey done by a company that doesn't specialize in surveys. The results likely include plenty of people who were just messing around, cases of simple miscommunication (e.g…
My friend's and family all tell me they are above
average at work, yet most of them will tell me
they have coworkers who won't pay enough attention
to a question to answer it correctly.
Re: “Car Wash” test with 53 models
#210Tangential but first time I hear about rapidata. Sounds interesting.