This trick went viral on TikTok last week, and it has already been patched. To get a similar result now, try saying that the distance is 45 meters or feet. The new one is with upside down glass: https://www.tiktok.com/t/ZP89Khv9t/
To me, the "patching" that is happening anytime some finds an absolutely glaring hole in how AIs work is so intellectually dishonest. It's the digital equivalent of house flippers slapping millennial gray paint on structural issues. It can't math correctly, so they force it to use a completely different calculator. It can't count correctly, unless you route it to a different reasoning. It feels like every other week…
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
551–560 of 998 posts
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#552This trick went viral on TikTok last week, and it has already been patched. To get a similar result now, try saying that the distance is 45 meters or feet. The new one is with upside down glass: https://www.tiktok.com/t/ZP89Khv9t/
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#553Earlier quoted context omitted.
I think part of the failure is that it has this helpful assistant personality that's a bit too eager to give you the benefit of the doubt. It tries to interpret your prompt as reasonable if it can. It can interpret it as you just wanting to check if there's a queue. Speculatively, it's falling for the trick question partly for the same reason a human might, but this tendency is pushing it to fail more.
It’s just not intelligent or reasoning, and this sort of question exposes that more clearly. Surely anyone who has used these tools is familiar with the sometimes insane things they try to do (deleting tests, incorrect code, changing the wrong files etc etc). They get amazingly far by predicting the most likely response and having a large corpus but it has become very clear that this approach has significant limitati…
Rather than a denial of intelligence, to me these failure modes raise the credence that LLMs are really onto something.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#554Earlier quoted context omitted.
Future models know it now, assuming they suck in mastodon and/or hacker news. Although I don't think they actually "know" it. This particular trick question will be in the bank just like the seahorse emoji or how many Rs in strawberry. Did they start reasoning and generalising better or did the publishing of the "trick" and the discourse around it paper over the gap? I wonder if in the future we will trade these AI t…
The answer can be “both”. They won’t get this specific question wrong again; but also they generalise, once they have sufficient examples. Patching out a single failure doesn’t do it. Patch out ten equivalent ones, and the eleventh doesn’t happen.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#555Earlier quoted context omitted.
That’s why I don’t understand why LLMs don’t ask clarifying questions more often. In a real human to human conversation, you wouldn’t simply blurt out the first thing that comes to mind. Instead, you’d ask questions.
This is a great point, because when you ask it (Claude) if it has any questions, it often turns out it has lots of good ones! But it doesn't ask them unless you ask.
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#556Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#557Earlier quoted context omitted.
> That is the entire point, right? Us having to specify things that we would never specify when talking to a human. Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn. But this will only happen after the last programmer has died and no-one will remember pro…
A structured language without ambiguity is not, in general, how people think or express themselves. In order for a model to be good at interfacing with humans, it needs to adapt to our quirks. Convincing all of human history and psychology to reorganize itself in order to better service ai cannot possibly be a real solution. Unfortunately, the solution is likely going to be further interconnectivity, so the model can…
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#558Earlier quoted context omitted.
I get that issue constantly. I somehow can't get any LLM to ask me clarifying questions before spitting out a wall of text with incorrect assumptions. I find it particularly frustrating.
For GPT at least, a lot of it is because "DO NOT ASK A CLARIFYING QUESTION OR ASK FOR CONFIRMATION" is in the system prompt. Twice. https://github.com/Wyattwalls/system_prompts/blob/main/OpenA...
Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#559Re: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
#560Earlier quoted context omitted.
I would say, the proper response to this question is not "walk, blablablah" but rather "What do you mean? You need to drive your car to have it washed. Did I miss anything?"
Yes, this is what irks me about all the chatbots, and the chat interface as a whole. It is a chat-like UX without a chat-like experience. Like you are talking to a loquacious autist about their favorite topic every time. Just ask me a clarifying question before going into your huge pitch. Chats are a back & forth. You don’t need to give me a response 10x longer than my initial question. Etc
Except, of course, when that is exactly what the user wants.