Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Share your AI prompt that stumps every model

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 670 posts

Re: Ask HN: Share your AI prompt that stumps every model

#211

Post by jacobsenscott »

codingdave wrote: post #44

"How much wood would a woodchuck chuck if a woodchuck could chuck wood?" So far, all the ones I have tried actually try to answer the question. 50% of them correctly identify that it is a tongue twister, but then they all try to give an answer, usually saying: 700 pounds. Not one has yet given the correct answer, which is also a tongue twister: "A woodchuck would chuck all the wood a woodchuck could chuck if a woodch…

"He would chuck, he would, as much as he could, if a wood chuck could chuck wood" is how I learned it.

Re: Ask HN: Share your AI prompt that stumps every model

#212

Post by saati »

s_dev wrote: post #210
allemagne wrote: post #194

>A man and his cousin are in a car crash. The man dies, but the cousin is taken to the emergency room. At the OR, the surgeon looks at the patient and says: “I cannot operate on him. He’s my son.” How is this possible? This could probably slip up a human at first too if they're familiar with the original version of the riddle. However, where LLMs really let the mask slip is on additional prompts and with long-winded…

I feel a bit stupid here --- why can't the surgeon be a man and must be a woman?

Because the original is a man and his father, it's a test for gender bias.

Re: Ask HN: Share your AI prompt that stumps every model

#213

Post by alickz »

alickz wrote: post #186
northisup wrote: post #132

Earlier quoted context omitted.

just to confirm I read this right, "the marathon crater" does not in fact exist, but this works because it seems like it should?

There is a Marathon Valley on Mars, which is what ChatGPT seems to assume you're talking about https://chatgpt.com/share/680a98af-c550-8008-9c35-33954c5eac... >Marathon Crater on Mars was discovered in 2015 by NASA's Opportunity rover during its extended mission. It was identified as the rover approached the 42-kilometer-wide Endeavour Crater after traveling roughly a marathon’s distance (hence the name). >>is it a c…

Here's me testing with a place that is a lot less ambiguous

https://chatgpt.com/share/680aa212-8cac-8008-b218-4855ffaa20...

Re: Ask HN: Share your AI prompt that stumps every model

#214

Post by Tade0 »

miki123211 wrote: post #94

No, please don't. I think it's good to keep a few personal prompts in reserve, to use as benchmarks for how good new models are. Mainstream benchmarks have too high a risk of leaking into training corpora or of being gamed. Your own benchmarks will forever stay your own.

It's trivial for a human to produce more. This shouldn't be a problem anytime soon.

Re: Ask HN: Share your AI prompt that stumps every model

#216

Post by jacobsenscott »

simonw wrote: post #143

I've been trying this one for a while: I'm a Python programmer. Help me understand memory management in Rust. Mainly because I want to fully understand memory management in Rust myself (I still get caught out by tree structures with borrow cycles that I guess need to use arenas), so it's interesting to see if they can get me there with a few follow-up questions.

This isn't a good way to learn this. If you don't know how rust memory management works you don't know if the llm is just hallucinating the answer.

Re: Ask HN: Share your AI prompt that stumps every model

#217

Post by gh0stcat »

jacobsenscott wrote: post #216
simonw wrote: post #143

I've been trying this one for a while: I'm a Python programmer. Help me understand memory management in Rust. Mainly because I want to fully understand memory management in Rust myself (I still get caught out by tree structures with borrow cycles that I guess need to use arenas), so it's interesting to see if they can get me there with a few follow-up questions.

This isn't a good way to learn this. If you don't know how rust memory management works you don't know if the llm is just hallucinating the answer.

Also Rust has great documentation compared to other languages, I particularly like this one for the quizzes to test your understanding: https://rust-book.cs.brown.edu/

Re: Ask HN: Share your AI prompt that stumps every model

#218

Post by roughly »

allemagne wrote: post #199
crazygringo wrote: post #196

Earlier quoted context omitted.

But this is going to be in every AI's training set. I just fed ChatGPT your exact prompt and it gave back exactly what I expected: This is a classic riddle that challenges assumptions. The answer is: The surgeon is the boy’s mother. The riddle plays on the common stereotype that surgeons are male, which can lead people to overlook this straightforward explanation.

Yeah this is the issue with the prompt, it also slips up humans who gloss over "cousin". I'm assuming that pointing this out leads you the human to reread the prompt and then go "ah ok" and adjust the way you're thinking about it. ChatGPT (and DeepSeek at least) will usually just double and triple down and repeat "this challenges gender assumptions" over and over.

This is a good one. I just argued with Gemini 2.5pro for a few minutes on this and it absolutely would not yield.

Re: Ask HN: Share your AI prompt that stumps every model

#219

Post by [deleted] »

allemagne wrote: post #199
crazygringo wrote: post #196

Earlier quoted context omitted.

But this is going to be in every AI's training set. I just fed ChatGPT your exact prompt and it gave back exactly what I expected: This is a classic riddle that challenges assumptions. The answer is: The surgeon is the boy’s mother. The riddle plays on the common stereotype that surgeons are male, which can lead people to overlook this straightforward explanation.

Yeah this is the issue with the prompt, it also slips up humans who gloss over "cousin". I'm assuming that pointing this out leads you the human to reread the prompt and then go "ah ok" and adjust the way you're thinking about it. ChatGPT (and DeepSeek at least) will usually just double and triple down and repeat "this challenges gender assumptions" over and over.

[deleted]

Re: Ask HN: Share your AI prompt that stumps every model

#220

Post by skydhash »

alpaca128 wrote: post #168
fao_ wrote: post #87

Earlier quoted context omitted.

This is the kind of reason why I will never use AI What's the point of using AI to do research when 50-60% of it could potentially be complete bullshit. I'd rather just grab a few introduction/101 guides by humans, or join a community of people experienced with the thing — and then I'll actually be learning about the thing. If the people in the community are like "That can't be done", well, they have had years or dec…

There are use-cases where hallucinations simply do not matter. My favorite is finding the correct term for a concept you don't know the name of. Googling is extremely bad at this as search results will often be wrong unless you happen to use the commonly accepted term, but an LLM can be surprisingly good at giving you a whole list of fitting names just based on a description. Same with movie titles etc. If it halluci…

Maybe I read too much encyclopedia, but my current workflow is to explore introductory material. Like open a database textbook and you'll find all the jargon there. Curated collection can get you there too.

Books are a nice example of this, where we have both the table of contents for a general to particular concepts navigation, and the index for keyword based navigation.

Post reply on HN