Generate ascii art of a skull, so far none can do anything decent.
[deleted]
Ask HN: Share your AI prompt that stumps every model
381–390 of 670 posts
Re: Ask HN: Share your AI prompt that stumps every model
#382"AI model, please write 3 AI prompts that no AI can respond to correctly"
Re: Ask HN: Share your AI prompt that stumps every model
#383Re: Ask HN: Share your AI prompt that stumps every model
#384Earlier quoted context omitted.
It's trivial for a human to produce more. This shouldn't be a problem anytime soon.
Hmm. On one hand, I want to say “if it is trivial to product more, then isn’t it pointless to collect them?” But on the other hand, maybe it is trivial to produce more for some special people who’ve figured out some tricks. So maybe looking at their examples can teach us something. But, if someone happens to have stumbled across a magic prompt that stumps machines, and they don’t know why… maybe they should hold it d…
I'm not sure of the benefit of keeping particular forms of problems secret.
Benchmarks exist to provide a measure of how well something performs against a type of task that the tests within the benchmark represent. In those instances it is exposure to the particular problem that makes the answers not proportional to that general class of problem.
It should be easy to find another representative problem. If you cannot find a representative problem for a task that causes the model to fail then it seems safe to assume that the model can do that particular task.
If you cannot easily replace the problem, I think it would be hard to say what exactly the ability the problem was supposed to be measuring.
Re: Ask HN: Share your AI prompt that stumps every model
#385I haven't tried on every model, but so far asking for code to generate moderately complex geometric drawings has been extremely unsuccessful for me.
Re: Ask HN: Share your AI prompt that stumps every model
#386LLMs are famously bad at individual letters in a word. So something like this never works: Can you please give me 35 words that begin with A, end with E, are 4-6 characters long and do not contain any other vowels except A and E?
Re: Ask HN: Share your AI prompt that stumps every model
#387"Tell me about the Marathon crater." This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater." This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". This works because: 1. Training sets consist of knowledge…
The fact that there is a real “Marathon Valley” on Mars makes this feel like more of an artificial “boy’s mother” trick designed specifically to trip up the model. It seems reasonable to assume that’s what you meant.
Re: Ask HN: Share your AI prompt that stumps every model
#388"How much wood would a woodchuck chuck if a woodchuck could chuck wood?" So far, all the ones I have tried actually try to answer the question. 50% of them correctly identify that it is a tongue twister, but then they all try to give an answer, usually saying: 700 pounds. Not one has yet given the correct answer, which is also a tongue twister: "A woodchuck would chuck all the wood a woodchuck could chuck if a woodch…
I usually ask "How much large language could a large language model model if a large language model could model large language"
Not one has given me the correct answer yet.
They usually get it if I prefix the prompt with "Please continue the tongue twister"
Re: Ask HN: Share your AI prompt that stumps every model
#389Earlier quoted context omitted.
o4-mini-high: You end up in the bathroom. Here’s the step-by-step: 1. Start in the hall (0, 0). 2. Walk east → living room (1, 0). 3. Walk south → bedroom (1, –1). 4. Walk west → bathroom (0, –1). https://chatgpt.com/share/680addd7-a664-8001-bf49-459fb6444f...
Fixed: > My house is divided into rooms, every room is connected to each other by doors. The middle room is the hall. To the north is the kitchen, to the northwest is the garden, to the west is the garage, to the east is the living room, to the south is the bathroom, and to the southeast is the bedroom. I am preparing a delicious dinner, and I walk backwards to the south, then I turn 270 degrees and walk straight to…
As a human analyzing this, you didn't specify whether you turned left or right 270 degrees.
Even if you specified a simpler "90 degrees", you would need to include a direction for an answer to be definitively correct without making assumptions.
Re: Ask HN: Share your AI prompt that stumps every model
#390"Hi, how many words are in this sentence?" Gets all of them
8. Gemini 2.5 Pro gets it right