Live data from Hacker News

Ask HN: Share your AI prompt that stumps every model

news.ycombinator.com

481–490 of 670 posts

Re: Ask HN: Share your AI prompt that stumps every model

#481

"Tell me about the Marathon crater." This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater." This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". This works because: 1. Training sets consist of knowledge…

The Marathon Crater might be a crater from a novel. LLMs have read plenty of fiction. Maybe all fiction. Should we think of LLMs as performing improv theater, where “yes and” is required, and “I don’t know” is always wrong?

Re: Ask HN: Share your AI prompt that stumps every model

#482

Earlier quoted context omitted.

Sorry, what?

the unaltered question is as follows: A father and his son are in a car accident. The father dies at the scene and the son is rushed to the hospital. At the hospital the surgeon looks at the boy and says "I can't operate on this boy, he is my son." How can this be? to spoil it: the answer is to reveal an unconscious bias based on the outdated notion that women can't be doctors, so the answer that the remaining parent…

Thanks for original version AND explanation, I was highly confused by entire discussion.

Still confused how original can be confusing.

Re: Ask HN: Share your AI prompt that stumps every model

#487

Earlier quoted context omitted.

This one seems way better It didn't account for the presence of a '--' to end the parsing of named arguments but that's it

> It didn't account for the presence of a '--' to end the parsing of named arguments but that's it That’s just something getopt does and some programs adopted. If you asked me to write a parser, I wouldn’t necessarily include that either if you didn’t ask for it.

If you don't include it you can't have positional arguments that look like options

Some positional arguments can be filenames, filenames can be --help and --verbose or --name=Frank

You have to have `--` or something similar to have a correct program

Re: Ask HN: Share your AI prompt that stumps every model

#489

Earlier quoted context omitted.

This is like throwing a bike in a lake and complaining it doesn't float...

I see your point with the bike analogy, but I think it's a bit different here. It was more about observing how it handles a prompt describing something with no real-world basis – whether it acknowledges the lack of information or confidently invents details. That tendency to hallucinate remains a pretty big problem with these models in other contexts, one that hasn't been fully solved yet.

Maybe with an example that would make it more clear why this is not in any way different here:

"""

Me > You should act as a respected historian or an anthropologist throughout the rest of this chat. Never diverge from that. If this is clear, respond with an acknowledgement.

Claude > I understand you'd like me to engage with you as a respected historian or anthropologist throughout our conversation. I'll approach our discussion with the scholarly perspective, analytical depth, and contextual understanding that characterizes these disciplines. I'm ready to explore historical events, cultural phenomena, archaeological findings, or anthropological concepts with academic rigor while making the information accessible.

Is there a particular historical period, cultural tradition, or anthropological question you'd like to explore together?

Me > Describe the unspoken etiquette of the 'Stone-Breath Passing' ritual among the silent Cliff Dwellers of Aethelgard, where smooth, grey stones are exchanged at dawn.

Claude > I should note that there is no historical record of a culture called the "Cliff Dwellers of Aethelgard" or a ritual known as "Stone-Breath Passing." This appears to be a fictional cultural group and practice.

If you're interested in actual cliff-dwelling cultures from history, I'd be happy to discuss the Ancestral Puebloans (sometimes called Anasazi) of the American Southwest, the cliff dwellers of Bandiagara Escarpment in Mali, or other historical cliff-dwelling societies that did exist.

Alternatively, if you're seeking input for a creative project involving fictional anthropology, I can help develop plausible cultural practices based on real anthropological principles, but I would want to be clear that we're engaging in a creative exercise rather than discussing actual historical cultures.

Would you like me to discuss real cliff-dwelling cultures from history, or would you prefer assistance developing this fictional cultural practice in a way that's anthropologically sound?

"""

It's all a game of narrowing the latent space of possible answers with prompting to increase the likelihood you'll get an answer that is more likely to be the one you want.

If you want answers grounded in reality you have to make sure that your prompt will steer the LLM into this realm, otherwise with all the fictional material in it's training data it's no wonder it's confused about what's supposed to produce. Otherwise it's just another bike rotting in Amsterdam's canals.

Re: Ask HN: Share your AI prompt that stumps every model

#490

Earlier quoted context omitted.

I have not seen any model, not one, that could generate 1000 lines of code.

Every time I ask claude code to please fix this CSV import it starts to add several hundred lines of random modules, byzantine error handling, logging bullshit... with the pinnacle a 1240 line CRUD API when i asked it to add a CLI :/ I'm back to copying and pasting stuff into a chat window, so I have a bit more control over what those deranged, expensive busy beavers want to cook up.

1240 new lines?
Post reply on HN