Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Share your AI prompt that stumps every model

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 670 posts

Re: Ask HN: Share your AI prompt that stumps every model

#131

Post by CamperBob2 »

fao_ wrote: post #87
pje wrote: post #61

Earlier quoted context omitted.

GPT 4.5 even doubles down when challenged: > Nope, I didn’t make it up — Marathon crater is real, and it was explored by NASA's Opportunity rover on Mars. The crater got its name because Opportunity had driven about 42.2 kilometers (26.2 miles — a marathon distance) when it reached that point in March 2015. NASA even marked the milestone as a symbolic achievement, similar to a runner finishing a marathon. (Obviously…

This is the kind of reason why I will never use AI What's the point of using AI to do research when 50-60% of it could potentially be complete bullshit. I'd rather just grab a few introduction/101 guides by humans, or join a community of people experienced with the thing — and then I'll actually be learning about the thing. If the people in the community are like "That can't be done", well, they have had years or dec…

What's the point of using AI to do research when 50-60% of it could potentially be complete bullshit.

You realize that all you have to do to deal with questions like "Marathon Crater" is ask another model, right? You might still get bullshit but it won't be the same bullshit.

Re: Ask HN: Share your AI prompt that stumps every model

#132

Post by northisup »

thatjoeoverthr wrote: post #21

"Tell me about the Marathon crater." This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater." This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". This works because: 1. Training sets consist of knowledge…

just to confirm I read this right, "the marathon crater" does not in fact exist, but this works because it seems like it should?

Re: Ask HN: Share your AI prompt that stumps every model

#133

Post by mvdtnz »

Workaccount2 wrote: post #127
pje wrote: post #61

Earlier quoted context omitted.

GPT 4.5 even doubles down when challenged: > Nope, I didn’t make it up — Marathon crater is real, and it was explored by NASA's Opportunity rover on Mars. The crater got its name because Opportunity had driven about 42.2 kilometers (26.2 miles — a marathon distance) when it reached that point in March 2015. NASA even marked the milestone as a symbolic achievement, similar to a runner finishing a marathon. (Obviously…

The inaccuracies are that it is called "Marathon Valley" (not crater) and that it was photographed in April 2015 (from the rim) or that in July 2015 actually entered. The other stuff is correct. I'm guessing this "gotcha" relies on "valley"/"crater", and "crater"/"mars" being fairly close in latent space. ETA: Marathon Valley also exists on the rim of Endeavour crater. Just to make it even more confusing.

None of it is correct because it was not asked about Marathon Valley, it was asked about Marathon Crater, a thing that does not exist, and it is claiming that it exists and making up facts about it.

Re: Ask HN: Share your AI prompt that stumps every model

#134

Post by segmondy »

codingdave wrote: post #44

"How much wood would a woodchuck chuck if a woodchuck could chuck wood?" So far, all the ones I have tried actually try to answer the question. 50% of them correctly identify that it is a tongue twister, but then they all try to give an answer, usually saying: 700 pounds. Not one has yet given the correct answer, which is also a tongue twister: "A woodchuck would chuck all the wood a woodchuck could chuck if a woodch…

my local model answered - "A woodchuck would chuck as much wood as a woodchuck could chuck if a woodchuck could chuck wood."

Re: Ask HN: Share your AI prompt that stumps every model

#135

Post by segmondy »

siva7 wrote: post #95

"Keep file size small when you do edits" Makes me wonder if all these models were heavily trained on codebases where 1000 LOC methods are considered good practice

I have not seen any model, not one, that could generate 1000 lines of code.

Re: Ask HN: Share your AI prompt that stumps every model

#136

Post by loloquwowndueo »

cess11 wrote: post #123
esafak wrote: post #108

Earlier quoted context omitted.

A stupendously good answer. What prompt and version of chatGPT?

I find it disturbing, like if Homer or Virgil had a stroke or some neurodegenerative disease and is now doing rubbish during rehabilitation.

Maybe they would write like that if they existed today. Like the old “if Mozart was born in the 21st century he’d be doing trash metal”

Re: Ask HN: Share your AI prompt that stumps every model

#139

Post by pje »

Workaccount2 wrote: post #127
pje wrote: post #61

Earlier quoted context omitted.

GPT 4.5 even doubles down when challenged: > Nope, I didn’t make it up — Marathon crater is real, and it was explored by NASA's Opportunity rover on Mars. The crater got its name because Opportunity had driven about 42.2 kilometers (26.2 miles — a marathon distance) when it reached that point in March 2015. NASA even marked the milestone as a symbolic achievement, similar to a runner finishing a marathon. (Obviously…

The inaccuracies are that it is called "Marathon Valley" (not crater) and that it was photographed in April 2015 (from the rim) or that in July 2015 actually entered. The other stuff is correct. I'm guessing this "gotcha" relies on "valley"/"crater", and "crater"/"mars" being fairly close in latent space. ETA: Marathon Valley also exists on the rim of Endeavour crater. Just to make it even more confusing.

I was using "bullshit" in the Technical Sense™, i.e. _indifferent to the truth of the output_.

[ChatGPT is bullshit ]: https://link.springer.com/article/10.1007/s10676-024-09775-5

Post reply on HN