Ask HN: What is your go-to prompt to prove AI can be wrong?

#1

Post by throwaway_887 » Mon, Jul 06, 2026, 8:56 AM UTC

Just over-heard one of our seniors trying to explain to a new intern that AI can actually be wrong, no matter how much its answers sound very eloquent and well-structured.

I was wondering what question/prompt everyone uses to prove so to others.

I assume it depends on each person's expertise, or some people ask about a fact that is well-known as having been a victim of the mandela effect.

P.S. As a non-native speaker, I was about to pass my submission through AI to improve text clarity and understandability, but after remembering some comments about prefering the original text, I decided against it :)

Re: Ask HN: What is your go-to prompt to prove AI can be wrong?

#2

Post by saidnooneever » Mon, Jul 06, 2026, 9:50 AM UTC

i think the proof is in understanding how LLMs work, not a certain prompt. this because output isnt determenistic and hence a single prompt couldnt (theoretically) reliably prove anything. thats the exactly problem :')

Re: Ask HN: What is your go-to prompt to prove AI can be wrong?

#3

Post by al_borland » Mon, Jul 06, 2026, 12:50 PM UTC

Ask an AI anything you have even a remotely decent understanding of and it becomes pretty obvious that they are often wrong. I run into this on a daily basis for questions spanning multiple topics.

It worries me that interns are needing to have this proven to them.

Re: Ask HN: What is your go-to prompt to prove AI can be wrong?

#4

Post by haute_cuisine » Mon, Jul 06, 2026, 1:11 PM UTC

saidnooneever wrote: post #2

i think the proof is in understanding how LLMs work, not a certain prompt. this because output isnt determenistic and hence a single prompt couldnt (theoretically) reliably prove anything. thats the exactly problem :')

I'm curious, why exactly output is non-deterministic? Static weights + matrix multiplication should always return the same output, right?

Re: Ask HN: What is your go-to prompt to prove AI can be wrong?

#5

Post by mejutoco » Mon, Jul 06, 2026, 6:52 PM UTC

haute_cuisine wrote: post #4

saidnooneever wrote: post #2

i think the proof is in understanding how LLMs work, not a certain prompt. this because output isnt determenistic and hence a single prompt couldnt (theoretically) reliably prove anything. thats the exactly problem :')

I'm curious, why exactly output is non-deterministic? Static weights + matrix multiplication should always return the same output, right?

No, even with the same temperature.