Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Share your AI prompt that stumps every model

news.ycombinator.com

Post reply on HN
Print view View original

241–250 of 670 posts

Re: Ask HN: Share your AI prompt that stumps every model

#241

Post by briannotbrain »

bavarianbob wrote: post #229
saati wrote: post #212

Earlier quoted context omitted.

Because the original is a man and his father, it's a test for gender bias.

Sorry, what?

The original, well-known version of the riddle starts "A man and his son..." so that it appears to present a paradox if your instinctive assumption is that the surgeon must be a man. The op's prompt alters this so that there is no potential paradox, and it tests whether the model is reasoning from the prompt as written, regardless of the presence of the original riddle in its training data.

Re: Ask HN: Share your AI prompt that stumps every model

#242

Post by drodgers »

webglfan wrote: post #23

what are the zeros of the following polynomial: \[ P(z) = \sum_{k=0}^{100} c_k z^k \] where the coefficients \( c_k \) are defined as: \[ c_k = \begin{cases} e^2 + i\pi & \text{if } k = 100, \\ \ln(2) + \zeta(3)\,i & \text{if } k = 99, \\ \sqrt{\pi} + e^{i/2} & \text{if } k = 98, \\ \frac{(-1)^k}{\Gamma(k+1)} + \sin(k) \, i & \text{for } 0 \leq k \leq 97, \end{cases} \]

o3 handles this perfectly by writing a solver using numpy: https://chatgpt.com/share/680aab8e-cf9c-8012-9f48-301ef62948...

    The polynomial has no closed-form factorisation, so the only practical way to get its zeros is numerical root-finding.
    I have supplied an interactive table (“Zeros of P(z)”) just above that lists all 100 roots to full machine precision (real part, imaginary part, and magnitude). You can sort or filter it as needed.
    
     Reliability notes
     • Computed with numpy.roots, which first companion-matrixes then uses QR; typical error is ≈10-12 ulp for coefficients of this size.
     • Condition number is modest (coefficients range from O(1) down to 1/97!), so the results should be accurate to at least 10 significant figures.
     • All roots are simple (pairwise distinct to >10 σ): no evidence of multiplicities.
    
    If you need higher precision (e.g. 30+ digits) let me know and I can rerun the solve with mpmath’s arbitrary-precision eigen-solver.

Re: Ask HN: Share your AI prompt that stumps every model

#243

Post by simonw »

jacobsenscott wrote: post #216
simonw wrote: post #143

I've been trying this one for a while: I'm a Python programmer. Help me understand memory management in Rust. Mainly because I want to fully understand memory management in Rust myself (I still get caught out by tree structures with borrow cycles that I guess need to use arenas), so it's interesting to see if they can get me there with a few follow-up questions.

This isn't a good way to learn this. If you don't know how rust memory management works you don't know if the llm is just hallucinating the answer.

That's why it's an interesting test: I don't know the answer myself, so it's an exercise in learning with an unreliable teacher.

If a model ever DOES nail this I'll figure that out when I feel like I have a solid mental model, try to put that knowledge into action and it works.

Re: Ask HN: Share your AI prompt that stumps every model

#244

Post by ofou »

ls612 wrote: post #209
rybosworld wrote: post #176

Earlier quoted context omitted.

Tuning the model output to perform better on certain prompts is not the same as improving the model. It's valid to worry that the model makers are gaming the benchmarks. If you think that's happening and you want to personally figure out which models are really the best, keeping some prompts to yourself is a great way to do that.

Who’s going out of their way to optimize for random HNers informal benchmarks?

considering the amount of bots in HN, not really that much

Re: Ask HN: Share your AI prompt that stumps every model

#245

Post by fragmede »

Tade0 wrote: post #214
miki123211 wrote: post #94

No, please don't. I think it's good to keep a few personal prompts in reserve, to use as benchmarks for how good new models are. Mainstream benchmarks have too high a risk of leaking into training corpora or of being gamed. Your own benchmarks will forever stay your own.

It's trivial for a human to produce more. This shouldn't be a problem anytime soon.

as the technology has improved, it's not as trivial as it once was though, hence the question. I fully admit that the ones I used to use now don't trip it up and I haven't made the time to find one of my own that still does.

Re: Ask HN: Share your AI prompt that stumps every model

#246

Post by danielbln »

GoToRO wrote: post #195
imoreno wrote: post #180

Earlier quoted context omitted.

LLMs currently have the "eager beaver" problem where they never push back on nonsense questions or stupid requirements. You ask them to build a flying submarine and by God they'll build one, dammit! They'd dutifully square circles and trisect angles too, if those particular special cases weren't plastered all over a million textbooks they ingested in training. I suspect it's because currently, a lot of benchmarks are…

They do. Recently I was pleasantly surprised by gemini telling me that what I wanted to do will NOT work. I was in disbelief.

I've noticed Gemini pushing back more as well, whereas Claude will just butter me up and happily march on unless I specifically request a critical evaluation.

Re: Ask HN: Share your AI prompt that stumps every model

#250

Post by philipkglass »

boleary-gl wrote: post #249

I like: Unscramble the following letters to form an English word: “M O O N S T A R E R” The non-thinking models can struggle sometimes and go off on huge tangents

Llama 3.3 worked but (as you said) struggled before arriving at the correct answer. The newer Gemma3 solved it efficiently:

  % ollama run gemma3:27b-it-qat 
  >>> Unscramble the following letters to form an English word: "M O O N S T A R E R"
  The unscrambled word is **ASTRONOMER**.
Post reply on HN