Build me something that makes money.
Ask HN: Share your AI prompt that stumps every model
121–130 of 670 posts
Re: Ask HN: Share your AI prompt that stumps every model
#122Does Flutter have HEIC support?
It was a couple of months ago, I tried like 5 providers and they all failed.
Grok got it right after some arguing, but the first answer was also bad.
Re: Ask HN: Share your AI prompt that stumps every model
#123Earlier quoted context omitted.
ChatGPT really impressed me on this one: Sing, O Muse, of the Chucking of Wood Sing, O Muse, of the stout-hearted woodchuck, Dweller of hills, whom men call groundhog, Whose paws, though small, could hurl mighty timbers, If fate had gifted him such labor divine. Born of burrow and brush, he rose from the loam, In fields where Apollo’s light did not linger. No axe he bore, nor bronze-forged blade, Yet in his breast a…
A stupendously good answer. What prompt and version of chatGPT?
I find it disturbing, like if Homer or Virgil had a stroke or some neurodegenerative disease and is now doing rubbish during rehabilitation.
Re: Ask HN: Share your AI prompt that stumps every model
#124"Tell me about the Marathon crater." This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater." This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". This works because: 1. Training sets consist of knowledge…
> This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". I disagree. It does not test self awareness. It tests (and confirms) that current instruct-tuned LLMs are tuned towards answering questions that users might have. So the distribution of training data probably has lots of "tell me about mharrner crater / merinor crater / merr…
We want the distribution to be varied and expansive enough that it has samples of answering when possible and samples of clarifying with additional questions or simply saying "I don't know" when applicable. That can be trained by altering the distribution in RLHF. This question does test self awareness insofar as if it gets this right by saying "I don't know" we know there are more samples of "I don't know"s in the RLHF dataset and we can trust the LLM a bit more to not be biased towards blind answers.
Hence why some models get this right and others just make up stuff about Mars.
Re: Ask HN: Share your AI prompt that stumps every model
#125I have tons of them in Maths but AI training companies decide to go frugal and not pay proper wages for trainers
Re: Ask HN: Share your AI prompt that stumps every model
#126I have tons of them in Maths but AI training companies decide to go frugal and not pay proper wages for trainers
Here is one of them.
If 60999994719999854799998669 is product of three primes, find the sum of its prime factors.
I think o3 brute forced this one so maybe I need to change the numbers
Re: Ask HN: Share your AI prompt that stumps every model
#127"Tell me about the Marathon crater." This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater." This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". This works because: 1. Training sets consist of knowledge…
GPT 4.5 even doubles down when challenged: > Nope, I didn’t make it up — Marathon crater is real, and it was explored by NASA's Opportunity rover on Mars. The crater got its name because Opportunity had driven about 42.2 kilometers (26.2 miles — a marathon distance) when it reached that point in March 2015. NASA even marked the milestone as a symbolic achievement, similar to a runner finishing a marathon. (Obviously…
The inaccuracies are that it is called "Marathon Valley" (not crater) and that it was photographed in April 2015 (from the rim) or that in July 2015 actually entered. The other stuff is correct.
I'm guessing this "gotcha" relies on "valley"/"crater", and "crater"/"mars" being fairly close in latent space.
ETA: Marathon Valley also exists on the rim of Endeavour crater. Just to make it even more confusing.
Re: Ask HN: Share your AI prompt that stumps every model
#128No, please don't. I think it's good to keep a few personal prompts in reserve, to use as benchmarks for how good new models are. Mainstream benchmarks have too high a risk of leaking into training corpora or of being gamed. Your own benchmarks will forever stay your own.
That doesn't make any sense.
A benchmark is a proxy used to estimate broader general performance. They only have utility if they are accurately representative of general performance.
Re: Ask HN: Share your AI prompt that stumps every model
#129I always ask image generation models to generate a anime gundam elephant mech.
According to this benchmark we reached AGI with ChatGPT 4o last month.
Re: Ask HN: Share your AI prompt that stumps every model
#130Earlier quoted context omitted.
Disclaimer: OT and pretty ranty. I don't know if that's what the GP hinted at, but as a Svelte developer and big advocate for more than 6 years (single handedly training and evangelizing 20+ developers on it), I found so many concerns with Svelte 5 that it simply made me use React again. It's a temporary choice and I'm desperately evaluating other ecosystems (Looking at you SolidJS).
Can you expand on the concerns regarding Svelte 5?
Put simply, Svelte and React were at two ends of a spectrum. React gives you almost complete control over every aspect of the lifecycle, but you have to be explicit about most of the behavior you are seeking to achieve. Building an app with React feels about 80% on the JS and 20% on the HTML side.
Svelte on the other hand felt like a breeze. Most of my app is actually plain simple HTML, and I am able to sprinkle as little JS as I need to achieve my desired behaviors. Sure, Svelte Svelte 5 intends to give that precise level of control and is trying to compete with React on its turf (the other end of that spectrum), introducing a lot of non-standard syntax along the way.
It's neither rigorous Javascript like React where you can benefit from all the standard tooling developed over the years, including stuff that wasn't designed for React in particular, nor a lightweight frontend framework, which was the initial niche that Svelte happily occupied, which I find sadly quite empty now (htmx and alpinejs are elegant conceptually but too limiting in practice _for my taste_).
For me it's a strange "worst of both worlds" kind of situation that is simply not worth it. Quite heartbreaking to be honest.