The fact that it completely autonomously read in a 5 MB firmware image of an old piece of test equipment and generated a Python script to generate license keys:
https://tomverbeure.github.io/2026/04/12/AMIQ-License-Key-Ge...
Ask HN: What was your "oh shit" moment with GenAI?
321–330 of 1001 posts
Re: Ask HN: What was your "oh shit" moment with GenAI?
#322When deepseek found a fix for a bug I couldn't find in minutes.
When deepseek again produced an entire web app that somewhat looked alright.
When Gemini could finally produce json was I specified.
The issue is, all LLMs can do. When they do, is boilerplate and code a mediocre coder could produce if they cared to try and insist.
In a way we should praise the ability of these things, but at what (in) efficiency. Code still need to be reviewed as we can't trust these things and context got a limit to entertain the idea of possibly having them fix their own mess.
Re: Ask HN: What was your "oh shit" moment with GenAI?
#323I was trying to use Opus 4.6 in Claude Code to add some functionality to python code intended to run on a cluster and it didn't have any python environment in its remote environment. It needed to look at the schema of a parquet file to make sure it did things right and couldn't figure out how to do so with code because for god knows what reason there is no python environment in the dev environment for code intended to be run on a compute cluster in Python. Eventually it decided to just examine the raw binary bytes of the header, and then wrote perfectly functional code based on that.
On a different note I recently uploaded several thousand scraped IPO prospectuses to the gpt 5.4 mini API to parse and extract certain data. I ordered it in the system prompt to respond exactly with a specified JSON schema. When I got the results back and processed them there was not a single JSON parse error whatsoever. The model didn't have a single hallucination that created malformed JSON or JSON not matching the given schema across several hundred million input tokens and several million output tokens. And this was 5.4 Mini!
Re: Ask HN: What was your "oh shit" moment with GenAI?
#324That it could create mugshots of myself better than I could have managed to take.
Aka handsome, confident successful, affluent alpha male on a boat, yet looking perfectly like me.
Re: Ask HN: What was your "oh shit" moment with GenAI?
#325I won’t deny they are useful tools, but the hyperbole from the tech CEOs about them replacing all white collar workers in 12-18 months set the expectation so high that I’m still in the “fancy auto-complete” camp. It still feels nowhere close to replacing anyone, at least where I work. While useful, they haven’t been anywhere close to as useful as promised. Hallucinations and poor guidance are still a regular day-to-d…
> Hallucinations and poor guidance are still a regular day-to-day issue that makes it impossible for me to trust agents with anything.
I often hear this. Can you give me a question where a major LLM hallucinates or provides poor guidance? Reproducible would be great
Just a question to stump it.
Re: Ask HN: What was your "oh shit" moment with GenAI?
#326I've had many, but a recent one was when I figured I'd try asking Claude for help with my attempts at learning to draw, specifically anatomy. I uploaded one of my sketches and asked for feedback, expecting it to not be too useful, but it actually pointed out many issues that no one had ever pointed out to me, but perfectly explained some of the things that felt off to me. Out of curiosity I then also asked it to labe…
That's super interesting! What sort of feedback? Anatomical feedback ("Uhh, that's not where arms go...") or drawing (like tips on shading etc...)?
Re: Ask HN: What was your "oh shit" moment with GenAI?
#327I won’t deny they are useful tools, but the hyperbole from the tech CEOs about them replacing all white collar workers in 12-18 months set the expectation so high that I’m still in the “fancy auto-complete” camp. It still feels nowhere close to replacing anyone, at least where I work. While useful, they haven’t been anywhere close to as useful as promised. Hallucinations and poor guidance are still a regular day-to-d…
I remember in the weeks and months after ChatGPT was released, there were plenty of comments here - seemingly respected comments getting plenty of upvotes - about exponential grown meaning that all programmers - or even all knowledge workers - would have their jobs made unnecessary in, wow, two weeks! Or, well, maybe two months! Wait, actually, two years! Always two of something.
It’s the full-self-driving of the 2020s (complete with the never-ending ‘we actually have it now you just don’t understand!’)
[Edit: I don’t mean it’s useless, just that its boosters are overhyping it - expanding on and agreeing with Had they been more realistic with the promises and didn’t frame it as replacing all of us within 2 years, I would have been more excited about the tech.]
Re: Ask HN: What was your "oh shit" moment with GenAI?
#328I won’t deny they are useful tools, but the hyperbole from the tech CEOs about them replacing all white collar workers in 12-18 months set the expectation so high that I’m still in the “fancy auto-complete” camp. It still feels nowhere close to replacing anyone, at least where I work. While useful, they haven’t been anywhere close to as useful as promised. Hallucinations and poor guidance are still a regular day-to-d…
> Hallucinations and poor guidance are still a regular day-to-day issue that makes it impossible for me to trust agents with anything. I often hear this. Can you give me a question where a major LLM hallucinates or provides poor guidance? Reproducible would be great Just a question to stump it.
> Reproducible would be great
Wouldn’t it be great? I’m still waiting for reproducibility from LLMs.
Re: Ask HN: What was your "oh shit" moment with GenAI?
#329[deleted]
Re: Ask HN: What was your "oh shit" moment with GenAI?
#330I won’t deny they are useful tools, but the hyperbole from the tech CEOs about them replacing all white collar workers in 12-18 months set the expectation so high that I’m still in the “fancy auto-complete” camp. It still feels nowhere close to replacing anyone, at least where I work. While useful, they haven’t been anywhere close to as useful as promised. Hallucinations and poor guidance are still a regular day-to-d…
> I won’t deny they are useful tools, but the hyperbole from the tech CEOs about them replacing all white collar workers in 12-18 months set the expectation so high that I’m still in the “fancy auto-complete” camp.
Why would someone else's unrealistic assessment affect your assessment of the actual abilities you see?
Seems like your opinion is mostly politics-based