Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What was your "oh shit" moment with GenAI?

news.ycombinator.com

Post reply on HN
Print view View original

301–310 of 1001 posts

Re: Ask HN: What was your "oh shit" moment with GenAI?

#301

Post by kylecazar »

A couple of years ago now.

I asked it to write a script that would search for a specific string in footers in a massive series of DOCX files and change them according to some rules. The strings ended up being embedded in cells within an invisible table in the footers, the LLM realized this and switched strategy to a full deep traversal of the underlying XML. It correctly processed like 50 of these files in about 10 minutes, using libraries I wasn't aware of. I had spent an hour being annoyed before trying.

It was an "oh shit" moment for at least that category of work.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#303

Post by j_bum »

SamuelAdams wrote: post #227
jp57 wrote: post #175

Actually seems absurdly simple now, but sometime last year I was trying to figure out what I'd need to tow my daughter's car cross country with my truck: what are the trailer/dolly options, what do they cost, can my truck actually tow the combined weight, etc. I started out prompting ChatGPT kinda how I would with Google, one small prompt at a time, asking about various details. But after one or two of those I just t…

Similarly, I used gen ai to review a real estate purchase. I provided Zillow listing photos and serial numbers of all appliances, the electric panel, and a few additional not pictured areas that I took during the walk through. I prompted the AI to write a report as if it were a home inspector and it actually did a better job and identified some issues the paid 750 usd inspector missed.

From pictures alone? What are some examples?

Re: Ask HN: What was your "oh shit" moment with GenAI?

#304

Post by sothatsit »

I gave GPT-4 some source code and my existing tests, and asked it to write a new test, and it did it! It didn’t even run straight away, I had to fix it, but it still blew my mind.

Later, I wrote a ~5k line proxy for work in C, and gave the whole thing to ChatGPT o1 and asked it to review it. It found several real memory bugs, and now that service has been running since with no problems.

Just this week, I was trying to write a greedy solver to pick the best subset of block sizes to keep from a larger sweep for shorter testing. Opus 4.8 suggested that this could actually be solved as a MILP problem, and found the perfect solution in 5 mins. I’d never even heard of MILP before.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#305

Post by forgetfreeman »

For me the "oh shit" moment is when I realized that otherwise sane professionals, frequently in positions of authority, insist on taking these tools seriously. Zero thought put into any of the implications around unchecked anthropomorphism, security issues, employee knowledge retention, liability and other legal concerns, etc.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#306

Post by novaleaf »

just yesterday I felt that claude code was being aggressive in it's defense, so I lead my response with "Spicy Take! Here's why I think the bug is happening...."

Because of syncopathy it took my "Spicy Take" and decided to say basically "Even more than it could, your bug is happening RIGHT NOW"... which was just made up lies for dramatic fit.

Back to talking to Claude like I'm a robot I guess.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#307

Post by kstrauser »

gscott wrote: post #287
evdubs wrote: post #51

I tried to see if an LLM service provider could rewrite some legal docs where nothing was hallucinated in order to follow a consistent format to see what may be missing in the document. It could do that. Next, I wanted to see if this could be done with a local LLM. Gemma-4 handles this fine with an 8GB video card and a large context (128k). Next, I wanted to see if the model could also OCR these docs and translate th…

My son is in a lawsuit with his bank where they put through fraudulent charges and wouldn't charge them back then the bank sued him for the money. He is using Claude and Gemini fighting the original lawsuit and now has a counter-suit 100% using AI for everything. He puts it into different AI's to check everything against each other and to come up with more ideas. He started with ChatGPT, moved to Grok, then Claude, b…

I'm about as pro-AI as anyone here. I say this with love: anyone using general-purpose, consumer-grade AI for healthcare, law, or taxes is mad. Best wishes to your son, bless his heart, but please have him consult a qualified lawyer before showing up to court with model-drafted legal documents. Among other things, those chats are not privileged information[0] and the banks could subpoena chat transcripts to see what else he might have told them.

[0]https://natlawreview.com/article/new-york-court-rules-ai-doc...

Re: Ask HN: What was your "oh shit" moment with GenAI?

#308

Post by flyinglizard »

jp57 wrote: post #175

Actually seems absurdly simple now, but sometime last year I was trying to figure out what I'd need to tow my daughter's car cross country with my truck: what are the trailer/dolly options, what do they cost, can my truck actually tow the combined weight, etc. I started out prompting ChatGPT kinda how I would with Google, one small prompt at a time, asking about various details. But after one or two of those I just t…

It very plausibly might have been totally wrong.

Out of laziness I several times asked Claude and ChatGPT each some torque figures and other simple, hard data related to my dirt bike. They often got it completely wrong, but full of confidence every time. I never trust LLMs with hard data, unless you RAG the PDF into the context and even then it's sketchy.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#309

Post by mindcrime »

I don't remember one specific moment, but I was fairly impressed with ChatGPT from the first time I started interacting with it. Was I ready to call it "AGI"? No, absolutely not. But it was clear that it was something new, and it was also intuitively obvious to me that "this AI is as bad today as it will ever be" and that predicting the rate of change would be difficult.

The more I use these things, the more I'm 100% convinced that it makes sense to say they are "intelligent" (for some meaning of "intelligent"). AGI or "human level intelligence"? Still no[1]. But some kind of intelligence. And I'm quite happy to allow that there can be "intelligence" that doesn't work anything at all like human intelligence, so arguments of the form "this isn't real intelligence", etc, etc. carry very (very) little weight with me. I've actually been sitting on a half written blog post on this very topic for a while, titled "The Marquee Sign Says 'Artificial' Intelligence"[2]. Finding time to finish it has been the challenge.

And before somebody says "Use AI to write it for you". Nah. I am generally what you might call "pro AI" and / or an "AI enthusiast" but I still draw lines. I'll use AI for research, for outlining, for brainstorming, etc. sure. But I have a hard-line stance against letting AI fundamentally write for me. I want anything that goes out with my name associated with it to have my genuine voice.

[1]: I like the term "jagged intelligence" that Demis Hassabis has been using. That is to say, the bounds of the intelligence are jagged or spiky: very intelligent in certain areas, much less so in others.

[2]: for any old-skool pro-wrestling fans, yes, that is an intentional nod to "Double A" Arn Anderson and his "The marquee sign says 'wrestling'" catchphrase. :-)

Re: Ask HN: What was your "oh shit" moment with GenAI?

#310

Post by jsharf »

jzemeocala wrote: post #118

I bought an Alesis QS8.1 super cheap in perfect condition (was a top grade digital piano/synth in the 90s). and then i realized that ALL of the software (which i collected from defunct websites and archived on github) related to it was ancient and after a while of getting tired of using WINE every single time i decided i wanted a cross platform modern equivalent that did everything that several of these different pro…

Related story, while applying a firmware update to my Kawai CA49 piano, I bricked it due to flashing the wrong file (The process was broken, and I got desperate and tried something stupid, which bricked the piano). Claude walked me through looking for signs of life, and since OTA from the phone app wasn't working for me, it downloaded the Kawai Android APK, decompiled the Java, figured out the hardcoded key used for encrypting the firmware update. Extracted the piano firmware update, decrypted it, and then wrote a flashing script to program the piano from my laptop via bluetooth. My piano was back to working within an hour.
Post reply on HN