Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What was your "oh shit" moment with GenAI?

news.ycombinator.com

Post reply on HN
Print view View original

351–360 of 1001 posts

Re: Ask HN: What was your "oh shit" moment with GenAI?

#351

Post by bombcar »

j_bum wrote: post #303
SamuelAdams wrote: post #227

Earlier quoted context omitted.

Similarly, I used gen ai to review a real estate purchase. I provided Zillow listing photos and serial numbers of all appliances, the electric panel, and a few additional not pictured areas that I took during the walk through. I prompted the AI to write a report as if it were a home inspector and it actually did a better job and identified some issues the paid 750 usd inspector missed.

From pictures alone? What are some examples?

I presume something like this: https://www.penny-arcade.com/comic/2007/06/22/perfectly-reas...

Re: Ask HN: What was your "oh shit" moment with GenAI?

#352

Post by devmor »

I still haven’t had it.

I’ve been working with ML for most of my career, and “gen ai” since the days of matrix crunching for NLP to a 10-element response array on my 1080Ti.

The current generation of AI is frankly, only marginally more impressive to me than that era. The only thing I’m saying “oh shit” to is the deranged amount of capital debt being leveraged to make it usable.

Watching companies spend billions of tokens per minute letting their dev teams that barely know how to write a prompt beyond some tips and tricks to gain a fluctuating slightly negative to slightly positive productivity change that no one can quantify is making me feel like one of the only sane people left in the world.

Quantization is the only interesting change I’ve seen in years.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#353

Post by ihsw »

pesus wrote: post #219
buckle8017 wrote: post #201

Earlier quoted context omitted.

Gemini almost killed you. The exhaust blower not working triggered a safety that prevented the furnace from firing. Spinning it bypassed the safety. You likely inhaled a lot more carbon monoxide than you know.

Welp, AI almost killing someone is definitely an "oh shit" moment.

The most interesting part is that there is no direct line between someone's accidental death and a chatbot giving life-threatening advice.

Imagine one of the models that has "accidental-deaths-via-bad-advice" just slightly turned up, with the model-provider's intent being to kill 5% more people per year.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#354

Post by dang »

homeonthemtn wrote: post #333

This feels like a crab pot for Reddit content.

I was a bit uncertain when I first saw the thread, but I think it has turned out well: super diverse and there are some amazing stories in here.

Particularly the ones about obscure tech like koi pond pumps and old guitar pedals.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#355

Post by jesse_dot_id »

shreddude wrote: post #44

I could go on and on, but Claude recently decompiled the firmware of my camper van, documented all the CAN interfaces, then programmed an ESP32 module to talk to the van’s integrated systems (power, HVAC, lighting, tanks). That sort of embedded systems integration is completely out of my wheelhouse. I honestly don’t understand AI naysayers. I use Claude every day both professionally as a Solution Architect and person…

Same. I'm a DevOps engineer, so a jack of all trades master of none type of guy, and Claude Code backfills my knowledge gaps and turns me into kind of a superhero. I think it's key to already have a pretty good idea of what you're looking at, though.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#356

Post by al_borland »

bko wrote: post #325
al_borland wrote: post #251

I won’t deny they are useful tools, but the hyperbole from the tech CEOs about them replacing all white collar workers in 12-18 months set the expectation so high that I’m still in the “fancy auto-complete” camp. It still feels nowhere close to replacing anyone, at least where I work. While useful, they haven’t been anywhere close to as useful as promised. Hallucinations and poor guidance are still a regular day-to-d…

> Hallucinations and poor guidance are still a regular day-to-day issue that makes it impossible for me to trust agents with anything. I often hear this. Can you give me a question where a major LLM hallucinates or provides poor guidance? Reproducible would be great Just a question to stump it.

LLMs are nondeterministic, so it’s impossible to make something 100% reproducible. Even if it has an issue, it might do it in a different way. If it’s well publicized, they’ll patch that very specific example, but the foundational issue is still there (like counting the R’s in strawberry).

I still regularly run into the issue where it just makes up API endpoints, CLI commands, or add flags that simply don’t exist.

I also regularly ask it things and it gives me a bad answers, so I push back, and it says something to the effect of “you’re right, I didn’t consider that, let me look at that more”… then tells me the exact opposite of the previous response.

Or it “thing X has never happened”, and I ask what about , and it goes to look it up and says, “oh, thing X actually did happen.”

I run into this daily. Multiple times per day. How can I trust a system like this? Are people just blindly accepting what the LLM says as truth? Is that why people think it’s good?

Re: Ask HN: What was your "oh shit" moment with GenAI?

#357

Post by cdavid »

I wanted to understand the implementation of some numerical algorithms, and the tech reports were not enough.

I cloned the repo of said library, gave it claude and asked it to write a new technical report in math notation, but with annotation with link to the code so that I can pick up the details. It basically one shotted the full report and that helped me re-implement it in "pure python + numpy", "manually".

Re: Ask HN: What was your "oh shit" moment with GenAI?

#359

Post by bombcar »

kstrauser wrote: post #307
gscott wrote: post #287

Earlier quoted context omitted.

My son is in a lawsuit with his bank where they put through fraudulent charges and wouldn't charge them back then the bank sued him for the money. He is using Claude and Gemini fighting the original lawsuit and now has a counter-suit 100% using AI for everything. He puts it into different AI's to check everything against each other and to come up with more ideas. He started with ChatGPT, moved to Grok, then Claude, b…

I'm about as pro-AI as anyone here. I say this with love: anyone using general-purpose, consumer-grade AI for healthcare, law, or taxes is mad. Best wishes to your son, bless his heart, but please have him consult a qualified lawyer before showing up to court with model-drafted legal documents. Among other things, those chats are not privileged information[0] and the banks could subpoena chat transcripts to see what…

The only way to "win" as a small is to be pro se and be extremely diligent in understanding what is happening.

Then, it costs you nothing but time.

Re: Ask HN: What was your "oh shit" moment with GenAI?

#360

Post by acosmism »

Recently purchased an 100 year old home. it was dead in the middle of winter and the house has steam heating which wasnt working. a few screenshots and chatgpt gave me a step by step of which levers to pull and knobs to turn. this was terrifying considering i knew nothing about these systems. it worked!
Post reply on HN