Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

241–250 of 526 posts

Re: A ChatGPT mistake cost us $10k

#241

Post by gravescale »

foota wrote: post #225
gravescale wrote: post #224

Earlier quoted context omitted.

Like people who post "here's what ChatGPTx said" instead of their own answer. Quite literally, what is the point? However, I don't think it's really bad for the technical industries long term. It probably does mean that some companies with loose internal quality control and enough shiftless employees pasting enough GPT spew without oversight will go to the wall because their software became unmaintainable and not use…

Is the idea here that normal (read: low?) interest rates will let companies spend more time getting things right?

No, the idea is that historically-normal interest rates around the 5-10% mark won't be conducive to free VC cash being sprayed around for start-ups to wank themselves silly over "piv-iterating" endlessly over spamming complete nonsense and using headcount and office shininess as a substitute for useful and robust products.

Yes, it makes the barrier higher even for good products and helps entrench incumbents, but short of a transnational revolution, the macroeconomic system is what is it and you can only chose to find the good things in it or give up entirely.

Re: A ChatGPT mistake cost us $10k

#242

Post by theamk »

beachy wrote: post #137
throw46365 wrote: post #125

Earlier quoted context omitted.

> I don't think this is indicative of people who don't know what they're doing. I think this is indicative of people using "AI" tools to help with programming at all. I think using AI tools to write production code is probably indicative of people who don't really know what they are doing. The best way not to have subtle bugs is to think deeply about your code, not subcontract it out -- whether that is to people far…

> And where is the benefit in doing this repeatedly? You're just going to end up with a codebase nobody really owns on a cognitive level. You could say the same about the output of a compiler. No one owns that at a cognitive level. They own it at a higher level - the source code. Same thing here. You own the output of the AI at a cognitive level, because you own the prompts that created it.

Yes, and when compilers fail, it's a very complex problem to solve, that usually requires many hours from experienced dev. Luckily,

(1) Compilers are reproducible (or at least repeatable), so you can share your problem with other, and they can help.

(2) For common languages, there are multiple compilers and multiple optimization options, which (and that's _very important_) produce identically-behaving programs - so you can try compiling same program with different settings, and if they differ, you know compiler is bad.

(3) The compilers are very reliable, and bugs when compiler succeeds, but generates invalid code are even rarer - in many years of my career, I've only seen a handful of them.

Compare to LLMs, which are non-reproducible, each one is giving a different answer (and that's by design) and finally have huge appear-to-succeed-but-produce-bad-output error rate, with value way more than 1%. If you had a compiler that bad, you'd throw it away in disgust and write in assembly language.

Re: A ChatGPT mistake cost us $10k

#243

Post by ruszki »

Terretta wrote: post #99
userbinator wrote: post #21

Earlier quoted context omitted.

It read like no one really knew what they were doing. "We just let it generate the code and everything seemed to work" is certainly not a good way to market your company.

> It read like no one really knew what they were doing. "We just let [devs] generate the code and everything seemed to work" is certainly not a good way to [whatever]. Except, have you met startup devs? This is by and large the "move fast then unbreak things" approach.

It’s possible to move fast the same way, but break less things than this. For example, in this case, they said that they introduced tests to mitigate this. I can assure you that introducing tests takes more time than Google searches to check in like 2 minutes what each lines really does.

Re: A ChatGPT mistake cost us $10k

#244

Post by crabmusket »

sanderjd wrote: post #175
tmpz22 wrote: post #149

Earlier quoted context omitted.

Its very humbling coming out of startup-land and working with big tech engineers and realizing their tooling runs circles around everybody else and enables them to be much more precise with their work and scale, though it isn't without trade-offs.

Yeah but a lot of that is just the accrual of improvements that is possible with a lot of resources over a long period of time. People working in "big tech" aren't fundamentally better at building reliable tools and systems; the time and resource constraints are entirely different.

And the stakes! This outage might have cost the OP $10k. A similar snafu at a larger company might have cost tens of millions or more.

Re: A ChatGPT mistake cost us $10k

#245

Post by [deleted] »

Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

[deleted]

Re: A ChatGPT mistake cost us $10k

#246

Post by marginalia_nu »

hitekker wrote: post #186
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

That's an alright takeaway: the team made a rookie mistake and then they made a PR mistake by oversharing. Otherwise, I think this comment thread is a classic example why company engineering blogs choose to be boring. Better ten articles that have some useful information, than a single article that allows the commentariat to pile on and ruin your reputation.

Eh, I think it speaks fairly well for them.

On the one hand it does seem like a fairly inexperienced organization with some pretty undercooked release and testing processes, but on the other hand all that stuff is ultimately fixable. This is a relatively harmless way of learning that lesson. Admitting a problem is the first step toward fixing it.

A culture of ass-covering is much harder to fix, and will definitely get in the way of addressing these types of issues

Re: A ChatGPT mistake cost us $10k

#247

Post by AdamJacobMuller »

throwaway346434 wrote: post #142
dalemhurley wrote: post #129

Earlier quoted context omitted.

I was wondering that too. Why wouldn’t the error be in the logs?

Guess: the logs were on an ec2 instance that was thrown away regularly, and the overnight reports didn't give reproduce steps or timestamps; so when they checked it "works fine". There's value in having your backtrace surfaced to end users rather than swallowing an exception and displaying "didn't work".

it was on some temporary AWS service like lambda or something? (We had eight ECS tasks on AWS, all running five instances of our backend), but, regardless logs should be somewhere persistent.

If they weren't, that should be the first thing you fix.

Re: A ChatGPT mistake cost us $10k

#249

Post by [deleted] »

peter_l_downs wrote: post #118
internetter wrote: post #4

"Note: I want to preface this by saying yes the practices here are bad and could have been avoided. This was from a different time under large time constraints. Please read with that in mind" These "constraints" are why I'm terrified of subscribing to software

I have immense respect for the OP for writing up the story, and even more so for giving this preface. It's really useful to know what mistakes other people make, but can be quite embarrassing to tell others about mistakes you've made. Thanks, OP.

[deleted]

Re: A ChatGPT mistake cost us $10k

#250

Post by owl57 »

jorams wrote: post #41
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

Yeah this is not a good thing to advertise. - They were under large time constraints, but decided a full rewrite to a completely different stack was a good idea. - They copy-pasted a whole bunch of code, tested it manually once locally, once in production, and called it a day. - The debugging procedure for this issue so significant it made them dread waking up involved... testing it once and moving on. Every day. The…

> trivial to reproduce if you just try more than once

A lot more than once: they had 40 instances of their app, and the bug was only triggered by getting two requests on the same instance.

A bunch of developers including me once spent a whole weekend trying to reproduce a bug that was affecting production and/or guess from the logs where to look for it. Monday morning, team lead called a meeting, asked for everything we could find out, and… Opened the app in six tabs simultaneously and pressed the button in question in one of the tabs. And it froze! Knowing how to reproduce on our computers, we found and fixed the bug in the next 30 minutes.

Post reply on HN