Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

371–380 of 526 posts

Re: A ChatGPT mistake cost us $10k

#371

Post by brabel »

KennyBlanken wrote: post #352
primitivesuave wrote: post #314

I spotted the error instantly. With all due respect to your team - this has nothing to do with ChatGPT and everything to do with using a programming model that your team does not have sufficient expertise in. Even if this error managed to slip by code review, it would have been caught with virtually any monitoring solution, many of which take less than 5 minutes to set up.

It's not some innocent mistake. The title is purposefully clickbait / keyword-y, implying that it was chatgpt that made the 'mistake' for SEO and to generate panicked clicks. "We made a programming error in our use of an LLM, didn't do any QA, and it cost us $10k" doesn't generate the C-suite "oh shit what if ChatGPT fucks up, what's our exposure!?" reaction. There's a million middle and upper management posting this…

> By definition they're not capable of "mistakes" because nothing they generate is remotely guaranteed to be correct or accurate.

By that logic, nothing is capable of making mistakes :D.

> Hilarious that a post which is insanely clickbait (which the rules say should result in a title rewrite) got boosted by the mods.

You have a distorted view of what clickbait is and the rules of this site. I suggest you go calm down and try to stop hating on a technology which is just that: a technology! Like any other, it can be misused, but think about why exactly you feel so passionate about this particular technology.

Re: A ChatGPT mistake cost us $10k

#372

Post by tempcommenttt »

This is error is one that most Python programmers must have experienced early in their career. Usually the other way around, when they define a mutable default value and are hit by strange results. On when adding the current time (datetime.datetime.now()) as a function default.

As a rookie programmer you might not notice this pattern, but after getting hit a few times you’ll immediately see that. And default to either a factory or to None and then set the value inside the function.

ChatGPT code is only safe to use if you understand it. If you don’t, there’s always the risk that it will bite you.

Re: A ChatGPT mistake cost us $10k

#373

Post by hehdhdjehehegwv »

thruway516 wrote: post #206
hehdhdjehehegwv wrote: post #191

Earlier quoted context omitted.

Depending on your viewpoint, ORM are by nature a mistake. The time people spend learning the quirks of an ORM is much better put into learning SQL.

Honestly same can be said about a lot of frameworks. You will pry my vanilla JS debugged with print statements hand-coded in vi from these hands only when they're cold and dead.

Yeah, I have a workflow where I inject JavaScript into arbitrary webpages and store the results. There’s no substitute for vanilla js knowledge when you’re in the weeds.

Re: A ChatGPT mistake cost us $10k

#374

Post by KennyBlanken »

patates wrote: post #360
fredthedeadhead wrote: post #355

The blog post is 404ing, here's a Web archive link https://web.archive.org/web/20240610032818/https://asim.bear... The author has added an important edit: > I want to preface this by saying yes the practices here are very bad and embarrassing (and we've since added robust unit/integration tests and alerting/logging), could/should have been avoided, were human errors beyond anything, and very obvious in hindsight. > >…

Could they have deleted it because of all the negativity? They did make a silly mistake, but we are humans, and humans, be it individually or collectively, do make silly mistakes.

If you code for a hobby/fun, yeah, sure, it's a silly mistake.

If you're earning past six figures, are part of a team of programmers, call yourself an professional / engineer, and have technical management above you like a VP of Engineering, yadda yadda....then it's closer to systematic failure of the company's engineering practices than "mistake."

There is a reason we call it software engineering, not software fuckarounding (or, cough, "DevOps Engineeer".)

Software engineering practices assume people are going to make mistakes, and implements procedures to reduce the chances of that making it into production, and reduce the impact of those mistakes if they do make it into production.

Re: A ChatGPT mistake cost us $10k

#376

Post by cbozeman »

KennyBlanken wrote: post #352
primitivesuave wrote: post #314

I spotted the error instantly. With all due respect to your team - this has nothing to do with ChatGPT and everything to do with using a programming model that your team does not have sufficient expertise in. Even if this error managed to slip by code review, it would have been caught with virtually any monitoring solution, many of which take less than 5 minutes to set up.

It's not some innocent mistake. The title is purposefully clickbait / keyword-y, implying that it was chatgpt that made the 'mistake' for SEO and to generate panicked clicks. "We made a programming error in our use of an LLM, didn't do any QA, and it cost us $10k" doesn't generate the C-suite "oh shit what if ChatGPT fucks up, what's our exposure!?" reaction. There's a million middle and upper management posting this…

The only part of your post that I do not agree with is:

> It's like the Mr. Beast open-mouth-surprised expression thumbnail nonsense; you feel incredibly compelled to click it.

I feel incredibly compelled to ignore it.

Re: A ChatGPT mistake cost us $10k

#377

Post by raggi »

If that's representative of the order volume I'm quite curious what the motivation/business case is for shifting technology stacks from one slow dynamic environment to another slow dynamic environment?

Re: A ChatGPT mistake cost us $10k

#378

Post by [deleted] »

KennyBlanken wrote: post #374
patates wrote: post #360

Earlier quoted context omitted.

Could they have deleted it because of all the negativity? They did make a silly mistake, but we are humans, and humans, be it individually or collectively, do make silly mistakes.

If you code for a hobby/fun, yeah, sure, it's a silly mistake. If you're earning past six figures, are part of a team of programmers, call yourself an professional / engineer, and have technical management above you like a VP of Engineering, yadda yadda....then it's closer to systematic failure of the company's engineering practices than "mistake." There is a reason we call it software engineering , not software fuck…

[deleted]

Re: A ChatGPT mistake cost us $10k

#379

Post by conductr »

OptionOfT wrote: post #310
dalemhurley wrote: post #128

I have seen the same mistake made in code created by humans. Many times, especially in react / typescript/ JavaScript, someone will forget to use a lambda. I felt the blog post failed to articulate the root cause of the issue and went straight to blaming ChatGPT. When you rush and make large or non peer code reviewed commits to main it is going to happen. The real issue was when you rush, take shortcuts and don’t ade…

The code is written by ChatGPT, pushed, and subsequently reviewed by ChatGPT. /s (I hope)

Just another day livin’ in the matrix

Re: A ChatGPT mistake cost us $10k

#380

Post by rvnx »

astromaniak wrote: post #344
arjvik wrote: post #329

Earlier quoted context omitted.

To be fair, if I wasn't looking for this bug I never would have spotted it. That being said, you're entirely right that any monitoring or even the most basic manual testing should have instantly caught this.

Easy enough, ChatGPT can write verification code along the main line. Just ask.

+ humans could have done this mistake as well
Post reply on HN