Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

351–360 of 526 posts

Re: A ChatGPT mistake cost us $10k

#351

Post by crabmusket »

dalemhurley wrote: post #128

I have seen the same mistake made in code created by humans. Many times, especially in react / typescript/ JavaScript, someone will forget to use a lambda. I felt the blog post failed to articulate the root cause of the issue and went straight to blaming ChatGPT. When you rush and make large or non peer code reviewed commits to main it is going to happen. The real issue was when you rush, take shortcuts and don’t ade…

This sort of issue seems common in a few places. E.g. Vue's component props can have defaults, and woe betide you if you use a literal object or array as a default, instead of a function that returns an object or array.

I'm surprised there was no lint rule for this case.

Re: A ChatGPT mistake cost us $10k

#352

Post by KennyBlanken »

primitivesuave wrote: post #314

I spotted the error instantly. With all due respect to your team - this has nothing to do with ChatGPT and everything to do with using a programming model that your team does not have sufficient expertise in. Even if this error managed to slip by code review, it would have been caught with virtually any monitoring solution, many of which take less than 5 minutes to set up.

It's not some innocent mistake. The title is purposefully clickbait / keyword-y, implying that it was chatgpt that made the 'mistake' for SEO and to generate panicked clicks.

"We made a programming error in our use of an LLM, didn't do any QA, and it cost us $10k" doesn't generate the C-suite "oh shit what if ChatGPT fucks up, what's our exposure!?" reaction. There's a million middle and upper management posting this article on LinkedIn, guaranteed.

It's like the Mr. Beast open-mouth-surprised expression thumbnail nonsense; you feel incredibly compelled to click it.

While we're on the subject: LLMs can't make "mistakes." They are not deterministic.

They cannot reason, think, or do logic.

They are very fancy word salad generators that use a lot of statistical probabilities. By definition they're not capable of "mistakes" because nothing they generate is remotely guaranteed to be correct or accurate.

Edit: The mods boosted the post; it got downvoted into oblivion, for obvious reasons, and then skyrocketed instantly in rank, which means they boosted it: https://hnrankings.info/40627558/

Hilarious that a post which is insanely clickbait (which the rules say should result in a title rewrite) got boosted by the mods.

I'm sure it's a complete coincidence that the story was apparently authored by someone at a Ycombinator company: https://news.ycombinator.com/item?id=40629998

Re: A ChatGPT mistake cost us $10k

#353

Post by vsuperpower2020 »

hedora wrote: post #299
dalemhurley wrote: post #128

I have seen the same mistake made in code created by humans. Many times, especially in react / typescript/ JavaScript, someone will forget to use a lambda. I felt the blog post failed to articulate the root cause of the issue and went straight to blaming ChatGPT. When you rush and make large or non peer code reviewed commits to main it is going to happen. The real issue was when you rush, take shortcuts and don’t ade…

My mental model for ChatGPT is that it’s an entry-level engineer that will never be promoted to a terminal level and will eventually be let go. However, this engineer can type infinitely fast, which means it might be useful if used very carefully. Anyway, letting such a person near financially important code would lead to similar issues, and in both cases, I’d question the judgment of the person that decided to deplo…

I want to reclaim the word engineer from Americans. Turns out everyone is an engineer, even the chatbots.

Re: A ChatGPT mistake cost us $10k

#354

Post by patates »

internetter wrote: post #4

"Note: I want to preface this by saying yes the practices here are bad and could have been avoided. This was from a different time under large time constraints. Please read with that in mind" These "constraints" are why I'm terrified of subscribing to software

Doing a rewrite under "large" time constraints is also unusual.

Re: A ChatGPT mistake cost us $10k

#355

Post by fredthedeadhead »

The blog post is 404ing, here's a Web archive link

https://web.archive.org/web/20240610032818/https://asim.bear...

The author has added an important edit:

> I want to preface this by saying yes the practices here are very bad and embarrassing (and we've since added robust unit/integration tests and alerting/logging), could/should have been avoided, were human errors beyond anything, and very obvious in hindsight.

>

> This was from a different time under large time constraints at the very earliest stages (first few weeks) of a company. I'm mostly just sharing this as a funny story with unique circumstances surrounding bug reproducibility in prod (due again to our own stupidity) Please read with that in mind

Re: A ChatGPT mistake cost us $10k

#357

Post by patates »

danpalmer wrote: post #105
contextnavidad wrote: post #43

> Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI This is the eye opener for me, how is a startup justifying a re-write when they don't even have customers?

And rewriting into a language that they lack experience in so much so that they can’t spot what are in my opinion really quite obvious bugs.

> a language that they lack experience in

Perhaps also the tooling because any remotely decent IDE should show an error there, let alone the potential warnings of some code analysis software.

Re: A ChatGPT mistake cost us $10k

#358

Post by randomdata »

TheDong wrote: post #303
sneak wrote: post #278

Earlier quoted context omitted.

TBH, if the backend were written in Go, this probably wouldn’t have happened to the extent it did. Somewhere in a log a descriptive error would have shown up. One of the reasons I use Go whenever possible is that it removes a lot of the classic Python footguns. If you are going to rewrite your backend from Javascript, why would you rewrite it in another untyped, error-prone language?

In python it's harder to ignore errors than in Go. In go, I've definitely seen: tx, err := db.Tx() defer tx.Commit() // silently ignores the error on committing, which is the important one That would have masked this error so it didn't get logged by the application. In python, if you ignore an exception entirely, like I did that error above, you instead get an exception logged by default. Python's exceptions also inc…

> I've definitely seen

What did they say was the thinking behind it? defer tx.Rollback() would make sense, but defer tx.Commit() is nonsensical, regardless of whether or not the error is handled. It seems apparent that the problem there isn't forgetting to check an error, but that someone got their logic all mixed up, confusing rollback with commit.

Re: A ChatGPT mistake cost us $10k

#359

Post by raverbashing »

MrJohz wrote: post #290
sdwr wrote: post #281

Earlier quoted context omitted.

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

It is correct, it's one of the most surprising things about Python and it causes a number of mistakes, even for experts. The easiest way to see this is by running something like this and seeing what gets printed out and when: print("1. start") def function(arg=print("2. func definition")): print("4. func call") print("3. after definition") function() function() function() You should see that the print statement in th…

> it's one of the most surprising things about Python and it causes a number of mistakes, even for experts.

Except every python 101 text seems to go over it, and people seem to have suddenly forgotten about it

ChatGPT driven development maybe?

Re: A ChatGPT mistake cost us $10k

#360

Post by patates »

fredthedeadhead wrote: post #355

The blog post is 404ing, here's a Web archive link https://web.archive.org/web/20240610032818/https://asim.bear... The author has added an important edit: > I want to preface this by saying yes the practices here are very bad and embarrassing (and we've since added robust unit/integration tests and alerting/logging), could/should have been avoided, were human errors beyond anything, and very obvious in hindsight. > >…

Could they have deleted it because of all the negativity?

They did make a silly mistake, but we are humans, and humans, be it individually or collectively, do make silly mistakes.

Post reply on HN