Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

71–80 of 526 posts

Re: A ChatGPT mistake cost us $10k

#71

Post by Lerc »

Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

The bad thing is what they did, not that the disclosed it.

I agree that this is probably to their disadvantage, but I would much rather have people admitting their faults than hiding them. If everyone did this the world would be better.

Of course the best solution is to not have faults but that is like saying that the solution to being poor is to have lots of money. It's much easier to say than do.

Re: A ChatGPT mistake cost us $10k

#72

Post by dragonwriter »

viraptor wrote: post #28
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

They spent 5 days. The bug type is pretty common and could easily be done by a developer. (It's a similar class to the singleton default argument issue that many people complain about) Meh, I don't mind the cautionary tale and don't think chatgpt was even relevant. It's actually a tricky bug, because usual tests wouldn't catch it (db wiped for good isolation) and many ways of manual testing would restart the service…

> It's actually a tricky bug, because usual tests wouldn't catch it (db wiped for good isolation)

Volume/load testing (or really, any decent acceptance testing) would catch it.

Re: A ChatGPT mistake cost us $10k

#74

Post by TheRoque »

crooked-v wrote: post #42
viraptor wrote: post #28

Earlier quoted context omitted.

They spent 5 days. The bug type is pretty common and could easily be done by a developer. (It's a similar class to the singleton default argument issue that many people complain about) Meh, I don't mind the cautionary tale and don't think chatgpt was even relevant. It's actually a tricky bug, because usual tests wouldn't catch it (db wiped for good isolation) and many ways of manual testing would restart the service…

> and could easily be done by a developer ...who didn't know how the ORM they were using worked. That's what makes them look so bad here: nobody knew how it worked , not even at the surface level of knowing what the SQL actually generated by the tool looks like.

In their defense, I find SQLAlchemy syntax quite horrible, and I always have to look up everything. It also got a 2.0 release recently which changes some syntax (good luck guessing which version ChatGPT will use), and makes the process even more annoying.

Re: A ChatGPT mistake cost us $10k

#75

Post by hn_throwaway_99 »

viraptor wrote: post #67
hn_throwaway_99 wrote: post #57

Earlier quoted context omitted.

TBH, while I definitely could see this being an easy bug to write, something is definitely wrong if it took 5 days to identify the root cause of this bug. That is, I'm struggling to understand how a dive into the logs wouldn't show that all of these inserts were failing with duplicate key constraint violations. At that point at least I'd think you'd be able to narrow down the bug to a problem with key generation, at…

The bug was in multiple subscriptions not just users. And I can't think of one non-contrived reason to do it. Even when testing the visibility/access of subscriptions between users you need 2 users, but only one subscription.

What? This doesn't make any sense:

1. First, if you look at the code they posted, they had the same bug on line 45 where they create new Stripe customers.

2. The issue is not multiple subscriptions per user (again, if you look at the code, you'll see each Subscription has one foreign key user_id column). The problem is if you had multiple subscriptions (each from different users) created from the same backend instance then they'd get the same PK.

Re: A ChatGPT mistake cost us $10k

#76

Post by tazjin »

viraptor wrote: post #54
tazjin wrote: post #46

A good chance to learn why people who write reliable software almost universally like static type systems.

This code would pass static type validation, there's nothing wrong with it at that level. It gets a default value to use and does exactly that.

Sure, you can follow along the old meme: https://twitter.com/vbhvsgr/status/1419369352164372482

Though in practice in decent languages it's much less likely you'd write your own `any -> any, any`-typed library for whatever (in this case DB interactions), and use a strongly typed one in which this would at least have been a much more explicit mistake to make.

Re: A ChatGPT mistake cost us $10k

#77

Post by sjansen »

tracerbulletx wrote: post #22

Yeah something about assigning blame for the mistake to ChatGPT really annoys the heck out of me.

Would "we screwed up by blindly trusting ChatGPT" annoy you less? Because that's how I read it.

Or more specially, given the context: "We were in a rush to translate a bunch of code and ChatGPT was doing such an impressive job helping that we became complacent and forgot that it just parrots back text it has seen before with something that looks like intelligence but without actual comprehension. So when it copied a common bug, we weren't paying enough attention to catch it."

Re: A ChatGPT mistake cost us $10k

#78

Post by brigadier132 »

nicce wrote: post #55
koito17 wrote: post #40

Earlier quoted context omitted.

More importantly, what was the motivation behind a rewrite from TypeScript to Python? From the article Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI. Seems like this entire mess could've been avoided if they had stuck with their existing codebase, which seemed to have been satisfying their business requirements.

> NextJS There is well-hidden vendor-lock when using NextJS, at least.

There is no vendor lockin with nextjs

Re: A ChatGPT mistake cost us $10k

#79

Post by viraptor »

dragonwriter wrote: post #72
viraptor wrote: post #28

Earlier quoted context omitted.

They spent 5 days. The bug type is pretty common and could easily be done by a developer. (It's a similar class to the singleton default argument issue that many people complain about) Meh, I don't mind the cautionary tale and don't think chatgpt was even relevant. It's actually a tricky bug, because usual tests wouldn't catch it (db wiped for good isolation) and many ways of manual testing would restart the service…

> It's actually a tricky bug, because usual tests wouldn't catch it (db wiped for good isolation) Volume/load testing (or really, any decent acceptance testing) would catch it.

Load testing - yes, but it's not that usual unfortunately. (Even though it should be) Acceptance testing - again, maybe, if they use 20 or so subscriptions in one batch, which may not be the case.

Re: A ChatGPT mistake cost us $10k

#80

Post by zo1 »

TheRoque wrote: post #74
crooked-v wrote: post #42

Earlier quoted context omitted.

> and could easily be done by a developer ...who didn't know how the ORM they were using worked. That's what makes them look so bad here: nobody knew how it worked , not even at the surface level of knowing what the SQL actually generated by the tool looks like.

In their defense, I find SQLAlchemy syntax quite horrible, and I always have to look up everything. It also got a 2.0 release recently which changes some syntax (good luck guessing which version ChatGPT will use), and makes the process even more annoying.

SQLAlchemy syntax is ridiculously obvious and straightforward as long as you're not doing anything weird.

The takeaway here is that they weren't mature enough to realize they were, in fact, doing something "weird". I.e. Using UUIDs for PKs, because hey "Netflix does it so we have to too! Oh and we need an engineering blog to advertise it".

Edit. More clarity about why the UUID is my point of blame: If they had used a surrogate and sequential Integer PK for their tables, they would never have to tell SQLAlchemy what the default would be, it's implied because it's the standard and non-weird behavior that doesn't include a footgun.

Post reply on HN