[dead]
A ChatGPT mistake cost us $10k
121–130 of 526 posts
Re: A ChatGPT mistake cost us $10k
#122Earlier quoted context omitted.
TBH, while I definitely could see this being an easy bug to write, something is definitely wrong if it took 5 days to identify the root cause of this bug. That is, I'm struggling to understand how a dive into the logs wouldn't show that all of these inserts were failing with duplicate key constraint violations. At that point at least I'd think you'd be able to narrow down the bug to a problem with key generation, at…
The bug was in multiple subscriptions not just users. And I can't think of one non-contrived reason to do it. Even when testing the visibility/access of subscriptions between users you need 2 users, but only one subscription.
create a subscription for a test user. delete it. Make sure you can create another subscription for the same user.
create subscriptions with and without overlapping effective windows
Those seem like very basic tests that would have highlighted the underlying issue
Re: A ChatGPT mistake cost us $10k
#123On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…
I appreciate the author's honesty. Its better to see transparently what happened so customers know the problem is fixed.
Re: A ChatGPT mistake cost us $10k
#124> Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI. Tell me you had no business being invested in without telling me. I’m going to be harsh here but I honestly have no clue how else to respond. You wrote your backend in Node/Typescript and then decided to change it to Python. What in the world would make that a good idea? No seriously, there is absolutely nothin…
It’s probably because they wanted to use some Python library to run inference or something. Although perhaps IPC would be cheaper than a rewrite.
Re: A ChatGPT mistake cost us $10k
#125Earlier quoted context omitted.
It read like no one really knew what they were doing. "We just let it generate the code and everything seemed to work" is certainly not a good way to market your company.
Eh I imagine they looked over the code as well, doing code review -- and at first glance, the code looks reasonable. I certainly wasn't able to catch the bug even though I tried to find it (and I was given a tiny collection of lines and the knowledge that there's a bug there!). If anything, I think this says something about how dangerous ChatGPT and similar tools are: reading code is harder than writing code, and whe…
> I don't think this is indicative of people who don't know what they're doing. I think this is indicative of people using "AI" tools to help with programming at all.
I think using AI tools to write production code is probably indicative of people who don't really know what they are doing.
The best way not to have subtle bugs is to think deeply about your code, not subcontract it out -- whether that is to people far away who both cannot afford to think as deeply about your code and aren't as invested in it, or to an AI that is often right and doesn't know the difference between correct and incorrect.
It's just a profound abrogation of good development principles to behave this way. And where is the benefit in doing this repeatedly? You're just going to end up with a codebase nobody really owns on a cognitive level.
At least when you look at a StackOverflow answer you see the discussion around it from other real people offering critiques!
ETA in advance: and yes, I understand all the comparison points about using third party libraries, and all the left-pad stuff (don't get me started on NPM). But the point stands: the best way not to have bugs is to own your code. To my mind, anyone who is using ChatGPT in this way -- to write whole pieces of business logic, not just to get inspiration -- is failing at their one job. If it's to be yours, it has to come from the brain of someone who is yours too. This is an embarrassing and damaging admission and there is no way around it.
ETA (2): code review, as a practice, only works when you and the people who wrote the code have a shared understanding of the context and the goal of the code and are roughly equally invested in getting code through review. Because all the niche cases are illuminated by those discussions and avoided in advance. The less time you've spent on this preamble, the less effective the code review will be. It's a matter of trust and culture as much as it's a matter of comparing requirements with finished code.
Re: A ChatGPT mistake cost us $10k
#126This kind of thing must be happening much more often than we're hearing about it, right? I'd love to start a site that collects AI "horror stories", where trusting an AI's output led to significant consequences.
I have no idea how to validate people's anecdotes, though. (To be clear I don't doubt this story at all. But if I set up a site where people could submit stories I wouldn't trust any submissions at face value.)
Re: A ChatGPT mistake cost us $10k
#127During the work day, this was fine. We probably committed 10-20 times a day That's... scary, to put it mildly. I wonder how many of those are fixes to things broken by previous commits. Then again, I work on software where the average is far less than one commit per day, although it's a mature product. Nonetheless, "slow down and think" is probably good advice in this case.
One pearl of current software wisdom is "don't think, just do", as a corollary to "move fast, and unbreak things later". Never mind that the cost of unbreaking things is usually far higher than whatever expenses were notionally saved by going to plaid in a hurry.
Re: A ChatGPT mistake cost us $10k
#128I have seen the same mistake made in code created by humans. Many times, especially in react / typescript/ JavaScript, someone will forget to use a lambda.
I felt the blog post failed to articulate the root cause of the issue and went straight to blaming ChatGPT.
When you rush and make large or non peer code reviewed commits to main it is going to happen.
The real issue was when you rush, take shortcuts and don’t adequately test and peer code review then errors will occur.
I would have imagined that a test that tried a few different signup options would have found the issue immediately.
Re: A ChatGPT mistake cost us $10k
#129On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…
The fact that they couldn't find it by looking at error logs is weird to me. This is an entirely forgivable error but should have been found the first time they got an email about it: "Oh, look, the error logs have a duplicate key exception for the primary key, how do we generate primary keys.... (facepalm)" Funnily enough, I saw the error in their snippet as soon as I read it but dismissed it thinking there was some…
I was wondering that too. Why wouldn’t the error be in the logs?
Re: A ChatGPT mistake cost us $10k
#130Didn't the logs say something like "duplicate key value violates unique constraint [...]" ?
That's what I don't get. Five days to query CloudWatch logging? This should have been caught before the first email even came in. "Gee, isn't it strange how we get these spikes in stderr output on our backend last night?"