Thanks for telling. Bookmarked for the next time we're told ChatGPT's code error rate is acceptable because we review its code just like an intern's.
Your take-away is that this is ChatGPT's fault rather than a failure of testing?
A ChatGPT mistake cost us $10k
31–40 of 526 posts
Re: A ChatGPT mistake cost us $10k
#32I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?
Yes, but that line is only evaluated once when the class is declared (aka when the application starts), it's not evaluated for every instance.
Re: A ChatGPT mistake cost us $10k
#33I'm not familiar with this library, how does `text("(now())")` evaluate and why there are extra parentheses? And should that be a lambda expression as well, so that `create_date` isn't just the timestamp when the python process was started?
As to the extra parentheses: I bet that's a force-of-habit thing to prevent potential issues. For example, it seems Sqlite requires them for exactly this kind of default definition[1]. It could also read to nasty bugs when the lack of parentheses in the resulting SQL could result in a different parse than expected[2]. Adding them just-to-be-safe isn't the worst thing to do.
[0]: https://docs.sqlalchemy.org/en/13/core/metadata.html
Re: A ChatGPT mistake cost us $10k
#34I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?
Re: A ChatGPT mistake cost us $10k
#35I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?
Re: A ChatGPT mistake cost us $10k
#36I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?
Re: A ChatGPT mistake cost us $10k
#37I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?
Re: A ChatGPT mistake cost us $10k
#38Re: A ChatGPT mistake cost us $10k
#39Re: A ChatGPT mistake cost us $10k
#40On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…
Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI.
Seems like this entire mess could've been avoided if they had stuck with their existing codebase, which seemed to have been satisfying their business requirements.