Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

31–40 of 526 posts

Re: A ChatGPT mistake cost us $10k

#31
post #20
post #2

Thanks for telling. Bookmarked for the next time we're told ChatGPT's code error rate is acceptable because we review its code just like an intern's.

Your take-away is that this is ChatGPT's fault rather than a failure of testing?

i can see missing this during testing since it relies on doing the flow twice. though i have a hard time imagining a team of people not figuring this out in less than 5 days

Re: A ChatGPT mistake cost us $10k

#32
post #25

I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?

> line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it

Yes, but that line is only evaluated once when the class is declared (aka when the application starts), it's not evaluated for every instance.

Re: A ChatGPT mistake cost us $10k

#33
post #3

I'm not familiar with this library, how does `text("(now())")` evaluate and why there are extra parentheses? And should that be a lambda expression as well, so that `create_date` isn't just the timestamp when the python process was started?

I'm not familiar with the library either, but that seems to be a SQL expression executed on the database server. It's basically a copy-paste from the official documentation[0]. So no, not a lambda expression, because it's not computed in Python.

As to the extra parentheses: I bet that's a force-of-habit thing to prevent potential issues. For example, it seems Sqlite requires them for exactly this kind of default definition[1]. It could also read to nasty bugs when the lack of parentheses in the resulting SQL could result in a different parse than expected[2]. Adding them just-to-be-safe isn't the worst thing to do.

[0]: https://docs.sqlalchemy.org/en/13/core/metadata.html

[1]: https://github.com/sqlalchemy/sqlalchemy/issues/4474

[2]: https://github.com/sqlalchemy/sqlalchemy/issues/5344

Re: A ChatGPT mistake cost us $10k

#34
post #25

I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?

I'm not a Pythonista, but I think the deal is that line 56 was only executed once, at class definition, so every time the server spun up, you got a new uuid that could only use once

Re: A ChatGPT mistake cost us $10k

#35
post #25

I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?

The issue is that uuid4() would be called a single each time the app was launched when that code was first loaded. Each record produced by an individual instance of the app would have the same ID.

Re: A ChatGPT mistake cost us $10k

#36
post #25

I dont get it , line 56 every time it was invoked would call the uuid4 function to generate a unique id isn’t it? Or was the issue due to the uuid4 function not getting invoked for any reason?

It's invoked once when the class is instanced in Python, then reuses that same value for as long as the class lives in memory.

Re: A ChatGPT mistake cost us $10k

#40
post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

More importantly, what was the motivation behind a rewrite from TypeScript to Python? From the article

  Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI.
Seems like this entire mess could've been avoided if they had stuck with their existing codebase, which seemed to have been satisfying their business requirements.
Post reply on HN