Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

191–200 of 526 posts

Re: A ChatGPT mistake cost us $10k

#191

Post by hehdhdjehehegwv »

SmellTheGlove wrote: post #69
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

This is a pretty common mistake with sqlalchemy whether you’re using ChatGPT or not. I learned the same lesson years ago, although I caught it while testing. I write plenty of python and I just don’t often pass functions in as parameters. In this case you need to! For something like this where you’re generating a unique id and probably need it in every model, it’s better to write a new Base model that includes things…

Depending on your viewpoint, ORM are by nature a mistake.

The time people spend learning the quirks of an ORM is much better put into learning SQL.

Re: A ChatGPT mistake cost us $10k

#192

Post by serf »

i'm not a big openAI fan and even I think the title is crummy.

I think it should be "How we used chatGPT to make a 10k mistake." -- at least then it's honest about the party at fault, that being the startup that didn't vet generative code.

Relatedly i've been throwing AIs at the problem of ordering mods and dependencies for game engines; it's pretty astonishing the error rates you see involving medium-sized text lists.

A good experiment: take a 100 line text file of whatever, ask an AI to sort the lines by some criteria and output a text file, you'll get files back with less than 100 lines routinely.

These kind of things really limit my faith in those systems to work without a heavily leashed supervisor along side.

Re: A ChatGPT mistake cost us $10k

#193

Post by AndyKelley »

Two more problems identified solely from the screenshot:

* you have two competing subscription id columns.

* a uuid is not a string, it is a 128 bit integer. If your database limits to 64 bit integers then use a 64 bit integer for the id instead of a string, or use an array of 128 bytes.

Re: A ChatGPT mistake cost us $10k

#195

Post by hehdhdjehehegwv »

jorams wrote: post #41
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

Yeah this is not a good thing to advertise. - They were under large time constraints, but decided a full rewrite to a completely different stack was a good idea. - They copy-pasted a whole bunch of code, tested it manually once locally, once in production, and called it a day. - The debugging procedure for this issue so significant it made them dread waking up involved... testing it once and moving on. Every day. The…

Ironically one of my frequent GPT questions is “X is supposed to do Y, but is doing Z. What logs should I look at and what error messages to keep an eye out for?”

Re: A ChatGPT mistake cost us $10k

#196

Post by jbattle »

zeroonetwothree wrote: post #159
jbattle wrote: post #122

Earlier quoted context omitted.

create a subscription for a test user. delete it. Make sure you can create another subscription for the same user. create subscriptions with and without overlapping effective windows Those seem like very basic tests that would have highlighted the underlying issue

Hindsight is 20/20. It’s always easy to think of tests that would have caught the issue. Can you think of tests that will catch the next issue though?

I'd argue that a suite of tests that exercise all reasonably likely scenarios is table stakes. And would have caught this particular bug.

I'm not talking about 100% branch coverage, but 100% coverage of all happy paths and all unhappy paths that a user might reasonably bump into.

OK maybe not 100% of the scenarios the entire system expresses, but pretty darn close for the business critical flows (signups, orders, checkouts, whatever).

Re: A ChatGPT mistake cost us $10k

#197

Post by mbreese »

mort96 wrote: post #103
userbinator wrote: post #21

Earlier quoted context omitted.

It read like no one really knew what they were doing. "We just let it generate the code and everything seemed to work" is certainly not a good way to market your company.

Eh I imagine they looked over the code as well, doing code review -- and at first glance, the code looks reasonable. I certainly wasn't able to catch the bug even though I tried to find it (and I was given a tiny collection of lines and the knowledge that there's a bug there!). If anything, I think this says something about how dangerous ChatGPT and similar tools are: reading code is harder than writing code, and whe…

This is an error that should probably have been caught just based upon the color of the text when it was typed/pasted into the source code. Of the uuid() call was in quotes, it would have appeared as text. When you’re blindly using so much copy/pasted code (regardless of the source), it’s really easy to miss errors like this.

But our existing tools are already built to help us avoid this.

Back in the day, I used a tool from a group in Google called “error-prone”. It was great at catching things like this (and Lorne goal NPE in Java). It would examine code before compiling to find common errors like this. I wish we had more “quick” check tools for more languages.

Re: A ChatGPT mistake cost us $10k

#198

Post by thruway516 »

SmellTheGlove wrote: post #69
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

This is a pretty common mistake with sqlalchemy whether you’re using ChatGPT or not. I learned the same lesson years ago, although I caught it while testing. I write plenty of python and I just don’t often pass functions in as parameters. In this case you need to! For something like this where you’re generating a unique id and probably need it in every model, it’s better to write a new Base model that includes things…

That explains where ChatGpt is getting that from. I guess it is only as smart as the average coder.

Re: A ChatGPT mistake cost us $10k

#199

Post by 65 »

koito17 wrote: post #40
Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

More importantly, what was the motivation behind a rewrite from TypeScript to Python? From the article Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI. Seems like this entire mess could've been avoided if they had stuck with their existing codebase, which seemed to have been satisfying their business requirements.

Or if you really want to rewrite your back end, why not just use Express? It would be wildly quicker to rewrite than switching languages. That along with the article makes me question the competency of the company. They got customers, sure, but in the long run these decisions will pile up.

Re: A ChatGPT mistake cost us $10k

#200

Post by mattdw »

mort96 wrote: post #103
userbinator wrote: post #21

Earlier quoted context omitted.

It read like no one really knew what they were doing. "We just let it generate the code and everything seemed to work" is certainly not a good way to market your company.

Eh I imagine they looked over the code as well, doing code review -- and at first glance, the code looks reasonable. I certainly wasn't able to catch the bug even though I tried to find it (and I was given a tiny collection of lines and the knowledge that there's a bug there!). If anything, I think this says something about how dangerous ChatGPT and similar tools are: reading code is harder than writing code, and whe…

I'm a some-time Django developer and... I caught the bug instantly. Once I saw it was model/ORM code it was the first thing I looked for.

I say that not to brag because (a) default args is a known python footgun area already and (b) I'd hope most developers with any real Django or SQLAlchemy experience would have caught this pretty quick. I guess I'm just suggesting that maybe domain experience is actually worth something?

Post reply on HN