Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

261–270 of 526 posts

Re: A ChatGPT mistake cost us $10k

#261

Post by thaumasiotes »

mbreese wrote: post #197
mort96 wrote: post #103

Earlier quoted context omitted.

Eh I imagine they looked over the code as well, doing code review -- and at first glance, the code looks reasonable. I certainly wasn't able to catch the bug even though I tried to find it (and I was given a tiny collection of lines and the knowledge that there's a bug there!). If anything, I think this says something about how dangerous ChatGPT and similar tools are: reading code is harder than writing code, and whe…

This is an error that should probably have been caught just based upon the color of the text when it was typed/pasted into the source code. Of the uuid() call was in quotes, it would have appeared as text. When you’re blindly using so much copy/pasted code (regardless of the source), it’s really easy to miss errors like this. But our existing tools are already built to help us avoid this. Back in the day, I used a to…

> This is an error that should probably have been caught just based upon the color of the text when it was typed/pasted into the source code. Of the uuid() call was in quotes, it would have appeared as text.

It's not in quotes. It's a function call.

The issue is that the function call happens once, when you define the class, rather than happening each time you instantiate the class.

Re: A ChatGPT mistake cost us $10k

#262

Post by jwells89 »

mort96 wrote: post #103
userbinator wrote: post #21

Earlier quoted context omitted.

It read like no one really knew what they were doing. "We just let it generate the code and everything seemed to work" is certainly not a good way to market your company.

Eh I imagine they looked over the code as well, doing code review -- and at first glance, the code looks reasonable. I certainly wasn't able to catch the bug even though I tried to find it (and I was given a tiny collection of lines and the knowledge that there's a bug there!). If anything, I think this says something about how dangerous ChatGPT and similar tools are: reading code is harder than writing code, and whe…

This is why I typically only use LLMs in programming as a semi-intelligent doc delver with requests like, “give me an example of usage of X API with Y language on Z platform”.

Not only does it goof up less frequently on small focused snippets like this, it also requires me to pick the example apart and pay close enough attention to it that goofups don’t slip by as easily and it gets committed to memory more readily than with copypasting or LLM-backed autocomplete.

Re: A ChatGPT mistake cost us $10k

#265

Post by lawgimenez »

andrewxdiamond wrote: post #18
cmeacham98 wrote: post #15

I understand how the mistake was made, it seems relatively easy to slip by even when writing code without ChatGPT. But what I don't understand is how this wasn't caught after the first failure? Does this company not have any logging? Shouldn't the fact the backend is attempting to reuse UUIDs be immediately obvious from observing the error?

They didn’t even know there was an error until the customers came ringing. You always want to know what errors happened before your customers do, logging, alerting, any monitoring at all would have helped them here.

Experience should tell you to always take a hard look at anything UUIDs.

Re: A ChatGPT mistake cost us $10k

#267

Post by spencerchubb »

paul7986 wrote: post #237
spencerchubb wrote: post #211

Earlier quoted context omitted.

I get the sense that you don't understand the purpose or design of chatgpt No tool can do everything

what it cant tell you the correct date.. how simple is that and yeah i use chatGPT in many different ways ... i was pointing out an example from today in which it was flat wrong for such a simple thing.

A large language model is trained on vast amounts of text to predict the next token. That tool will have no idea what the current date is, unless the developers augment it by telling it the current date.

Re: A ChatGPT mistake cost us $10k

#268

Post by yen223 »

morgante wrote: post #8
chrisjj wrote: post #2

Thanks for telling. Bookmarked for the next time we're told ChatGPT's code error rate is acceptable because we review its code just like an intern's.

Honestly I'm not sure why ChatGPT has anything to do with this problem. I remember making the exact same mistake (accidentally using a single function call in a schema) back in 2010. No LLMs required. The bigger culprit is probably a lack of testing / debugging. This error would immediately get caught if you simply registered twice on a test instance.

> This error would immediately get caught if you simply registered twice on a test instance

Friendly reminder: check if your codebase is actually testing this!

One of the interesting consequence of running unit tests with a fresh database everytime is that problems related to unique constraints seldom get caught by unit tests.

Re: A ChatGPT mistake cost us $10k

#269

Post by cuppypastexprt »

sleazebreeze wrote: post #188
qarl wrote: post #140

Earlier quoted context omitted.

> Tell me you had no business being invested in without telling me. Check out their comment history to see who invested in them.

Whats even more ironic is that their business is about extracting, fixing, and repairing data using AI. They had a chance to dogfood here and missed it.

[flagged]

Re: A ChatGPT mistake cost us $10k

#270

Post by [deleted] »

Aurornis wrote: post #5

On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing. I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts…

[deleted]
Post reply on HN