Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

Post reply on HN
Print view View original

491–500 of 526 posts

Re: A ChatGPT mistake cost us $10k

#491

Post by tk90 »

4hg4ufxhy wrote: post #346
internetter wrote: post #4

"Note: I want to preface this by saying yes the practices here are bad and could have been avoided. This was from a different time under large time constraints. Please read with that in mind" These "constraints" are why I'm terrified of subscribing to software

Having worked with some legacy subscription code it can be quite nasty. We had race conditions where we would charge users twice This has made me paranoid that any time I see timeout or error related to money I assume it went through and come back later.

out of curiosity, what's the technical solution? My guess: add an idempotency key to the request and a message queue? Then when you try to consume it, you check whether that request was made previously.

Re: A ChatGPT mistake cost us $10k

#492

Post by talldayo »

throw46365 wrote: post #294
qarl wrote: post #140

Earlier quoted context omitted.

> Tell me you had no business being invested in without telling me. Check out their comment history to see who invested in them.

The lesson I learned from the dot-com era is that people who are dependent on the hype to make profit will crane their necks to believe the hype. Salespeople, executives, engineers, it doesn’t matter. Every day on HN reminds me a little more of 1998.

Yep. Hacker News is one of the only social medias that I've seen where you have to reject reality in order to be accepted and upvoted by the majority. After a while it becomes more funny than sad, really.

Re: A ChatGPT mistake cost us $10k

#493

Post by ihumanable »

8n4vidtmkvmk wrote: post #215
brown9-2 wrote: post #141

Earlier quoted context omitted.

looking at the query logs for the nighttime period should have made the bug fairly obvious

They even said they had sentry set up.. they'd notice the duplicate key error immediately.

I read the part where they said they poured through "hundreds of sentry logs" and immediately was like "no you didn't."

This is not an error that would be difficult to spot in an error aggregator, it would throw some sort of constraint error with a reasonable error message.

Re: A ChatGPT mistake cost us $10k

#494

Post by LigmaBaulls »

arjvik wrote: post #329
primitivesuave wrote: post #314

I spotted the error instantly. With all due respect to your team - this has nothing to do with ChatGPT and everything to do with using a programming model that your team does not have sufficient expertise in. Even if this error managed to slip by code review, it would have been caught with virtually any monitoring solution, many of which take less than 5 minutes to set up.

To be fair, if I wasn't looking for this bug I never would have spotted it. That being said, you're entirely right that any monitoring or even the most basic manual testing should have instantly caught this.

I spotted it right away. If you plan to use a library in your project RTFM

Re: A ChatGPT mistake cost us $10k

#495

Post by divbzero »

For future reference, the ChatGPT’s mistake in the SQLAlchemy model:

  id = Column(String, primary_key=True, default=str(uuid.uuid4()), unique=True, nullable=False)
Can be fixed by making the default a function:

  id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()), unique=True, nullable=False)
Or, if you choose to use native UUID types in Python and SQL:

  id = Column(Uuid, primary_key=True, default=uuid.uuid4, unique=True, nullable=False)

Re: A ChatGPT mistake cost us $10k

#496

Post by namaria »

chx wrote: post #239
OutOfHere wrote: post #89

This is why you should always rush your engineers, never giving them enough time to validate or understand what ChatGPT just spewed. Good job. /s Meanwhile, I go through the tedious process of understanding ChatGPT's code letter-by-letter, also reading the docs, searching StackOverflow, even offering and rewarding bounties on StackOverflow, all to see if the code makes a shred of sense.

When I was coding https://stackoverflow.com/a/77210784/308851 well, I had no idea how to open an SSH connection using go, I barely knew basic Go so I asked ChatGPT. The meat of the SSH connecting code is still pretty much ChatGPT written (bad chx!) but it contained a couple defers like defer session.Close() which I needed to understand and remove before it became usable as a utility function. I did search for ssh.Pub…

> It's just an SSH connection, the risk is quite low

Yeah just a secure shell what ever could go wrong with code you don't understand? Probably nothing much right?

Re: A ChatGPT mistake cost us $10k

#497

Post by chx »

namaria wrote: post #496
chx wrote: post #239

Earlier quoted context omitted.

When I was coding https://stackoverflow.com/a/77210784/308851 well, I had no idea how to open an SSH connection using go, I barely knew basic Go so I asked ChatGPT. The meat of the SSH connecting code is still pretty much ChatGPT written (bad chx!) but it contained a couple defers like defer session.Close() which I needed to understand and remove before it became usable as a utility function. I did search for ssh.Pub…

> It's just an SSH connection, the risk is quite low Yeah just a secure shell what ever could go wrong with code you don't understand? Probably nothing much right?

well, it's client. What could go wrong? I mean, it might not connect, right? I did understand it to some degree.

Re: A ChatGPT mistake cost us $10k

#498

Post by threecheese »

beala wrote: post #283
threecheese wrote: post #154

In your defense, at least your solution wasn’t: ‘$ EXPORT MAX_REQUESTS=1 gunicorn bear:app’ … which would have in fact fixed your subscription problem, but gave you a new problem :) Great share!

Heads up Bear [1] is just the blogging platform, not the company this post is about. [1] https://bearblog.dev

lol, whoops :) Thank you.

Re: A ChatGPT mistake cost us $10k

#499

Post by OutOfHere »

chx wrote: post #497
namaria wrote: post #496

Earlier quoted context omitted.

> It's just an SSH connection, the risk is quite low Yeah just a secure shell what ever could go wrong with code you don't understand? Probably nothing much right?

well, it's client. What could go wrong? I mean, it might not connect, right? I did understand it to some degree.

I would make sure the SSH client doesn't connect to a man-in-the-middle!

Re: A ChatGPT mistake cost us $10k

#500

Post by chx »

OutOfHere wrote: post #499
chx wrote: post #497

Earlier quoted context omitted.

well, it's client. What could go wrong? I mean, it might not connect, right? I did understand it to some degree.

I would make sure the SSH client doesn't connect to a man-in-the-middle!

Well, that's fairly self evident. It's not like we are auditing thousands of lines of code: ssh.Dial("tcp", serverName, config)

the tricky part , really , is the syntax for ssh config but again that's easy to understand even if it's very hard to write.

Post reply on HN