Live data from Hacker News

A ChatGPT mistake cost us $10k

asim.bearblog.dev

431–440 of 526 posts

Re: A ChatGPT mistake cost us $10k

#431

Earlier quoted context omitted.

This is how I felt when a Gitlab employee deleted the production database by doing it in the wrong terminal window.

Get into the habit of colour-coding your important SSH sessions, red/green/blue can give you a very powerful subconscious check before doing something very silly.

How do you do this so that it lasts? I find mine resets to default

Re: A ChatGPT mistake cost us $10k

#432

Felt like a clickbait headline to me, but there's no link back to the project. So I guess not. Definitely respect for telling an embarrassing story even if I disagree wholly with the title (both the implication that the ChatGPT mistake is to blame, and that it cost them $10k.) Anyway I believe the product in question is https://agentgpt.reworkd.ai

>AgentGPT is an autonomous AI Agent platform that empowers users to create and deploy customizable autonomous AI agents directly in the browser. Simply assign a name and goal to your AI agent, and watch as it embarks on an exciting journey to accomplish the assigned objective.

from https://docs.reworkd.ai/introduction

Whereas the blogpost clearly demonstrates that AI agents cannot be left totally "autonomous", their output might seem reasonable for those not well versed in particular domain but might have disastrous consequences.

VC bros are clearly gambling big on Linear Algebra.

Re: A ChatGPT mistake cost us $10k

#433
post #174

Earlier quoted context omitted.

They could also have done default=uuid.uuid4 to have a new id each time, or default=lambda : str(uuid.uuid4()). It's not really related to whether or not it's a database default. It seems quite unfair to place the blame on SQLAlchemy here, or even Python. Even a statically typed language wouldn't prevent this kind of issue - the author of the code is the only person who can decide when they mean "use this exact strin…

I could imagine some sort of static analysis saying that using a constant value instead of a function is a mistake for columns with unique set.

That's a good point! I suppose in Python that could be a guard in the column definition function.

Re: A ChatGPT mistake cost us $10k

#434

The blog post is 404ing, here's a Web archive link https://web.archive.org/web/20240610032818/https://asim.bear... The author has added an important edit: > I want to preface this by saying yes the practices here are very bad and embarrassing (and we've since added robust unit/integration tests and alerting/logging), could/should have been avoided, were human errors beyond anything, and very obvious in hindsight. > >…

the page on archive.org could not be loaded now...

but google cache still serves a copy...

https://webcache.googleusercontent.com/search?q=cache%3Ahttp...

Re: A ChatGPT mistake cost us $10k

#435
post #405

Earlier quoted context omitted.

Also facepalms here: UUIDs as strings and UUIDv4. UUIDs are just 128-bit values. They might be conventionally encoded for humans as hex, but storing them as 36-byte (plus a few more for length) strings is a pointless waste of both space and performance.

They don't have logs and commit directly to production 10/20 times a day. I don't think 128 bits vs 36 byte performance it's a main concern right now

It is most likely not a performance issues right now, but every pessismisation compound and catch up to you eventually.

36B vs 16B today, tomorrow you need an array of it, and now it isn't cache aligned, and more than twice the overhead.

Most likely instead of manipulating a 36B fixed length string, it is handled as a dynamic string, for extra runtime memory allocations, most likely consuming at least 64B per allocation. Etc etc.

Do this all over the codebase and now you know why all the moderne software is a sloth on what was a supercomputer 30y ago.

Re: A ChatGPT mistake cost us $10k

#436
not sure how. But when I asked GPT with this line and the issue was found exactly.

The issue is tiny and slipery for a very big table. But I am still curious of why test can not find it.

>During the work day, this was fine. We probably committed 10-20 times a day (directly to main of course) which would cause new backend deployments to occur, giving us 40 new IDs for customers to potentially use.

They just use the test env for prod? When to push code, the CICD should be run and some examples should be run too here. And every time, the env should be clean. Here the database does not change from test to production.

Re: A ChatGPT mistake cost us $10k

#437

The blog post is 404ing, here's a Web archive link https://web.archive.org/web/20240610032818/https://asim.bear... The author has added an important edit: > I want to preface this by saying yes the practices here are very bad and embarrassing (and we've since added robust unit/integration tests and alerting/logging), could/should have been avoided, were human errors beyond anything, and very obvious in hindsight. > >…

Could they have deleted it because of all the negativity? They did make a silly mistake, but we are humans, and humans, be it individually or collectively, do make silly mistakes.

If they would have made that mistake by writing code and just misunderstood something or oversaw the problem, fine. But making this mistake by copy-pasting from ChatGPT without proper review is just terrible.

Re: A ChatGPT mistake cost us $10k

#438

Earlier quoted context omitted.

If you code for a hobby/fun, yeah, sure, it's a silly mistake. If you're earning past six figures, are part of a team of programmers, call yourself an professional / engineer, and have technical management above you like a VP of Engineering, yadda yadda....then it's closer to systematic failure of the company's engineering practices than "mistake." There is a reason we call it software engineering , not software fuck…

This is how I felt when a Gitlab employee deleted the production database by doing it in the wrong terminal window.

Why do employees have write access to production DB?

Re: A ChatGPT mistake cost us $10k

#439
I consider a shame to read some of you, literally trashing and blaming the whole team after the article.

One thing is to healthly discuss how dangerous can be assuming a GPT-generated code is safe or not. Or how unit tests could identify this (could really in this specific case?). Or why you need oncall shifts and good alerts. But, come on.

I spotted the issue in the code at first sight, but that doesn't make me morally superior, nor smart enough to blame someone to publicly talk about their mistake. It only means I'm currently reading that kind of code a lot, and I know where to look at. Pass me some clever ARM code and I'll be unable to spot even the most superfluous mistake.

It seems HN is crowded by the most smart guys on the planet, who never had dumb mistakes and are SO "quality inclined" they need to blame someone for theirs.

edit: of course, the decision to make it public is questionable, but that's topic for another thread, IMHO.

Re: A ChatGPT mistake cost us $10k

#440

Earlier quoted context omitted.

Could they have deleted it because of all the negativity? They did make a silly mistake, but we are humans, and humans, be it individually or collectively, do make silly mistakes.

If they would have made that mistake by writing code and just misunderstood something or oversaw the problem, fine. But making this mistake by copy-pasting from ChatGPT without proper review is just terrible.

I don’t find the source of the error being a careless human writing original code without proper review vs a careless human copy/pasting code without proper review to be significantly different.
Post reply on HN