A ChatGPT mistake cost us $10k • Hacker News Board

Re: A ChatGPT mistake cost us $10k

#281

Post by sdwr » Mon, Jun 10, 2024, 4:04 AM UTC

kazinator wrote: post #257

I've written less than 1000 lines of Python in total probably, but I correctly spotted the problem. Python has this misfeature whereby it didn't correctly crib Common Lisp's evaluation strategy for the expressions that give default values to optional function arguments. When you have an argument like foo=obj.whatever() the obj.whatever() is evaluated (would you believe it!) at the time the definition of the function…

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called.

This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

Re: A ChatGPT mistake cost us $10k

#282

Post by ncallaway » Mon, Jun 10, 2024, 4:07 AM UTC

nomilk wrote: post #273

Part of the skill in using LLMs is knowing when and how to use them, how to set the 'temperature' (how 'creative' it will be in its response), and how to write a prompt that is less prone to illusory responses. My eyes were opened one relaxing morning when sipping my coffee and pondering how to tidy up a database column by migrating from string to enum. I asked ChatGPT for its thoughts and its response seemed perfunc…

I’ve found it’s particularly useful at questions like “how do I do X idiomatically in Rust?”

Idioms are, very conveniently, questions about what the most common shape of X among the broader community, so it tends to do quite well with that.

I appreciate when it spits out example code, but I never copy/paste from it, I always rewrite anything myself to ensure I don’t slip up and accidentally… well, what it tried to sneak in to you…

That’s quite the scary anecdote

Re: A ChatGPT mistake cost us $10k

#283

Post by beala » Mon, Jun 10, 2024, 4:10 AM UTC

threecheese wrote: post #154

In your defense, at least your solution wasn’t: ‘$ EXPORT MAX_REQUESTS=1 gunicorn bear:app’ … which would have in fact fixed your subscription problem, but gave you a new problem :) Great share!

Heads up Bear [1] is just the blogging platform, not the company this post is about.

[1] https://bearblog.dev

Re: A ChatGPT mistake cost us $10k

#284

Post by chii » Mon, Jun 10, 2024, 4:13 AM UTC

sdwr wrote: post #281

kazinator wrote: post #257

I've written less than 1000 lines of Python in total probably, but I correctly spotted the problem. Python has this misfeature whereby it didn't correctly crib Common Lisp's evaluation strategy for the expressions that give default values to optional function arguments. When you have an argument like foo=obj.whatever() the obj.whatever() is evaluated (would you believe it!) at the time the definition of the function…

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

https://stackoverflow.com/questions/1132941/the-mutable-defa... and https://www.valentinog.com/blog/tirl-python-default-argument...

basically, having a default argument value in a function definition means to evaluate it during definition time of that function, not when the function is invoked.

This is a foot gun.

Re: A ChatGPT mistake cost us $10k

#285

Post by [deleted] » Mon, Jun 10, 2024, 4:13 AM UTC

crabmusket wrote: post #244

sanderjd wrote: post #175

Earlier quoted context omitted.

Yeah but a lot of that is just the accrual of improvements that is possible with a lot of resources over a long period of time. People working in "big tech" aren't fundamentally better at building reliable tools and systems; the time and resource constraints are entirely different.

And the stakes! This outage might have cost the OP $10k. A similar snafu at a larger company might have cost tens of millions or more.

[deleted]

Re: A ChatGPT mistake cost us $10k

#286

Post by winterrx » Mon, Jun 10, 2024, 4:14 AM UTC

steelbrain wrote: post #107

nicce wrote: post #94

Earlier quoted context omitted.

Probably the most known example https://github.com/vercel/next.js/discussions/19065 It is not an issue if you host in Vercel. Implementing the requested feature would make the framework much better and easier to use when self-hosted elsewhere. But there is neglection to resolve the issue. This is just one case.

FWIW, we host on Cloudflare and use their API to resize images on the fly and we're fine. Not so much a "lock-in" if other vendors can fill-in, is it?

What API do you use to resize images? Cloudflare? I know Vercel and NextJS have an resizing/optimization component that gets pricy.

Re: A ChatGPT mistake cost us $10k

#287

Post by paul7986 » Mon, Jun 10, 2024, 4:14 AM UTC

spencerchubb wrote: post #267

paul7986 wrote: post #237

Earlier quoted context omitted.

what it cant tell you the correct date.. how simple is that and yeah i use chatGPT in many different ways ... i was pointing out an example from today in which it was flat wrong for such a simple thing.

A large language model is trained on vast amounts of text to predict the next token. That tool will have no idea what the current date is, unless the developers augment it by telling it the current date.

Few weeks or more ago I discovered Burger King has a cotton candy slurpee/icee ... I've been enjoying one once to a few times a week. But not all locations have it yet chatGPT seemed to know which locations have it & each one it told me ..they had it available to buy. Very useful so today went to one closest to me which always has it but today they said we no longer do. Thus I asked chatGPT has been discontinued as well asked for its sources which it provide saying it's still available per information as of today. Then I asked it what is today it gave the wrong date ..two days in the past.

I'm surely not using it to ask for just the date yet still it should know such a simple thing as current date if asked..ppl expect it to like Siri and Alexa can tell u it..users expect the same UX and way better!

Re: A ChatGPT mistake cost us $10k

#288

Post by Dylan16807 » Mon, Jun 10, 2024, 4:15 AM UTC

whoknowsidont wrote: post #272

kfarr wrote: post #151

Earlier quoted context omitted.

Yeah ChatGPT is a red herring -- it doesn't matter what generates the code, it's what you do with it.

Surely current events explains why ChatGPT is topical?

You can have topical red herrings.

Re: A ChatGPT mistake cost us $10k

#289

Post by faizshah » Mon, Jun 10, 2024, 4:16 AM UTC

cmeacham98 wrote: post #15

I understand how the mistake was made, it seems relatively easy to slip by even when writing code without ChatGPT. But what I don't understand is how this wasn't caught after the first failure? Does this company not have any logging? Shouldn't the fact the backend is attempting to reuse UUIDs be immediately obvious from observing the error?

This sort of process failure is why there’s a Correction of Errors (postmortem) process at Amazon: https://aws.amazon.com/blogs/mt/why-you-should-develop-a-cor...

Specifically asking why did it take so long to detect and why did it take so long to diagnose is useful in these situations.

Re: A ChatGPT mistake cost us $10k

#290

Post by MrJohz » Mon, Jun 10, 2024, 4:16 AM UTC

sdwr wrote: post #281

kazinator wrote: post #257

I've written less than 1000 lines of Python in total probably, but I correctly spotted the problem. Python has this misfeature whereby it didn't correctly crib Common Lisp's evaluation strategy for the expressions that give default values to optional function arguments. When you have an argument like foo=obj.whatever() the obj.whatever() is evaluated (would you believe it!) at the time the definition of the function…

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

It is correct, it's one of the most surprising things about Python and it causes a number of mistakes, even for experts.

The easiest way to see this is by running something like this and seeing what gets printed out and when:

    print("1. start")
    
    def function(arg=print("2. func definition")):
        print("4. func call")
    
    print("3. after definition")

    function()
    function()
    function()

You should see that the print statement in the default position is called once, when the function definition itself is being evaluated, not when the function gets called.