A ChatGPT mistake cost us $10k

Post by fragmede » Mon, Jun 10, 2024, 4:26 AM UTC

MrJohz wrote: post #290

sdwr wrote: post #281

Earlier quoted context omitted.

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

It is correct, it's one of the most surprising things about Python and it causes a number of mistakes, even for experts. The easiest way to see this is by running something like this and seeing what gets printed out and when: print("1. start") def function(arg=print("2. func definition")): print("4. func call") print("3. after definition") function() function() function() You should see that the print statement in th…

    def func(arg=[]):
        arg.append(1)
        print(arg)
    func()
    func()
    func()

Post by TheDong » Mon, Jun 10, 2024, 4:29 AM UTC

sneak wrote: post #278

wavemode wrote: post #65

No, a lack of monitoring cost you $10K. Your app was throwing a database exception and nobody was alerted that this was not only happening, but happening continuously and in large volumes. Such an alert would have made this a 5-minute investigation rather than 5 days. If you haven't fixed that alerting deficiency, then you haven't really fixed anything.

TBH, if the backend were written in Go, this probably wouldn’t have happened to the extent it did. Somewhere in a log a descriptive error would have shown up. One of the reasons I use Go whenever possible is that it removes a lot of the classic Python footguns. If you are going to rewrite your backend from Javascript, why would you rewrite it in another untyped, error-prone language?

   tx, err := db.Tx()
   defer tx.Commit() // silently ignores the error on committing, which is the important one

Post by OptionOfT » Mon, Jun 10, 2024, 4:30 AM UTC

MrJohz wrote: post #290

sdwr wrote: post #281

Earlier quoted context omitted.

> When you have an argument like foo=obj.whatever(), the obj.whatever() is evaluated at the time the definition of the function is being processed, not at the time when the function is being called. This can't be correct, surely? What if .whatever() relies on internal state that changes after obj is initialized (or after the function surrounding foo is declared, not sure what you're saying)?

It is correct, it's one of the most surprising things about Python and it causes a number of mistakes, even for experts. The easiest way to see this is by running something like this and seeing what gets printed out and when: print("1. start") def function(arg=print("2. func definition")): print("4. func call") print("3. after definition") function() function() function() You should see that the print statement in th…

Post by dalemhurley » Mon, Jun 10, 2024, 4:30 AM UTC

8n4vidtmkvmk wrote: post #219

throwaway346434 wrote: post #142

Earlier quoted context omitted.

Guess: the logs were on an ec2 instance that was thrown away regularly, and the overnight reports didn't give reproduce steps or timestamps; so when they checked it "works fine". There's value in having your backtrace surfaced to end users rather than swallowing an exception and displaying "didn't work".

I don't think showing stack traces to users is good practice? Every time one of my users gets a didn't work message I log the stack trace instead.

Post by dalemhurley » Mon, Jun 10, 2024, 4:34 AM UTC

whoknowsidont wrote: post #272

kfarr wrote: post #151

Earlier quoted context omitted.

Yeah ChatGPT is a red herring -- it doesn't matter what generates the code, it's what you do with it.

Surely current events explains why ChatGPT is topical?