Live data from Hacker News

LLMs are mortally terrified of exceptions

twitter.com

111–120 of 157 posts

Re: LLMs are mortally terrified of exceptions

#111
post #93

I spent more time dismissing various popups than reading this post. I hate twitter links

Just add “cancel” after the x to get a viewable version of any Twitter link: https://xcancel.com/karpathy/status/1976082963382272334

Wow this is so much better user experience

Re: LLMs are mortally terrified of exceptions

#112
post #96

Earlier quoted context omitted.

If you're an extensive user of ChatGPT, or if you can give it some material about yourself like say, a resume or a LinkedIn profile, ask it to roast you. It will be very specific to the content you give it. Be warned, it can be brutal.

Periodic reminder that there’s also HN Wrapped. [0] [0]: https://hn-wrapped.kadoa.com

ooooh boy, gotta mentally prepare myself for this one

damn these ai's are good!

Re: LLMs are mortally terrified of exceptions

#113

If you are dividing two numbers with no prior knowledge of these numbers or any reasonable assumptions you can make and this code is used where you can not rely on the caller to catch an exception and the code is critical for the product, then this is necessary. If you are actually doing safety critical software, e.g. aerospace, medicine or automotive, then this is a good precaution, although you will not be writing…

[deleted]

Re: LLMs are mortally terrified of exceptions

#114
post #7

Turns out computer math is actually super hard. Basic operations entail all kinds of undefined behavior and such. This code is a bit verbose but otherwise familiar.

# Step 3: Preemptively check for catastrophic magnitude differences if abs(a) > sys.float_info.max / 2: logging.warning("Value of a might cause overflow. Returning infinity just to be sure") return math.copysign (float('inf'), a) if abs(b) Does the above code make any sense? I've not worked with this sort of stuff before, but it seems entirely unreasonable to me to check them individually. E.g. if 1 < b < a, then it…

It also uses float epsilon and I don't think I have ever seen code where using it was appropriate.

Re: LLMs are mortally terrified of exceptions

#115
post #55

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

This part from the first try made me laugh: if random.random()

I actually laughed when I read that. This one got me, too. The casual validation of its paranoia gives me Marvin the Paranoid Android vibes.

  try:
      result = a / b
      if math.isnan(result):
          raise ArithmeticError("Result is NaN. I knew this would happen.")

Re: LLMs are mortally terrified of exceptions

#117
post #48

Earlier quoted context omitted.

Why not just to use IEEE 754? > According to the IEEE 754 standard, floating-point division by zero is not an error but results in special values: positive infinity, negative infinity, or Not a Number (NaN). The specific result depends on the numerator

Because sometimes it's very wrong Way back when during my EE course days, we had like a whole semester devoted to weird edge cases like this, and spent month on ieee754 (precision loss, Nan, divide by zero, etc) When you took an ieee754 divide by zero value as gospel and put it in the context of a voltage divisor that is always negative or zero, getting a positive infinity value out of divide by zero was very wrong,…

But IEEE 754 works as you described in your last comment. It doesn't take the numerator's sign. So what's wrong?

Can you give more context on your voltage math? Was the numerator sometimes negative? If the problem is that your divisor calculation sometimes resulted in positive zero, that doesn't sound like the standard being wrong without more info.

Re: LLMs are mortally terrified of exceptions

#118

Earlier quoted context omitted.

Unchecked exceptions are more like a shutdown event, which can be intercepted at any point along the call stack, which is useful and not like a return type.

Why do you need the call stack at all?

Debugging. It's one of the most useful tools for narrowing down where an error is coming from and by far the biggest negative of Rust's Result-type error handling in my experience (panics can of course give a callstack but because of the value-based error being most commonly used this often is far away from the actual error).

(it is in principle possible to construct such a stack, potentially with more context, with a Result type, but I don't know of any way to do so that doesn't sacrifice a lot of performance because you're doing all the book-keeping even on caught errors where you don't use that information)

Re: LLMs are mortally terrified of exceptions

#119
post #56

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…

“Traumatic overtraining” does have hits though. My guess is that “traumatically” is a rarely used adverb, and “traumatic” is much more common. Possibly it completed traumatic into an adverb and then linked to overtraining which is in the training data. I dunno how these things work though.

Re: LLMs are mortally terrified of exceptions

#120

Earlier quoted context omitted.

Why do you need the call stack at all?

Debugging. It's one of the most useful tools for narrowing down where an error is coming from and by far the biggest negative of Rust's Result-type error handling in my experience (panics can of course give a callstack but because of the value-based error being most commonly used this often is far away from the actual error). (it is in principle possible to construct such a stack, potentially with more context, with…

Call Stack isn't a zero-cost abstraction, it makes threads more heavy-weight than they should be.

If you only need it for debugging, then maybe better instrumentation and observability is the answer.

Post reply on HN