LLMs are mortally terrified of exceptions • Hacker News Board

Re: LLMs are mortally terrified of exceptions

#11

Post by simonw » Thu, Oct 09, 2025, 7:36 PM UTC

mwkaufma wrote: post #2

Even when they're not AI slop, these kinds of "paranoid sanity checks" are the software equivalent of security-theater.

Yeah, I really hate code like this because it generally ends up full of codepaths that have never been exercised, so there's all sorts of potential for weird behavior and unexpected edge cases. Plus it's harder to review.

Re: LLMs are mortally terrified of exceptions

#12

Post by mewpmewp2 » Thu, Oct 09, 2025, 7:37 PM UTC

constantcrying wrote: post #10

If you are dividing two numbers with no prior knowledge of these numbers or any reasonable assumptions you can make and this code is used where you can not rely on the caller to catch an exception and the code is critical for the product, then this is necessary. If you are actually doing safety critical software, e.g. aerospace, medicine or automotive, then this is a good precaution, although you will not be writing…

I might agree with that, and maybe the example posted by Karpathy is not the greatest, but what I'm constantly being faced with is try catches where it will fail silently or return a fallback/mock response, which essentially means that system will behave unexpectedly in a more subtle way down the line while leaving you clueless to as what the issue was.

I have to constantly remind Claude that we want to fail fast.

Re: LLMs are mortally terrified of exceptions

#13

Post by metalcrow » Thu, Oct 09, 2025, 7:39 PM UTC

Given that the output describes the function as being done "with extraordinary caution, because you never know what can go wrong", i would guess that the undisclosed prompt was something similar to "generate a division function in python that handles all possible edges cases. be extremely careful". Which seems to say less about LLM training and more about them doing exactly what they are told.

Re: LLMs are mortally terrified of exceptions

#14

Post by sarchertech » Thu, Oct 09, 2025, 7:46 PM UTC

Expert beginners program like this. I call it what it driven development. Turns out a lot of code was written by expert beginners because by many metrics they are prolifically productive.

In go all SOTA agents are obsessed with being ludicrously defensive against concurrency bugs. Probably because in addition to what if driven development, there are a lot of blog posts warning about concurrency bugs.

Re: LLMs are mortally terrified of exceptions

#15

Post by bobogei81123 » Thu, Oct 09, 2025, 7:46 PM UTC

This is just AI trying to tell us how bad we designed our programming languages to be when exceptions can be thrown pretty much anywhere

Re: LLMs are mortally terrified of exceptions

#16

Post by Den_VR » Thu, Oct 09, 2025, 7:47 PM UTC

wffurr wrote: post #7

Turns out computer math is actually super hard. Basic operations entail all kinds of undefined behavior and such. This code is a bit verbose but otherwise familiar.

If we wanted defined behavior we’d build systems with Karnaugh maps all the way down.

Re: LLMs are mortally terrified of exceptions

#17

Post by criemen » Thu, Oct 09, 2025, 7:54 PM UTC

It's also logically incoherent - division by zero can't occur, because if b=0 then abs(b) Furthermore, the code is happy to return NaN from the pre-checks, but replaces a NaN result from the division by None. That doesn't make any sense from an API design standpoint.

Re: LLMs are mortally terrified of exceptions

#18

Post by dijksterhuis » Thu, Oct 09, 2025, 7:57 PM UTC

I mean, the first three cases are just attempting to turn dynamic into static typed... right? maybe just don't aim for uber-safety in a dynamically typed language? :shrugs:

(I used to look out for kaparthy's papers ten years ago... i tend to let out an audible sigh when i see his name today)

Re: LLMs are mortally terrified of exceptions

#19

Post by recursive » Thu, Oct 09, 2025, 7:58 PM UTC

bobogei81123 wrote: post #15

This is just AI trying to tell us how bad we designed our programming languages to be when exceptions can be thrown pretty much anywhere

So you think java's checked exceptions are a better model? No opinion myself, but that way seems widely considered bad too.

Re: LLMs are mortally terrified of exceptions

#20

Post by [deleted] » Thu, Oct 09, 2025, 7:59 PM UTC

metalcrow wrote: post #13

Given that the output describes the function as being done "with extraordinary caution, because you never know what can go wrong", i would guess that the undisclosed prompt was something similar to "generate a division function in python that handles all possible edges cases. be extremely careful". Which seems to say less about LLM training and more about them doing exactly what they are told.

[deleted]