Live data from Hacker News

LLMs are mortally terrified of exceptions

twitter.com

41–50 of 157 posts

Re: LLMs are mortally terrified of exceptions

#41

Earlier quoted context omitted.

So you think java's checked exceptions are a better model? No opinion myself, but that way seems widely considered bad too.

Why do you need exceptions at all? They’re just a different return types in disguise… Also, division by zero should return Inf

What about division of zero by zero?

Re: LLMs are mortally terrified of exceptions

#42

Earlier quoted context omitted.

So you think java's checked exceptions are a better model? No opinion myself, but that way seems widely considered bad too.

Why do you need exceptions at all? They’re just a different return types in disguise… Also, division by zero should return Inf

> division by zero should return Inf

Sometimes yes, sometimes no?

It's a domain specific answer, even ignoring the 0/0 case.

And also even ignoring the "which side of the limit are you coming from?" where "a" and/or "b" might be negative. (Is it positive infinity or negative infinity? The sign of "a" alone doesn't tell you the answer)

Because sometimes the question is like "how many things per box if there's N boxes"? Your answer isn't infinity, it's an invalid answer altogether.

The limit of 1/x or -1/x might be infinity (or negative infinity), and in some cases that might be what you want. But sometimes it's not.

Re: LLMs are mortally terrified of exceptions

#43

Earlier quoted context omitted.

Or -Inf, depending on the sign of the zero, which might catch some programmers by surprise, but is of course the correct thing to do.

a/0 = Inf when a>0 a/0 = -Inf when a

No this doesn't work either

In the context of say a/-0.001, a/-0.00000001, a/-0.0000000001, a/, a/0

Then a/0 is negative when a>0, and positive when a<0

Re: LLMs are mortally terrified of exceptions

#44
post #39

That code has many issues, but the one that bothers me the most in practice is this tendency of adding imports inside functions. I can only assume that it's an artifact of them optimizing for a minimal number of edits somewhere in the process, but I expect better.

It's to make imports lazy, to solve the issue of slow import at startup.

While there are some cases where lazy imports are appropriate, this function, and the vast majority of such lazy imports that I get from Claude are not.

In particular, I can't think of any non-pathological situation where a python developer should import logging and update logging.basicConfig within an inner function.

Re: LLMs are mortally terrified of exceptions

#45
post #43

Earlier quoted context omitted.

a/0 = Inf when a>0 a/0 = -Inf when a

No this doesn't work either In the context of say a/-0.001, a/-0.00000001, a/-0.0000000001, a/ , a/0 Then a/0 is negative when a>0, and positive when a<0

Why not just to use IEEE 754?

> According to the IEEE 754 standard, floating-point division by zero is not an error but results in special values: positive infinity, negative infinity, or Not a Number (NaN). The specific result depends on the numerator

Re: LLMs are mortally terrified of exceptions

#46
This is a parody but the phenomenon is real.

My uninformed suspicion is that this kind of defensive programming somehow improves performance during RLVR. Perhaps the model sometimes comes up with programs that are buggy enough to emit exceptions, but close enough to correct that they produce the right answer after swallowing the exceptions. So the model learns that swallowing exceptions sometimes improves its reward. It also learns that swallowing exceptions rarely reduces its reward, because if the model does come up with fully correct code, that code usually won’t raise exceptions in the first place (at least not in the test cases it’s being judged on), so adding exception swallowing won’t fail the tests even if it’s theoretically incorrect.

Again, this is pure speculation. Even if I’m right, I’m sure another part of the reason is just that the training set contains a lot of code written by human beginners, who also like to ignore errors.

Re: LLMs are mortally terrified of exceptions

#48
post #43

Earlier quoted context omitted.

No this doesn't work either In the context of say a/-0.001, a/-0.00000001, a/-0.0000000001, a/ , a/0 Then a/0 is negative when a>0, and positive when a<0

Why not just to use IEEE 754? > According to the IEEE 754 standard, floating-point division by zero is not an error but results in special values: positive infinity, negative infinity, or Not a Number (NaN). The specific result depends on the numerator

Because sometimes it's very wrong

Way back when during my EE course days, we had like a whole semester devoted to weird edge cases like this, and spent month on ieee754 (precision loss, Nan, divide by zero, etc)

When you took an ieee754 divide by zero value as gospel and put it in the context of a voltage divisor that is always negative or zero, getting a positive infinity value out of divide by zero was very wrong, in the sense of "flip the switch and oh shit there's the magic smoke". The solution was a custom divide function that would know the context, and yield negative infinity (or some placeholder value). It was a contrived example for EE lab, but the lesson was - sometimes the standard is wrong and you will cause problems if it's blindly followed.

Sometimes it's fine, but it depends on the domain

Re: LLMs are mortally terrified of exceptions

#50

Not sure why but it made me think of FizzBuzzEnterpriseEdition https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...

Woah, were they using junit 4.8.3 in that project? Someone was flying by the seat of their pants, I hope they got sign-off on that by legal & the CTO, that’s the kind of cowboy coding choice that can hurt a career.
Post reply on HN