Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are mortally terrified of exceptions

twitter.com

Post reply on HN
Print view View original

71–80 of 157 posts

Re: LLMs are mortally terrified of exceptions

#71

Post by justatdotin »

chis wrote: post #52
karpathy wrote: post #49

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

I think there’s always a danger of these foundational model companies doing RLHF on non-expert users, and this feels like a case of that. The AIs in general feel really focused on making the user happy - your example, and another one is how they love adding emojis to the stout and over-commenting simple code.

'over-commenting simple code' is preparing it for future agent work. pay attention to those comments to learn how you can better scaffold for agents.

Re: LLMs are mortally terrified of exceptions

#72

Post by exasperaited »

Why do LLMs do it for real: because you trained them by stealing all of Stack Overflow?

Less sarcastically but equally as true: they've learned from the tests you stole from people on the internet as well as the code you stole from people on the internet.

Most developers write tests for the wrong things, and many developers write tests that contain some bullshit edge case that they've been told to test (automatically to meet some coverage metric, or by a "senior" developer who got Dilbert principled away from the coalface and doesn't understand diminishing returns).

But then the end goal is to turn out code about as good as the average developer so they can be replaced more cheaply, so your LLM is meeting its objectives. Congrats.

Re: LLMs are mortally terrified of exceptions

#73

Post by magicalhippo »

jcelerier wrote: post #69
bjourne wrote: post #56

Earlier quoted context omitted.

This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…

LLMs operate at token level, not word. it doesn't operate in terms of "traumatic", "over-training", "over" or "training", but rather "tr" "aum" "at" "ic, ", etc.

> it doesn't operate in terms of "traumatic", "over-training", "over" or "training", but rather "tr" "aum" "at" "ic, ", etc.

And "毛片免费观看" (Free porn movies), "天天中彩票能" (Win the lottery every day), "热这里只有精品" (Hot, only fine products here) etc[1].

[1]: https://news.ycombinator.com/item?id=45483924

Re: LLMs are mortally terrified of exceptions

#74

Post by cruffle_duffle »

karpathy wrote: post #49

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

Kind of interesting it didn't add type hints though! You'd think for all that paranoia it would at least add type hints.

Re: LLMs are mortally terrified of exceptions

#75

Post by eru »

miki123211 wrote: post #68
chis wrote: post #52

Earlier quoted context omitted.

I think there’s always a danger of these foundational model companies doing RLHF on non-expert users, and this feels like a case of that. The AIs in general feel really focused on making the user happy - your example, and another one is how they love adding emojis to the stout and over-commenting simple code.

This feels like RLVR, not RLHF. With RLVR, the LLM is trained to pursue "verified rewards." On coding tasks, the reward is usually something like the percentage of passing tests. Let's say you have some code that iterates over a set of files and does processing on them. The way a normal dev would write it, an exception in that code would crash the entire program. If you swallow and log the exception, however, you can…

> This is an easy way to get "number of files successfully processed" up, without actually making your code any better.

Well, it depends a bit on what your goal is.

Sometimes the user wants to eg backup as many files as possible from a failing hard drive, and doesn't want to fail the whole process just because one item is broken.

Re: LLMs are mortally terrified of exceptions

#76

Post by bspammer »

why_at wrote: post #55
karpathy wrote: post #49

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

This part from the first try made me laugh: if random.random()

I think that’s the funniest joke I’ve ever seen an LLM make. Which probably means it’s copied from somewhere.

Re: LLMs are mortally terrified of exceptions

#77

Post by TZubiri »

jcelerier wrote: post #69
bjourne wrote: post #56

Earlier quoted context omitted.

This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…

LLMs operate at token level, not word. it doesn't operate in terms of "traumatic", "over-training", "over" or "training", but rather "tr" "aum" "at" "ic, ", etc.

I think you are confusing tokens with vectors/embedding/parameters.

king and rex (king in latin) map to different tokens but will map to very similar vectors.

Re: LLMs are mortally terrified of exceptions

#78

Post by TZubiri »

karpathy wrote: post #49

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

https://chatgpt.com/share/68e87072-e3ac-800f-a44c-af5666180a...

lgtm

Re: LLMs are mortally terrified of exceptions

#79

Post by dwd »

mwkaufma wrote: post #2

Even when they're not AI slop, these kinds of "paranoid sanity checks" are the software equivalent of security-theater.

Sometimes security theater is what you need to not trigger a false positive on a static code analysis.

I haven't needed to use a service like Fortinet recently and am now wondering if a LLM is part of their tool and if it's better/worse?

Re: LLMs are mortally terrified of exceptions

#80

Post by iagooar »

This issue has been one of the biggest issues with the Claude models, not so much with GPT-4 or GPT-5.

I even had this Cursor rule when I was using Claude:

"- Do not use statements to catch all possible errors to mask an error - let it crash, to see what happened and for easier debugging."

And even with this rule, Claude would not always adhere. Never had this issue with GPT-5.

Post reply on HN