Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are mortally terrified of exceptions

twitter.com

Post reply on HN
Print view View original

141–150 of 157 posts

Re: LLMs are mortally terrified of exceptions

#141

Post by nivertech »

CupricTea wrote: post #135
nivertech wrote: post #27

Earlier quoted context omitted.

Why do you need exceptions at all? They’re just a different return types in disguise… Also, division by zero should return Inf

Division by zero is mathematically undefined. So two's complement integer division by zero is always undefined. For floating point there is the interesting property that 0 is signed due to its signed magnitude representation. Mathematically 0 is not signed but in floating point signed magnitude representation, "+0" is equivalent to lim x->0+ x and "-0" is equivalent to lim x->0- x. This is the only situation where a…

OK, but I think it's not up to the programming language designers to define mathematical properties of the operations on specific data types.

I think the most pragmatic solution is to have 2 tiers:

1. use existing standards (i.e. IEEE 754 for FP, de-facto standards for integers, like two's complement, Big-Endian, etc.)

2. fast, native format per each compute device, using different sub-types so you will not be able to mix them in the same expression

Re: LLMs are mortally terrified of exceptions

#142

Post by Dylan16807 »

dmoy wrote: post #133
Dylan16807 wrote: post #117

Earlier quoted context omitted.

But IEEE 754 works as you described in your last comment. It doesn't take the numerator's sign. So what's wrong? Can you give more context on your voltage math? Was the numerator sometimes negative? If the problem is that your divisor calculation sometimes resulted in positive zero, that doesn't sound like the standard being wrong without more info.

> But IEEE 754 works as you described in your last comment. It doesn't take the numerator's sign. So what's wrong? The numerator was always positive. The denominator was always negative (negative voltage is a pretty common thing), except when it became zero. That led to surprising behavior. Right the whole point of the exercise was that sometimes the standard is wrong for your specific problem at hand. We spent lectu…

It's a good lesson in defensiveness! But if your value was always either less than zero or negative zero it would have done the right thing, both domain correct and standard correct. It's hard to say exactly why you got positive zero, but my bet is that it's more subtle than the standard doing something you can actually call "wrong".

Re: LLMs are mortally terrified of exceptions

#143

Post by rcxdude »

nivertech wrote: post #120
rcxdude wrote: post #118

Earlier quoted context omitted.

Debugging. It's one of the most useful tools for narrowing down where an error is coming from and by far the biggest negative of Rust's Result-type error handling in my experience (panics can of course give a callstack but because of the value-based error being most commonly used this often is far away from the actual error). (it is in principle possible to construct such a stack, potentially with more context, with…

Call Stack isn't a zero-cost abstraction, it makes threads more heavy-weight than they should be. If you only need it for debugging, then maybe better instrumentation and observability is the answer.

The instrumentation and observability are more heavyweight than the overhead of unwinding the stack which is already keeping track of the most important information (in most mainstream langauges, at least. And even if you don't have a contiguous stack there's usually still the same information around at the point an error is created, assuming that you have something like functions that are returning into other functions. Exceptions, as a model, basically allow the code that raises an error to determine where the error is going to be caught without unwinding and removing the information that lets you track from the top level to where the error was raised). It is still tradeoff, of course (returning errors is more expensive than success), but it's one in a much better place in practice than other options (as obvious by the fact that errors-as-values implementations rarely keep this information around, especially not by default)

Re: LLMs are mortally terrified of exceptions

#144

Post by freedomben »

keeda wrote: post #96
bspammer wrote: post #76

Earlier quoted context omitted.

I think that’s the funniest joke I’ve ever seen an LLM make. Which probably means it’s copied from somewhere.

If you're an extensive user of ChatGPT, or if you can give it some material about yourself like say, a resume or a LinkedIn profile, ask it to roast you. It will be very specific to the content you give it. Be warned, it can be brutal.

Whoa dude! It was brutal, but highly constructive! Actually extremely helpful (and quite funny, though I have a high sense of humor about things so others might not appreciate some of it :-D)

This was my favorite line after asking it to review my resume and roast me:

> Structure & Flow: “Like Kubernetes YAML — powerful, but not human-readable.”

Some other good ones:

> Content & Tone: “You’re a CTO — stop talking like a sysadmin with a thesaurus.”

> Overall Impression: “This resume is a technical symphony… that goes on for too many movements.”

I've got some resume work to do haha

Re: LLMs are mortally terrified of exceptions

#145

Post by bccdee »

bjourne wrote: post #56
karpathy wrote: post #49

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...

This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…

I don't know. It's a classic LLM-ism. "Traumatically over-X" is probably a common enough phrase. The prmpt says, "I don't know what labs are doing to these poor LLMs during RL," so the model connects that to some form of trauma. The training is traumatic, so the model is traumatically over-trained.

It sounds fine and flows nicely, but it doesn't quite make sense. Too much training over-fits an LLM; that's not what we're describing. Bad training might traumatize a model, but bad how? A creative response would suggest an answer to that question—perhaps the model has been made paranoid, scarred by repeat exposure to the subtlest and most severe bugs ever discovered—but the LLM isn't being creative. Its response has that spongy, plastic LLM texture that comes from the model rephrasing its prompt to provide a sycophantic preamble for the thing that was actually being asked for. It uses new words for the same old idea, and a bit of the precision is lost during the translation.

Re: LLMs are mortally terrified of exceptions

#146

Post by aoeusnth1 »

MakeAJiraTicket wrote: post #86
comex wrote: post #46

This is a parody but the phenomenon is real. My uninformed suspicion is that this kind of defensive programming somehow improves performance during RLVR. Perhaps the model sometimes comes up with programs that are buggy enough to emit exceptions, but close enough to correct that they produce the right answer after swallowing the exceptions. So the model learns that swallowing exceptions sometimes improves its reward.…

Defensive programming is considered "correct" by the people doing the reinforcing, and is a huge part of the corpus that LLM's are trained on. For example, most python code doesn't do manual index management, so when it sees manual index management it is much more likely to freak out and hallucinate a bug. It will randomly promote "silent failure" even when a "silent failure" results in things like infinite loops, be…

You used the word reinforcing, and then asserted there's no reward function. Can you explain how it's possible to perform RL without a reward function, and how the LLM training process maps to that?

Re: LLMs are mortally terrified of exceptions

#147

Post by cma »

cma wrote: post #57
chis wrote: post #52

Earlier quoted context omitted.

I think there’s always a danger of these foundational model companies doing RLHF on non-expert users, and this feels like a case of that. The AIs in general feel really focused on making the user happy - your example, and another one is how they love adding emojis to the stout and over-commenting simple code.

And more advanced users are more likely to opt out of training on their data, Google gets around it with a free api period where you can't opt out and I think from did some of that too, through partnerships with tool companies, but not sure if you can ever opt out there.

*grok, not 'from'

Re: LLMs are mortally terrified of exceptions

#148

Post by john01dav »

Hackbraten wrote: post #109
keeda wrote: post #96

Earlier quoted context omitted.

If you're an extensive user of ChatGPT, or if you can give it some material about yourself like say, a resume or a LinkedIn profile, ask it to roast you. It will be very specific to the content you give it. Be warned, it can be brutal.

Periodic reminder that there’s also HN Wrapped. [0] [0]: https://hn-wrapped.kadoa.com

Absolutely hilarious, and gives me some self awareness tbh

Re: LLMs are mortally terrified of exceptions

#149

Post by bjourne »

bccdee wrote: post #145
bjourne wrote: post #56

Earlier quoted context omitted.

This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…

I don't know. It's a classic LLM-ism. "Traumatically over-X" is probably a common enough phrase. The prmpt says, "I don't know what labs are doing to these poor LLMs during RL," so the model connects that to some form of trauma. The training is traumatic, so the model is traumatically over-trained. It sounds fine and flows nicely, but it doesn't quite make sense. Too much training over-fits an LLM; that's not what we…

Eh, you are rationalizing. The phrase "traumatically over-X" is extremely rare. Any problem is easy after you've seen the solution. :) The solution "traumatically over-trained LLM" to the problem "What description best fits karpathy's description?" is certainly not easy to find. Connecting RL, poor LLMs, extreme fear, and welfare to excess training and severe lasting emotional pain is pretty darn impressive. E.g., I know exactly what situation karpathy describes is, but I couldn't in a million years put it into writing as succinctly and as precisely as the LLM.

Re: LLMs are mortally terrified of exceptions

#150

Post by chipsrafferty »

freedomben wrote: post #144
keeda wrote: post #96

Earlier quoted context omitted.

If you're an extensive user of ChatGPT, or if you can give it some material about yourself like say, a resume or a LinkedIn profile, ask it to roast you. It will be very specific to the content you give it. Be warned, it can be brutal.

Whoa dude! It was brutal, but highly constructive! Actually extremely helpful (and quite funny, though I have a high sense of humor about things so others might not appreciate some of it :-D) This was my favorite line after asking it to review my resume and roast me: > Structure & Flow: “Like Kubernetes YAML — powerful, but not human-readable.” Some other good ones: > Content & Tone: “You’re a CTO — stop talking like…

They meant roast you, not your resume.
Post reply on HN