81–90 of 157 posts
Earlier quoted context omitted.
This part from the first try made me laugh: if random.random()
I think that’s the funniest joke I’ve ever seen an LLM make. Which probably means it’s copied from somewhere.
Earlier quoted context omitted.
This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…
LLMs operate at token level, not word. it doesn't operate in terms of "traumatic", "over-training", "over" or "training", but rather "tr" "aum" "at" "ic, ", etc.
Some LLMs can output nerd font glyphs and others can't.
If I recall grok code fast can but codex and sonnet can't
Earlier quoted context omitted.
This feels like RLVR, not RLHF. With RLVR, the LLM is trained to pursue "verified rewards." On coding tasks, the reward is usually something like the percentage of passing tests. Let's say you have some code that iterates over a set of files and does processing on them. The way a normal dev would write it, an exception in that code would crash the entire program. If you swallow and log the exception, however, you can…
> This is an easy way to get "number of files successfully processed" up, without actually making your code any better. Well, it depends a bit on what your goal is. Sometimes the user wants to eg backup as many files as possible from a failing hard drive, and doesn't want to fail the whole process just because one item is broken.
However, LLM generated code will often, at least in my experience, avoid raising any errors at all, in any case. This is undesirable, because some errors should result in a complete failure - for example, errors which are not transient or environment related but a bug. And in any case, a LLM will prefer turning these single file errors into warnings, though the way I see it, they are errors. They just don't need to abort the process, but errors nonetheless.
Earlier quoted context omitted.
This part from the first try made me laugh: if random.random()
I think that’s the funniest joke I’ve ever seen an LLM make. Which probably means it’s copied from somewhere.
This is a parody but the phenomenon is real. My uninformed suspicion is that this kind of defensive programming somehow improves performance during RLVR. Perhaps the model sometimes comes up with programs that are buggy enough to emit exceptions, but close enough to correct that they produce the right answer after swallowing the exceptions. So the model learns that swallowing exceptions sometimes improves its reward.…
These aren't operating on reward functions because there's no internal model to reward. It's word prediction, there's no intelligence.
I've noted that LLMs tend to produce defensive code to a fault. Lots of unnecessary checks, e.g. check for null/None/undefined multiple times for same valie. This can lead to really hard to read code, even for the LLM itself. The RL objectives probably heavily penalize exceptions, but don't reward much for code readability or simplicity.
It's so annoying.
LLMs often write tutorial-ish code without much care how it integrates with rest of codebase.
Swallowing exceptions is one such example.
Not sure why but it made me think of FizzBuzzEnterpriseEdition https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
Woah, were they using junit 4.8.3 in that project? Someone was flying by the seat of their pants, I hope they got sign-off on that by legal & the CTO, that’s the kind of cowboy coding choice that can hurt a career.
Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...