91–100 of 157 posts
That code has many issues, but the one that bothers me the most in practice is this tendency of adding imports inside functions. I can only assume that it's an artifact of them optimizing for a minimal number of edits somewhere in the process, but I expect better.
It's to make imports lazy, to solve the issue of slow import at startup.
Earlier quoted context omitted.
I think that’s the funniest joke I’ve ever seen an LLM make. Which probably means it’s copied from somewhere.
"Why is a laser beam like goldfish? Because neither one can whistle." - Mike, The Moon is a Harsh Mistress
I spent more time dismissing various popups than reading this post. I hate twitter links
Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7a28-8007-9a99-bc6f0010d1...
This is stunning English: "Perfect setup for satire. Here’s a Python function that fully commits to the bit — a traumatically over-trained LLM trying to divide numbers while avoiding any conceivable danger:" "Traumatically over-trained", while scoring zero google hits, is an amazingly good description. How can it intuitively know what "traumatic over-training" should mean for LLMs without ever having been taught the…
Because, and this is a hot take, LLMs have emergent intelligence
Earlier quoted context omitted.
This part from the first try made me laugh: if random.random()
I think that’s the funniest joke I’ve ever seen an LLM make. Which probably means it’s copied from somewhere.
This is a parody but the phenomenon is real. My uninformed suspicion is that this kind of defensive programming somehow improves performance during RLVR. Perhaps the model sometimes comes up with programs that are buggy enough to emit exceptions, but close enough to correct that they produce the right answer after swallowing the exceptions. So the model learns that swallowing exceptions sometimes improves its reward.…
Defensive programming is considered "correct" by the people doing the reinforcing, and is a huge part of the corpus that LLM's are trained on. For example, most python code doesn't do manual index management, so when it sees manual index management it is much more likely to freak out and hallucinate a bug. It will randomly promote "silent failure" even when a "silent failure" results in things like infinite loops, be…
Earlier quoted context omitted.
Why do you need exceptions at all? They’re just a different return types in disguise… Also, division by zero should return Inf
> Why do you need exceptions at all? They’re just a different return types in disguise… You don’t need exceptions, and they can be replaced by more intricate return types. OTOH, for the intended use case for signalling conditions that most code directly calling a function does not expect and cannot do anything about, unchecked exceptions reduce code clutter (checked exceptions are isomorphic to "more intricate return…
Most problems stem from poor PL semantics[1] and badly designed stdlibs/APIs.
For exogenous errors, Let It Crash, and let the layer above deal with it, i.e., Erlang/OTP-style.
For endogenous errors, simply use control flow based on return values/types (or algebraic type systems with exhaustive type checking). For simple cases, something like Railway Oriented Programming.
---
1. division by zero in Julia:
julia> 1 / 0
Inf
julia> 0 / 0
NaN
julia> -1 / 0
-InfOne reason for this is that you typically lack a type system that allows 'making illegal states unrepresentable' to some extent, or possibly lack a team that can leverage the available type system to that effect due to organisational pressure, insufficient experience or whatever.
Earlier quoted context omitted.
> This is an easy way to get "number of files successfully processed" up, without actually making your code any better. Well, it depends a bit on what your goal is. Sometimes the user wants to eg backup as many files as possible from a failing hard drive, and doesn't want to fail the whole process just because one item is broken.
You're right, but the way to achieve this is to allow the error to propagate at the file level, then catch it one function above and continue to the next one. However, LLM generated code will often, at least in my experience, avoid raising any errors at all, in any case. This is undesirable, because some errors should result in a complete failure - for example, errors which are not transient or environment related bu…
> And in any case, a LLM will prefer turning these single file errors into warnings, though the way I see it, they are errors.
Well, in general they are something that the caller should have opportunity to deal with.
In some cases, aborting back to the caller at the first problem is the best course of action. In some other cases, going forward and taking note of the problems is best.
In some systems, you might event want to tell the caller about failures (and successes) as they occur, instead of waiting until the end.
It's all very similar to the different options people have available when their boss sends them on an errand and something goes wrong. A good underling uses their best judgement to pick the right way to cope with problems; but computer programs don't have that, so we need to be explicit.
See https://en.wikipedia.org/wiki/Mission-type_tactics for a related concept in the military.