101–110 of 230 posts
I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…
Earlier quoted context omitted.
To this day frontier models think that A and not B means A and B when the sentence gets pushed far enough back in their context window. The context length that model can reason over without obvious errors is much smaller than the advertised context. Between a 1/4th to a 1/20th what is advertised on the tin.
Do you also happen to remember what you ate last thrusday?
Earlier quoted context omitted.
It means having taste. People say Picasso was a great painter, but that cannot be verified (at least, not in the sense of a verified reward).
"people say picasso was a great painter" is definitely not hard to verify . lol.
"Write this code in a way that is readable and maintainable" is another example.
Earlier quoted context omitted.
Oldheads remember when GPT 5.2 was at the forefront of agentic programming. December 2025 feels like eons ago, but alack it was an entire half year!
If I'm not using got 5.5 high reasoning I'm wasting time.
Earlier quoted context omitted.
If it’s not easily verifiable, LLMs aren’t good at it.
but what does it mean to be good at something that cant be verified. how do you know that they are not good at it, you are obviously using some measure. sounds like an oxymoron of a claim.
Earlier quoted context omitted.
"people say picasso was a great painter" is definitely not hard to verify . lol.
I don't know if you're being factitious or not but that was not what I meant. Picasso being a great painter is an example of "having taste"; "create an artistic image generation model with Picasso-level performance" is a valid problem statement we could attack with RLHF, but not with RLVR, because "taste" is not amenable to modeling with a reward function. "Write this code in a way that is readable and maintainable"…
Earlier quoted context omitted.
but what does it mean to be good at something that cant be verified. how do you know that they are not good at it, you are obviously using some measure. sounds like an oxymoron of a claim.
You just threw the "easily" away from the comment you are replying.
I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet. In my experience, for anything complex enough, I have to start adding more and more constraints,…