Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are LLM merge rates not getting better?

entropicthoughts.com

Post reply on HN
Print view View original

141–150 of 175 posts

Re: Are LLM merge rates not getting better?

#141

Post by aix1 »

cicko wrote: post #136
jygg4 wrote: post #78

Earlier quoted context omitted.

As long as humans are needed to review code, it sounds your role evolves toward prompting and reviewing. Which is akin to driving a car - the motor vehicle itself doesn’t know where to go. It requires you to prompt via steering and braking etc, and then to review what is happening in response. That’s not necessarily a bad thing - reviewing code ultimately matters most. As long as what is produced is more often than n…

I don't think that reviewing code is so important as reviewing results. Nobody is reviewing the IL or assembly code when they write in higher level languages. It's the end result that matters in most cases.

But we don't evolve IL or assembly code as the system evolves. We regenerate it from scratch every time.

It is therefore not important whether some intermediate version of that low-level code was completely impossible to understand.

It is not so with LLM-written high-level code. More often than not, it does need to be understood and maintained by someone or something.

These days, I mainly focus on two things in LLM code reviews:

1. Making sure unit tests have good coverage of expected behaviours.

2. Making sure the model is making sound architectural decisions, to avoid accumulating tech debt that'll need to be paid back later. It's very hard to check this with unit tests.

Re: Are LLM merge rates not getting better?

#142

Post by 0x3f »

jmalicki wrote: post #111
lich_king wrote: post #109

Earlier quoted context omitted.

You realize that the reason you need to manage this organizational complexity is largely because the organization is so huge?... The reality is that you could run LinkedIn with far, far fewer people. You probably need fewer than 100 for core engineering, and likely less than 1,000 overall if you include compliance, sales, and so on - especially since a lot of overseas compliance stuff is outsourced to consulting firm…

When doubling the number of employees can triple your revenue, you do it. Keeping a website running with high uptime is not the goal. Maximizing revenue and profit is. The extra people aren't waste, they're what drive the incremental imperceptible changes that make these companies profitable.

This seems like a just-so story.

Re: Are LLM merge rates not getting better?

#144

Post by 0x3f »

ahtihn wrote: post #137
lich_king wrote: post #109

Earlier quoted context omitted.

You realize that the reason you need to manage this organizational complexity is largely because the organization is so huge?... The reality is that you could run LinkedIn with far, far fewer people. You probably need fewer than 100 for core engineering, and likely less than 1,000 overall if you include compliance, sales, and so on - especially since a lot of overseas compliance stuff is outsourced to consulting firm…

And how much revenue did that company bring in compared to something like Meta? Maybe there's a correlation there?

I think the person you're replying to is perfectly aware of the correlation, considering it was a primary feature of their comment.

Re: Are LLM merge rates not getting better?

#145

Post by BoppreH »

globular-toast wrote: post #134
BoppreH wrote: post #71

Controversial opinion from a casual user, but state-of-art LLMs now feel to me more intelligent then the average person on the steet. Also explains why training on more average-quality data (if there's any left) is not making improvements. But LLMs are hamstrung by their harnesses. They are doing the equivalent of providing technical support via phone call: little to no context, and limited to a bidirectional stream…

It's so disrespectful to say an LLM is more intelligent than a person on the street. The LLM has nothing at stake, cares not a sausage about the consequences of what it spits out. People have all kinds of pressures, dependants, and personal issues like health. Our thoughts and actions have real consequences. It's so easy to be intelligent when you're the pretend human that gets switched on for five minutes then switc…

It's not a value judgement, I'm no misanthrope. But it's a fact or life that we humans must specialize, while LLMs can afford to have "studied" a staggering variety of topics. It's no different than being slower than a car, or weaker than a hydraulic press.

On a different note, LLMs are still not very wise, as displayed by all the prompt attacks and occasional inane responses like walking to the car wash.

Re: Are LLM merge rates not getting better?

#146

Post by globular-toast »

BoppreH wrote: post #145
globular-toast wrote: post #134

Earlier quoted context omitted.

It's so disrespectful to say an LLM is more intelligent than a person on the street. The LLM has nothing at stake, cares not a sausage about the consequences of what it spits out. People have all kinds of pressures, dependants, and personal issues like health. Our thoughts and actions have real consequences. It's so easy to be intelligent when you're the pretend human that gets switched on for five minutes then switc…

It's not a value judgement, I'm no misanthrope. But it's a fact or life that we humans must specialize, while LLMs can afford to have "studied" a staggering variety of topics. It's no different than being slower than a car, or weaker than a hydraulic press. On a different note, LLMs are still not very wise , as displayed by all the prompt attacks and occasional inane responses like walking to the car wash.

On the contrary, it's the machines that have the luxury of specialisation.

I'm faster than a car when scrambling up a mountain. It's easy to be fast when all you do is drive on smooth roads.

I'm stronger than a hydraulic press when carrying a load over a distance. It's easy to be strong when you're limited to a few centimetres of movement.

Intelligence of LLMs is a trick; they are literally trained to sound intelligent. But it's easy to sound intelligent when all you have to do is sound intelligent. A person has to live in the real world, deal with the weather, with feelings, stress, health and, above all, consequences. We don't have the luxury of just being able to sound intelligent, or have wheels that only work on smooth roads. We have to actually be intelligent in our actions. We have to traverse difficult ground, deal with obstacles we've never encountered before. When we make mistakes, people get hurt.

Re: Are LLM merge rates not getting better?

#147

Post by BoppreH »

globular-toast wrote: post #146
BoppreH wrote: post #145

Earlier quoted context omitted.

It's not a value judgement, I'm no misanthrope. But it's a fact or life that we humans must specialize, while LLMs can afford to have "studied" a staggering variety of topics. It's no different than being slower than a car, or weaker than a hydraulic press. On a different note, LLMs are still not very wise , as displayed by all the prompt attacks and occasional inane responses like walking to the car wash.

On the contrary, it's the machines that have the luxury of specialisation. I'm faster than a car when scrambling up a mountain. It's easy to be fast when all you do is drive on smooth roads. I'm stronger than a hydraulic press when carrying a load over a distance. It's easy to be strong when you're limited to a few centimetres of movement. Intelligence of LLMs is a trick; they are literally trained to sound intellige…

We're probably thinking about this at very different levels. Here's what I meant: I can ask Claude for "a bilingual German-Russian poem about the side effects of the most common drugs used in anesthesia". I would bet my left shoe that if I asked people on the street, no one will do a better job than Claude. And to me, answering questions correctly is a very good metric for intelligence.

We can debate whether that's real intelligence, and whether the question is fair, but this is still a real, measurable capability, that just eight years ago was a pipe dream. This capability is what OP is tracking, and what I believe is impressive but hamstrung by harnesses.

Re: Are LLM merge rates not getting better?

#148

Post by leoedin »

aerhardt wrote: post #61

I feel that two things are true at the same time: 1) Something happened during 2025 that made the models (or crucially, the wrapping terminal-based apps like Claude Code or Codex) much better. I only type in the terminal anymore. 2) The quality of the code is still quite often terrible. Quadruple-nested control flow abounds. Software architecture in rather small scopes is unsound. People say AI is “good at front end”…

This matches my experience too. The models write code that would never pass a review normally. Mega functions, "copy and pasted" code with small changes, deep nested conditionals and loops. All the stuff we've spent a lot of time trying to minimise!

You could argue it's OK because a model can always fix it later. But the problem comes when there's subtle logic bugs and its basically impossible to understand. Or fixing the bug in one place doesn't fix it in the 10 other places almost the same code exists.

I strongly suspect that LLMs, like all technologies, are going to follow an S curve of capability. The question is where in that S curve we are right now.

Re: Are LLM merge rates not getting better?

#149

Post by zx8080 »

aerhardt wrote: post #61

I feel that two things are true at the same time: 1) Something happened during 2025 that made the models (or crucially, the wrapping terminal-based apps like Claude Code or Codex) much better. I only type in the terminal anymore. 2) The quality of the code is still quite often terrible. Quadruple-nested control flow abounds. Software architecture in rather small scopes is unsound. People say AI is “good at front end”…

> People say AI is “good at front end” but I see the worst kind of atrocities there

It's commonly universal to say "AI is great in X", where one is not professional in X. It's because that's how AI is designed: to output tokens according to stats, not logic, not semantic, and not meaning: stats.

Re: Are LLM merge rates not getting better?

#150

Post by polyterative »

davecoffin wrote: post #23

I've been able to supercharge a hobby project of mine over the last couple months using Opus 4.6 in claude code. I had to collaborate and write code still, but claude did like 75% of the work to add meaningful new features to an iOS/Android native mobile app, including Live Activities which is so overly complicated i would not have been able to figure that out. I have it running in a folder that contains both my back…

I have a similar experience. My hobby project was put on hold after a burnout and lack of motivation. I got a big burst of energy back when I started implementing some long desired features quickly with these new models. I was able to get the project to the point of what I consider is maturity. I did in a month during free time the kind of work that would have burned me up in a good six months fulltime.
Post reply on HN