Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM Critics Are Right. I Use LLMs Anyway

theocharis.dev

Post reply on HN
Print view View original

181–190 of 330 posts

Re: The LLM Critics Are Right. I Use LLMs Anyway

#181

Post by westurner »

skydhash wrote: post #161
westurner wrote: post #116

Earlier quoted context omitted.

Like becoming a manager? When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax Managers and developers are responsible for setting a test coverage threshold for merge approval. If there is 100% branch coverage…

> When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. > Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax I can see managers getting involved into acceptance tests, but never in the other type of tests. And the verification mostly is involved into a quick m…

Given that it's so easy to generate better than test stubs with a model and manually review what it thinks covers, I think it's fair to expect a manager to run and add to the tests.

If devops has done their job, it should be trivial for a manager to contribute to the tests and run the build on git push (or manually re-run the build with the web UI).

If a manager has deploy rights, they should be able to run the tests.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#182

Post by bigstrat2003 »

qsort wrote: post #62
Diogenesian wrote: post #47

Earlier quoted context omitted.

I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents. Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt…

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

I have seen no productivity uplift at all from LLMs. They are at least about neutral these days (they used to be a productivity drain), but no gains at all. By the time I get done reviewing the code to make sure it hasn't done anything crazy, I've spent the same amount of time I would've taken to write the code myself. The only people I personally know who claim productivity gains are getting those gains by completely disregarding quality, or understanding the code, and just YOLO letting the LLM do everything without checking. I'm not willing to do that.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#183

Post by overgard »

My feeling is that LLMs are an interesting technology. I don't feel currently threatened by them, and they're handy in the hands of people that can use them with discipline. However, I think the people controlling and creating the LLMs are by and large scumbags who want to gamble with everyone's future. It's hard for me to look at their public statements and the actions of people like Altman, Amodei, Musk, and come away with any other conclusion than that these are fairly awful people that have bad intentions. Their message is essentially "we want to summon the machine god to destroy your livelihood because it will enrich us and give us power over the world, and there's a really high probability it gets out of control and causes massive problems [that we're completely going to not do anything about]. But we really like money! And we have bunkers!"

I suspect a lot of people are in this boat. We both use it and cheer against it. But it's not cognitive dissonance, it's just a recognition that there's a useful tool being controlled by vile people.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#184

Post by westurner »

coldtea wrote: post #171
tablarasa wrote: post #137

Earlier quoted context omitted.

> How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage? This is such a salient question. Sometimes (definitely not always) the test suites produced by LLMs are so trivial it's scary. Coverage can be an illusion for sure.

I have been seeing such trivial shit getting submitted and deleting them from the test suites, but it's disheartening to see that the devs didn't have the better taste to see those LLM "tests" are so trivial they border on noop before submitting them...

What new or existing metric would better assess test coverage than line coverage and branch coverage?

I wrote a tool called tert - I guess it's called an agent harness now - to run various test runners and log test output and coverage output to disk. FWIU stripping spaces from JSON does save tokens. It seems like feeding coverage lines-missing maps into the prompt results in better output, better LLM-authored tests.

"Refactor these tests for maintainability and coverage. Use fixtures, mocks, and parametrization"

Re: The LLM Critics Are Right. I Use LLMs Anyway

#185

Post by jdjdkfhbeb »

deadbabe wrote: post #145
qsort wrote: post #62

Earlier quoted context omitted.

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

Developers are cashing in on the productivity gains. Meaning instead of using the increased productivity to do more work, they just become lazier or do fun irrelevant side projects instead, where as before there was just not much time for such things. I have definitely procrastinated on work simply because I know I can swoop in with an LLM and do it all in 15 minutes, whereas before I would have spent a few hours. Th…

I would frame it differently ... Last year I was burning the fuck out hard. This year I am not.

My gain is that I produce good work in a normal workday and I am absolutely not writing code to 4 am anymore...

Overall for me it is a win for me and my company.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#186

Post by simonw »

grayhatter wrote: post #57
msdz wrote: post #2

> LLM’s amplify what you already have: opinions, structure, frameworks. So far, so agreeable, but… > If you have thoughts, they come out sharper and faster. I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles. Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thou…

> Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. [Excluding people who don’t want anything to do with LLMs out of moral principle, which curiously just like the overarching topic I also both respect and…

Peer-reviewed evidence of "productivity" is a pretty high bar to demand.

I don't know if I've ever seen a study that attempted to demonstrate productivity that was ironclad to the point that nobody could find any methodology holes to pick in it.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#187

Post by strken »

dawnerd wrote: post #138
breadzeppelin__ wrote: post #100

Earlier quoted context omitted.

I am both a career developer and experienced team manager. from first hand experience the 1.5x im getting from AI is not flowing down to my team / org because why would i output 50% more when the pay environment and leadership are already underwhelming. That additional 50% productivity goes completely to side projects built on my second computer between 9-5 tasks

That’s what I’ve heard from my dev team too. They’re using it to give themselves free time while still being on the clock, not to produce more output for the company. Roughly thinking about hours spent on projects I think have gone up per task, the opposite that should be happening.

I'm finding it almost impossible to fill that free time with work, unless it's just reading emails and chat messages.

I can context switch between two or three chats, but doing so speeds up my agent use at the cost of making reviews and discovery harder, so it might come out in the wash.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#188

Post by westurner »

BalinKing wrote: post #176
westurner wrote: post #116

Earlier quoted context omitted.

Like becoming a manager? When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax Managers and developers are responsible for setting a test coverage threshold for merge approval. If there is 100% branch coverage…

I've never really understood this argument. If someone's a manager of an incompetent team, no amount of management skill will save the quality of the resulting software. I don't think "just treat LLMs like smart junior developers" fixes this, because well-functioning teams usually also have senior developers to keep things on track. Like, if we handed a team of genius-but-junior developers to the best "people person"…

If you have a "this is an incompetent team" attitude, that's what quality software you'll get back; whether or not there's LLM use.

If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back.

There are people producing good and excellent quality software with LLMs. Presumably you must discard low-quality code in order to maintain quality.

There's certainly a limit to code quality with current models. On number of lines of code per unit of time, LLM tools certainly already win.

Can costly automated code review for PRs catch most of the problems before they're under consideration for merge?

For example, the vscode repo has extensive copilot integration. Every PR gets auto code reviewed. But with their tokens or the contributors'?

If I take poor quality code (AI-assisted or not) and spend a few hundred dollars on tokens for a next gen model and agent to get to 100% coverage and review for security bugs and CWE common weaknesses, what quality code will I have without refactoring with proven patterns and type annotations and polishing docstrings?

Re: The LLM Critics Are Right. I Use LLMs Anyway

#189

Post by bodge5000 »

qsort wrote: post #62
Diogenesian wrote: post #47

Earlier quoted context omitted.

I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents. Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt…

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

In the former case, which seems more likely, does it not fall to reason that like any tool there'll be some that will benefit from it and some that it just won't work as well for, but are equally as good at their job? Like that neovim, unless your an absolute zealot you wouldnt insist that everyone use it, or hire someone based off it, but those who do work well with it do find it to be a boost to productivity.

Also importantly, can it be a neovim? Neovim hasn't had literal trillions invested over it in the space of just 5 years, local llm's aside can those smaller productivity gains justify the huge investment put into them, and which will continue to be needed for further model development?

Re: The LLM Critics Are Right. I Use LLMs Anyway

#190

Post by archagon »

> I strongly agree that written text should be from humans to humans. Yet I still write all of my texts with LLMs.

Hypocritical garbage. Into the blacklist you go.

If you can’t even use your own words for basic communication then there’s absolutely no reason for me to take you seriously.

Post reply on HN