Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM Critics Are Right. I Use LLMs Anyway

theocharis.dev

Post reply on HN
Print view View original

111–120 of 330 posts

Re: The LLM Critics Are Right. I Use LLMs Anyway

#111

Post by deaton »

Diogenesian wrote: post #47
msdz wrote: post #2

> LLM’s amplify what you already have: opinions, structure, frameworks. So far, so agreeable, but… > If you have thoughts, they come out sharper and faster. I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles. Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thou…

I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents. Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt…

I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#112

Post by skydhash »

qsort wrote: post #62
Diogenesian wrote: post #47

Earlier quoted context omitted.

I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents. Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt…

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

> LLMs give no productivity uplift at all, which is much harder to defend

It’s not really hard to defend. Because when people says that productivity is uplifted, they are talking about amount of work, not the ROI. That’s why you keep hearing about LOC, amount of PR and prototypes, and the time taken is actually “time to PR” and not “time to production + time spent on bugs”.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#113

Post by prettyblocks »

Izkata wrote: post #67
TonyAlicea10 wrote: post #43

Earlier quoted context omitted.

I don't think this is equivalent. The calculator won't occasionally hallucinate a wrong answer. LLMs are a far leakier abstraction which means skill atrophy impacts evaluation and verification ability.

They're not an abstraction at all until we're committing prompts instead of code.

But many people are doing just that, automating workflows through prompt templates and skills.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#114

Post by forshaper »

msdz wrote: post #2

> LLM’s amplify what you already have: opinions, structure, frameworks. So far, so agreeable, but… > If you have thoughts, they come out sharper and faster. I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles. Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thou…

In a way, this happened before LLMs with more workforce-fit education, decision tree flowcharts, then software, and so on. If you take most people who started any field, the way they started the field would look very unorthodox, inefficient, etc. "From the margins," as pg might say. Margins where more intuitive skills for the craft are present. Ie, the skills to come up with the model pastry are not the same skills for the pastry line to be baked in a factory.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#116

Post by westurner »

deaton wrote: post #111
Diogenesian wrote: post #47

Earlier quoted context omitted.

I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents. Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt…

I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.

Like becoming a manager?

When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce.

Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax

Managers and developers are responsible for setting a test coverage threshold for merge approval.

If there is 100% branch coverage test coverage for a codebase, what would coverage-guided fuzzing or property testing find? If there is 100% branch coverage test coverage for a codebase, what is the value of spending resources on formal verification?

How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage?

Re: The LLM Critics Are Right. I Use LLMs Anyway

#117

Post by swiftcoder »

> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane.

What are people actually doing with all these tokens? I use LLMs pretty heavily for development, and I'm rarely spending all the tokens that come with a $10/month OpenCode Go subscription...

Re: The LLM Critics Are Right. I Use LLMs Anyway

#118

Post by swiftcoder »

mohamedkoubaa wrote: post #95
1970-01-01 wrote: post #39

Earlier quoted context omitted.

>Master craftsmen didn't take on apprentices to give them chores. Is today opposite day?

So the purpose of a graduate degree is TAing a few classes?

A big part of the value to the tenured professor is not having to teach an extra couple of sections of CS 101. The other part is plastering their name on a bunch of extra papers to game their h-index :D

I don't want to be too flippant here, but senior engineers generally gain very little from training juniors, not least because promo processes suck at basically every tech firm, and so the moment you train up a junior, they are going to leap to another company rather than go through promo...

Re: The LLM Critics Are Right. I Use LLMs Anyway

#119

Post by skydhash »

Rumudiez wrote: post #108
Levitz wrote: post #24

Earlier quoted context omitted.

>Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. My largest concern comes from something tangential to this: I'm not sure we're all that good at deciding what should be learned and sticking to it. Silly…

Knowing what to learn has always been an implicitly vital skill to career growth. Maybe it’s all right fewer people will know regex, just like as each year passes relatively fewer people know about var hoisting in JS What does the business benefit from you handwriting regex vs a clanker?

The benefits of knowing regex is not about the syntax (that’s what docs/google/books are for), but knowing when it’s right to use it.

Immersion with the codebase is not about coding and syntax (which is actually very easy for me). It’s about being able to intuit that a particular combination can result in a buggy state and provides the wrong result downstream. You lose that you stay in the dreamland of specs.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#120

Post by p2detar »

post-it wrote: post #8

> And this is where the value is for me: I can simply make things higher quality than I could do them alone. Yeah, that's the thing for me. LLMs have made my work easier and faster, and they've made my side projects easier and faster. I think there are very sensible and valid critiques but so far the tool works for me.

Same thing here. I was able to finish side projects on my list and even refactor decades old projects with the help of LLMs. It works great so far and yeah, I can still write source code when needed.
Post reply on HN