Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM Critics Are Right. I Use LLMs Anyway

theocharis.dev

Post reply on HN
Print view View original

171–180 of 330 posts

Re: The LLM Critics Are Right. I Use LLMs Anyway

#171

Post by coldtea »

tablarasa wrote: post #137
westurner wrote: post #116

Earlier quoted context omitted.

Like becoming a manager? When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax Managers and developers are responsible for setting a test coverage threshold for merge approval. If there is 100% branch coverage…

> How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage? This is such a salient question. Sometimes (definitely not always) the test suites produced by LLMs are so trivial it's scary. Coverage can be an illusion for sure.

I have been seeing such trivial shit getting submitted and deleting them from the test suites, but it's disheartening to see that the devs didn't have the better taste to see those LLM "tests" are so trivial they border on noop before submitting them...

Re: The LLM Critics Are Right. I Use LLMs Anyway

#172

Post by astrange »

skydhash wrote: post #161
westurner wrote: post #116

Earlier quoted context omitted.

Like becoming a manager? When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax Managers and developers are responsible for setting a test coverage threshold for merge approval. If there is 100% branch coverage…

> When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. > Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax I can see managers getting involved into acceptance tests, but never in the other type of tests. And the verification mostly is involved into a quick m…

Managers often still have code and team-lead responsibilities IME. First-line managers, that is.

And then sometimes you report a memory leak and it gets fixed by a VP and you wonder if he doesn't have something better to do.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#173

Post by nonameiguess »

msdz wrote: post #2

> LLM’s amplify what you already have: opinions, structure, frameworks. So far, so agreeable, but… > If you have thoughts, they come out sharper and faster. I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles. Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thou…

I guess I'm lucky to have taken the career path I have, but I write very little code these days being in pre-sales now. "Productivity" is pretty stark and obvious in sales. You can output all the text you want, maybe it even compiles and passes test suites and looks pretty in a demo. But does it sell? It's nice to be able to have the only true measure that matters actually be measurable. I have yet to use an LLM for anything at all, haven't touched an agent, not because I'm ideologically opposed, but I just haven't seen the need. Part of it is the largely advisory role. I'm largely expected on the customer side to field and answer questions and render technical assistance. I'm not much good at that if I'm just deferring to Claude. The customer can do that themselves. I need to know more than Claude. The other part is just I'm not being paid to generate text. I'm being paid to generate revenue. If there's a convincing argument that LLMs can help me, well, it's an argument, but truth on the ground is I'm performing every bit as good if not better than any of my peers that are blowing a grand a month on tokens. They're definitely having a lot of fun and flooding internal Github orgs with new repos and tooling, but it demonstrably isn't generating new revenue.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#174

Post by [deleted] »

swiftcoder wrote: post #117

> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane. What are people actually doing with all these tokens? I use LLMs pretty heavily for development, and I'm rarely spending all the tokens that come with a $10/month OpenCode Go subscription...

[deleted]

Re: The LLM Critics Are Right. I Use LLMs Anyway

#175

Post by cadamsdotcom »

LLMs are awesome, but sometimes we'd be better off with more friction.

This is because.. sometimes an idea is just bad. I learned the hard way the other day when I filed this PR against prek: https://github.com/j178/prek/pull/2302

My idea was bad.

The implementation was meant to solve my problem. All it did was expose more problems and waste my time and the maintainer's, and god knows how many tokens from Claude and Codex. In the end the PR was useless so I closed it.

The maintainer and I would have both been better off - less time wasted - if I was forced to forever wonder if the idea was good, letting it bounce around the back of my mind.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#176

Post by BalinKing »

westurner wrote: post #116
deaton wrote: post #111

Earlier quoted context omitted.

I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.

Like becoming a manager? When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax Managers and developers are responsible for setting a test coverage threshold for merge approval. If there is 100% branch coverage…

I've never really understood this argument. If someone's a manager of an incompetent team, no amount of management skill will save the quality of the resulting software. I don't think "just treat LLMs like smart junior developers" fixes this, because well-functioning teams usually also have senior developers to keep things on track. Like, if we handed a team of genius-but-junior developers to the best "people person" manager in the world (i.e. who doesn't actually read the code), would we really expect decent results? Even if the manager tested the code by hand? I honestly don't think so, at least once the software gets past a certain (fairly low) threshold of size/complexity.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#177

Post by [deleted] »

cadamsdotcom wrote: post #175

LLMs are awesome, but sometimes we'd be better off with more friction. This is because.. sometimes an idea is just bad. I learned the hard way the other day when I filed this PR against prek: https://github.com/j178/prek/pull/2302 My idea was bad. The implementation was meant to solve my problem. All it did was expose more problems and waste my time and the maintainer's, and god knows how many tokens from Claude and…

[deleted]

Re: The LLM Critics Are Right. I Use LLMs Anyway

#178

Post by [deleted] »

ra0x3 wrote: post #164
breadzeppelin__ wrote: post #100

Earlier quoted context omitted.

I am both a career developer and experienced team manager. from first hand experience the 1.5x im getting from AI is not flowing down to my team / org because why would i output 50% more when the pay environment and leadership are already underwhelming. That additional 50% productivity goes completely to side projects built on my second computer between 9-5 tasks

Great point. Devs have relatively no incentive to be more productive for their orgs, and all the incentive to be more productive on their own personal work. The AI benefit to devs is real, just not for large enterprises IMO outside of automating clerical/mundane work.

[deleted]

Re: The LLM Critics Are Right. I Use LLMs Anyway

#179

Post by grayhatter »

cognitiveinline wrote: post #151
grayhatter wrote: post #57

Earlier quoted context omitted.

> Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. [Excluding people who don’t want anything to do with LLMs out of moral principle, which curiously just like the overarching topic I also both respect and…

I won't be as diplomatic as the other reponse to you. You are welcome to your reticence to agentic coding. But as you're no doubt observing, all your peers are saying they're moving ahead leaps and bounds. Either all of them are wrong, or you will be left behind. Whatever the outcome, it will be on you, and you may make your peace with that.

I appreciate the candor :) But if I'm honest about the size of my ego; the people who write code I admire, the ones who feel remorse when something they built breaks. The people who care about others, and about quality, and don't list TC as something of value. Those are the people who I consider peers. None of them enjoy LLM codegen. They all tell me, that it's annoying because it takes more effort to fix it's output than to just write it by hand. So, no, none of my peers are saying they're moving ahead leaps and bounds. They equally don't understand why everyone thinks it's better. Excluding those people though; I'm more used to everyone around me being wrong. It'll be interesting to see if I, or we, get left behind.

I really appreciate your comment, I think I've discovered part of why I've never felt any anxiety over being replaced by an LLM. I've never tried or wanted to emit low quality code at high velocity, my standards and goals have always listed quality first. Feels more like asking a chef if they're afraid of llms. Why would they be, they're not doing the same thing.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#180

Post by westurner »

pjc50 wrote: post #147
westurner wrote: post #116

Earlier quoted context omitted.

Like becoming a manager? When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce. Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax Managers and developers are responsible for setting a test coverage threshold for merge approval. If there is 100% branch coverage…

Formal verification is always more valuable than mere testing, but it's hitherto more expensive. The thing that ultimately matters is closing the loop: how well do the tests match the requirements, both as written and as unwritten in the mind of the customer? Working with an LLM has given me a real eye opener on unwritten requirements. It's like outsourcing. "Yes, you've given me what I wrote down, but I never expect…

Causes of failure; Operator skill, prompt insufficiency, model insufficiency, agent insufficiency, insufficient code review, overloaded merge maintainer
Post reply on HN