Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM Critics Are Right. I Use LLMs Anyway

theocharis.dev

Post reply on HN
Print view View original

291–300 of 330 posts

Re: The LLM Critics Are Right. I Use LLMs Anyway

#291

Post by thaumasiotes »

GuB-42 wrote: post #253
qsort wrote: post #62

Earlier quoted context omitted.

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

Maybe we can make an analogy with taking the plane vs driving to your destination. A plane literally goes 10x faster than a car, so a 5 hour drive becomes a 30 minutes flight. But if you have to drive to the airport, arrive early, pass check-in, security, boarding, then pick up your luggage on arrival, rent a car and drive to your destination, you may realize that your 30 minute flight took you more than 5 hours in t…

> You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.

It's a 10% speedup, unless you think that a 100% speedup would reduce completion time to zero rather than cutting it in half.

Assuming that "10% of the project" is measured by time requirements, when you get a 10x speedup on that part of the project, you'll end up completing 100 units of work in 91 (formerly 100) units of time. Your work rate is then 100 / 91 = 1.099 times what it was before; that's an improvement of 10%, not 9%.

(Sanity check: suppose you have a project that will take 100 days. That project is reassigned to someone 10% faster than you. They will take 100 / 1.1 = 90.91 days to finish.

If they were 9% faster, they'd take 100 / 1.09 = 91.74 days to finish.

Does a savings of 9 out of 100 days of work look more like the project that saves 9.09 days, or the one that saves 8.26 days?)

Re: The LLM Critics Are Right. I Use LLMs Anyway

#292

Post by black3r »

matsemann wrote: post #48

My use of AI is correct. My coworkers use is not. I'm saying it in jest, but it's also a bit true. Not necessarily because we use it any differently. But because my use of AI saves me time. But their use of AI adds more to my plate, no matter if it's slop or not.

So much this. When coding I use my LLM to hone the code to perfection before sending it to review, including running AI reviews on it multiple times and fixing obvious bugs so that reviewers have less work to do. I also test all my code manually (LLM can be helpful by giving me a list of test scenarios to consider). When reviewing PRs I carefully read the AI reviewer output, consider which points are relevant, which are irrelevant and only post the relevant issues when requesting changes.

I have coworkers who routinely post PRs with comments stating "all tests green, ready to review" and then I actually start the application, test it manually and it doesn't work. And don't get me started on reviewers who post the raw AI output of their "reviewing skill" and then I have to read through 30 paragraphs of "findings" most of which are irrelevant (just today I got the "valuable" feedback of "View is @api_view(["POST"]); no test asserts GET/PUT/PATCH/DELETE → 405")...

Re: The LLM Critics Are Right. I Use LLMs Anyway

#293

Post by patrickmay »

hodder wrote: post #131
MisterTea wrote: post #126

Earlier quoted context omitted.

> but risking being left in the dust productivity-wise? What's the risk here? Left behind by who or what? > Or taking an “agent first” approach and trying to learn and improve more only on the side, as more of an afterthought? This reads like anxiety resulting from FOMO. Here's my take: I don't care about LLMs or AI in the sense that I don't feel any need or want to use them. I've only ever tinkered with the free Cha…

Economically if you are vastly outcompeted by other programmers on productivity, yes that is "no longer of value" from a purely employment perspective. Much like an old person who cant use a computer has little value in the job market beyond being a greeter at Walmart, a programmer who hand codes a loop is next to useless on a productivity basis such that it makes zero sense to employ them. It is unfortunate but true…

> Economically if you are vastly outcompeted by other programmers on productivity

That doesn't seem to be the case, at least currently. The credible numbers from from companies that are using LLMs extensively are in the 15-30% productivity improvements for some developers. Those figures don't factor in how that impacts end-to-end SDLC productivity and many of the published results ignore quality which impacts maintenance costs.

Before someone hops in with "Future models will do better", that's also by no means guaranteed. I think the poster you're replying too is safe for a while.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#294

Post by sivakon »

ryan_n wrote: post #267
sivakon wrote: post #246

> And then there are the geopolitical tensions. What happens if China or the US cut us off overnight from these technologies? Just a couple of weeks ago the US government showed it was able and willing to cut off non-US citizens from Anthropic’s latest frontier model. The only evidence you have is US here and all the frontier models from China are free to download. Pure projection. You can read China's AI strategy he…

I might be missing something but what exactly is being projected here? The US did what the author said. It's not farfetched to imagine geopolitical conflicts could, at some point, drive china to do something similar. > all the frontier models from China are free to download Maybe? Again, not hard to imagine that they might not be releasing their actual frontier models to the public.

The link I shared mentions their AI strategy.

Short version is this: putting export controls on AI open-source will hinder their own internal open-source ecosystem

Re: The LLM Critics Are Right. I Use LLMs Anyway

#295

Post by u8 »

> I strongly agree that written text should be from humans to humans. Yet I still write all of my texts with LLMs. And I don’t find that contradictory. What distinguishes “AI slop” from “good writing” is whether a human has put thoughts behind it. And you cannot outsource thinking.

Writing is thinking. If you're outsourcing your writing to an LLM, you are shortchanging yourself by skipping over much of the refinement of ideas that the process of writing provides.

I wish the author had clarified how he uses LLMs for writing. It's perfectly fine to have an LLM proofread and fix your grammatical errors--that's a mechanical task. But I think it's gross when it starts putting words and content on the page.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#296

Post by encyclopedism »

dannersy wrote: post #258

I think folks ITT are trivializing the issue here. Many keep drawing parallels to LLMs being akin to the loom or calculator. That seems intellectually dishonest to me. Those tools were not anthropomorphized interfaces, nor were they instant gratification. To do significant things with a calculator, you still had to know maths, order of operations, formulas, etc. We didn't lose mathematicians as a result of the calcul…

Exactly this!

LLM's are replacing your 'brain' i.e. the deepest most amazing part of you. The potential value of this to business is astronomical!

No there won't be an explosion of new software, a minor blip perhaps and even this one of diminishing returns (each cycle of LLM improvements requiring less and less human input).

If there are 25 million software developers today then the ROI by OpenAI, Anthropic et al pays off if in a few years time only 15 million developers are needed. That 10 million is the ROI and that is likely to be you.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#297

Post by deaton »

westurner wrote: post #188
BalinKing wrote: post #176

Earlier quoted context omitted.

I've never really understood this argument. If someone's a manager of an incompetent team, no amount of management skill will save the quality of the resulting software. I don't think "just treat LLMs like smart junior developers" fixes this, because well-functioning teams usually also have senior developers to keep things on track. Like, if we handed a team of genius-but-junior developers to the best "people person"…

If you have a "this is an incompetent team" attitude, that's what quality software you'll get back; whether or not there's LLM use. If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back. There are people producing good and excellent quality software with LLMs. Presumably you must discard low-quality code in…

> If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back.

As much as this sentiment is nice, it is completely divorced from reality, unless the competence is verifiably there. If you take a bunch of juniors and say "yeah I trust them to do everything well enough," you're going to have a disaster on your hands.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#298

Post by discreteevent »

quantummagic wrote: post #135
cj wrote: post #123

Earlier quoted context omitted.

> They're not wrong, they're just not helpful or accomplishing anything. I think what it accomplishes is developing a shared understanding of the pros/cons of the tech, the risks associated, how we should mitigate those risks, where the tech is appropriate vs. not appropriate, etc. It feels very dismissive to say that these conversations aren't worth having. Or that we're thinking about these things too much. I get t…

There are conversations worth having. But it feels very dismissive of the obvious power of the technology to constantly focus, in an unbalanced way, on the challenges. People are biased heavily by irrational emotion, on both sides of the debate. It's all just getting a little tedious. I'm all for rational discourse and debate, but it's hard to find through all the vitriol and contempt.

> The technology marches forward and will develop naturally. If you personally don't want to use it, then don't.

The debate marches forward and will develop naturally. If you personally don't want to read it, then don't.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#299

Post by eddyzh »

cognitiveinline wrote: post #153
eamonnsullivan wrote: post #134

The one thing that had me reading to the end was the mention near the top of the environmental impact of LLMs, but he never got back to it. I'm currently writing an onboarding doc for my team, encouraging LLM use for some tasks. (OK, well, I'm actually procrastinating by reading HN). At the same time, I'm in a darkened office with tinfoil on the windows and a fan pointed at me because it's hell outside and it has bee…

Your commute is worse for the environment than your day's token usage, by a magnitude.

This checks out 50-250kg CO2 are common guesses for 10k spend.

That would be in the same order of magnitude of an average daily commute with an older car.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#300

Post by YeGoblynQueenne »

>> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane.

No it just sounds completely stupid. You spent $10k to get a program to write text that you can write yourself just as well and for free. "Dissonance" doesn't even begin to describe it. It's like suddenly everyone has turned into the listless blobs of fat that humans have turned into in Wall-E.

I just hope all that will eventually correct itself: open (one way or another) models will reach parity with proprietary ones when diminishing returns eventually make it impossible for LLMs to grow any longer, OpenAI and Anthropic will succeed in bankrupting each other, it will become possible for everyone to run their own, personal, free and open (one way or another) model on their own laptop (assuming we still own our laptops) and at that point most of the problems with LLMs, particularly the ones that come from having to pay someone else to do the things you could always do yourself for free, will go away.

I just hope.

Post reply on HN