Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI coding tools can reduce productivity

secondthoughts.ai

Post reply on HN
Print view View original

51–60 of 284 posts

Re: AI coding tools can reduce productivity

#51

Post by Ancalagon »

journal wrote: post #34
tomcam wrote: post #16

What bothers me more than any of this particular discussion is that we seem to be incapable of determining programmer productivity in a meaningful way since my debut as a programmer 40 years ago.

what about the $ you make? isn't that an indicator? you've probably made more than me, so you are more successful while both of us might be doing the same thing.

In a vacuum I don’t believe pay alone is a very good indicator. What might be a better one is if someone has a history across their career of delivering working products to spec, doing this across companies and with increasing responsibility. This of course can only be determined after the fact.

Re: AI coding tools can reduce productivity

#52

Post by franciscop »

kylecazar wrote: post #26

Now do a study that specifically gauges how useful an LLM (including smart tab completion) is for a frontend dev working in react/next/tailwind on everyday Jira tickets. These were maintainers of large open source projects. It's all relative. It's clearly providing massive gains for some and not as much for others. It should follow that it's benefit to you depends on who you are and what you are working on. It isn't…

It's a very well controlled study about... what the study claims to do. Yes, they didn't study a different thing, for _many_ reasons. Yes, we shouldn't haphazardly extrapolate to other parts of Engineering. But it looks like it's a good study nonetheless.

There are some very good findings though, like how the devs thought they were sped up but they were actually slowed down.

Re: AI coding tools can reduce productivity

#53

Post by skissane »

xarope wrote: post #29

I think this for me is the most worrying: "You can see that for AI Allowed tasks, developers spent less time researching and writing code". My analogy to this is seeing people spend time trying to figure out how to change colors, draw shapes in powerpoint, rather than focus on the content and presentation. So here, we have developers now focusing their efforts on correcting the AI output, rather than doing the resear…

I find I’m most likely to use an LLM to generate code in certain specific scenarios: (i) times I’m suffering from “writer’s block” or “having trouble getting started”; (ii) a language or framework I don’t normally use; (iii) feeling tired/burnt out/demotivated

When I’m in the “zone” I wouldn’t go near an LLM, but when I’ve fallen out of the “zone” they can be useful tools in getting me back into it, or just finishing that one extra thing before signing off for the day

I think the right answer to “does LLM use help or hinder developer productivity” is “it depends on how you use them”

Re: AI coding tools can reduce productivity

#54

Post by dismalaf »

I find LLMs are decent at regurgitating boilerplate. Basically the same kind of stuff you could google then copy-paste... AI chatbots, now that they have web access, are also good at going over documentation and save you a little time searching through the docs yourself.

They're not great at business logic though, especially if you're doing anything remotely novel. Which is the difficult part of programming anyway.

But yeah, to the average corporate programmer who needs to recreate the same internal business tool that every other company has anyway, it probably saves a lot of time.

Re: AI coding tools can reduce productivity

#55

Post by jaredklewis »

tomcam wrote: post #16

What bothers me more than any of this particular discussion is that we seem to be incapable of determining programmer productivity in a meaningful way since my debut as a programmer 40 years ago.

I’m confused as to why anyone would think this would be possible to determine.

Like can we determine the productivity of doctors, lawyers, journalists, or pastry chefs?

What job out there is so simple that we can meaningfully measure all the positive and negative effects of the worker as well as account for different conditions between workers.

I could probably get behind the idea that you could measure productivity for professional poker players (given a long enough evaluation period). Hard to think of much else.

Re: AI coding tools can reduce productivity

#56

Post by benrutter »

strangescript wrote: post #33

This entire concept hinges on AI not getting better. If you believe AI is going continue to get better at the current ~5-10% a month range, then hand waiving over developer productivity today is about the same thing as writing an article about the internet being a fad in 1999.

If they do improve at 5-10% a month then that'd definitely be true (tbh I'm not sure they are even improving at that rate now - 10% for a year would be 3x improvement with compounding).

I guess the tricky bit is, nobody knows what the future looks like. "The internet is a fad" in 1999 hasn't aged well, but a lot of people touted 1960s AI, XML and 3d telivisions as things that'd be the tools in only a few years.

We're all just guessing till then.

Re: AI coding tools can reduce productivity

#57

Post by calrain »

I've been using Claude Code heavily for about 3 months now, and I'm pretty sure I'm between 10 and 20 times more productive while using it.

How I measure performance is how many features I can implement in a given period of time.

It's nice that people have done studies and have opinions, but for me, it's 10x to 20x better.

Re: AI coding tools can reduce productivity

#58

Post by shoo »

hackable_sand wrote: post #43
Tainnor wrote: post #10

Earlier quoted context omitted.

The sample size isn't 16 developers, it's 246 issues.

Okay, so why not 246,000 issues?

If you read through the methodology, including how they paid the participants $150 / hr, for 20-40 hours work per participant, you can probably hazard a guess why they didn't scale up the size of the study by 1000x.

Re: AI coding tools can reduce productivity

#59

Post by __loam »

jaredklewis wrote: post #55
tomcam wrote: post #16

What bothers me more than any of this particular discussion is that we seem to be incapable of determining programmer productivity in a meaningful way since my debut as a programmer 40 years ago.

I’m confused as to why anyone would think this would be possible to determine. Like can we determine the productivity of doctors, lawyers, journalists, or pastry chefs? What job out there is so simple that we can meaningfully measure all the positive and negative effects of the worker as well as account for different conditions between workers. I could probably get behind the idea that you could measure productivity…

Won't stop MBAs from trying though.

Re: AI coding tools can reduce productivity

#60

Post by fuomag9 »

calrain wrote: post #57

I've been using Claude Code heavily for about 3 months now, and I'm pretty sure I'm between 10 and 20 times more productive while using it. How I measure performance is how many features I can implement in a given period of time. It's nice that people have done studies and have opinions, but for me, it's 10x to 20x better.

Same, I’ve done stuff that should have taken me 2-3 weeks in days
Post reply on HN