Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Productivity Gap

bjorg.bjornroche.com

Post reply on HN
Print view View original

31–40 of 127 posts

Re: The AI Productivity Gap

#31

Post by yoz-y »

PostOnce wrote: post #5

Pre AI and Post AI code review hours are both 0.75 in this made up example. I find that implausible. Even with the same amount of code, AI code is less trustworthy* and requires more attention... but we know it won't be the same amount, it will be more. This means it will take longer to review, or there will be unforeseen consequences of not spending that extra time. *meaning no human eyes have looked at it and said…

To me the biggest gotcha with AI code is that the bugs are not “normal”.

When reviewing human code I focus on specific parts because I know that there are parts where a person will just not make a bug (unless very junior).

AI on the other hand, will not do an off-by-one mistake, but it will happily just delete perfectly working code for no obvious reason. Or monkey patch a dependency because it missed a config flag. Or generally fail in a very novel and creative way.

The effort it takes to review AI code is much greater. And this is in a code base I am deeply familiar with.

Imo the future lies in a solid core programs with powerful plugin frameworks that expect all plugins to be code that was never read.

Re: The AI Productivity Gap

#32

Post by dgellow »

mawadev wrote: post #14
dgellow wrote: post #6

Earlier quoted context omitted.

I stopped using coding agents after more than one and a half year of active use, it really started to become way too boring, and I’m t a point where I just hate having to babysit them and for the 200th time make it understand what the actual goal is… and to be honest, going back to writing code by hand without assistance is really hard at first you continuously have that little voice telling you how simple that would…

I'd be really interested to see all the software that is written by agents. Whenever I touch agents or ai I can't get much use out of them. My understanding is the value when I think aloud with them/treat them as a better google search, but thats about it. Except one off web stuff, that is a pretty neat use case. But lets be real, anything moderately complex that is out of the domain of publicly available sample code…

For what it’s worth, I’ve made really good, state of the art software in my areas of interest using LLMs. So I do believe you can produce really good software using them. But it’s domains where I have decade of experience.

But even with that result I don’t think it’s something we should bet the whole industry on, and something I personally don’t feel comfortable relying upon

Re: The AI Productivity Gap

#33

Post by hahahaa »

This assumes you are arranging deckchairs and not leaving the cruise ship for say, a boeing 747.

One example, let's say there is a side bet that makes everyone 10x more productive with a success rate of 1%

It takes 2 hrs to make the bet wit agent orchestration.

10 people can get this done in their spare time freed up by AI in 5 weeks.

Bet cashes in and you are much faster at everything.

It won't feel faster. Because the brain probably scores emotionally in roadblocks cleared per hour.

Back when you got a single punch card loaded in a day it felt like a fucking win.

The other factor is you get paid the same and there is more disruption and competition and job insecurity.

But objectively value gets shipped faster using AI.

Just not much if you go the faster horses route with AI. You need the cars. (Or planes!)

Re: The AI Productivity Gap

#34

Post by crnkofe »

It doesn't really matter how much more productive a developer is if all other roles at the company don't follow suit. Before a developer picks up something to work on a series of roles had to set their eyes on work to be done. Project/product leads, tech leads, business people stamping and deciding on priorities. Then there's all the work that happens after a developer finishes work which tends to be manual as well. Review, QA, education, ops changes, marketing material, education articles, webcasts, showcasing features to end users and lets not forget end users actually making good use of the amazing new features shipped and likely many more largely sequential processes depending on company size and product/project type.

There's no real way to get to a 10x developer nowadays. Even if a company somehow achieved the magic productivity increase in all employees you still need a 10x consumer to gulp it all down.

Re: The AI Productivity Gap

#35

Post by furyofantares »

adithyassekhar wrote: post #30

I don’t think human review is worth it for LLM generated code. We design abstractions and all around how humans think. LLMs writes code that is better understood by machines. If you are all in on LLMs, by all means, read the code figure out what it means. But trying to enforce a human flow to its logic is flawed and will be overwritten the next time.

LLMs definitely write worse code for LLM consumption than humans can. In my experience your claim can't be further from the truth. I can get much further with an LLM starting from a great codebase than I can starting from a vibed codebase.

And I can do even better than that if I design the codebase specifically with LLM coding in mind, making choices that make it hard or impossible for the LLM to make certain categories of error it tends to make, and make it easier for the LLM to observe the results.

Re: The AI Productivity Gap

#36

Post by hahahaa »

laszlojamf wrote: post #2

What I have noticed in my own work that a lot of the time that used to be for coding is now just waiting. I have three agents working on three different features in parallel, and I'll go back and forth with all of them, correcting things and steering etc, but then I find myself with three busy agents and nothing to myself except stare at the screen while they code away. There is a mental budget for me where I can't h…

You need loops so they run longer and use less of your context and brain power. Then (and this is where WFH is a super power) do stuff like walk, daydream, come up with killer ideas like a Madman episode laying on the office couch.

Re: The AI Productivity Gap

#37

Post by hahahaa »

dwedge wrote: post #12
laszlojamf wrote: post #2

What I have noticed in my own work that a lot of the time that used to be for coding is now just waiting. I have three agents working on three different features in parallel, and I'll go back and forth with all of them, correcting things and steering etc, but then I find myself with three busy agents and nothing to myself except stare at the screen while they code away. There is a mental budget for me where I can't h…

This has been my observation too. Because I'm chatting it feels like I'm not working, so any output can be "productive" in that context but I'm hyper aware of all the negative time here. Correcting, pushing it back to the prompt, reminding it that it doesn't have full context so do what I told you not what you think, and then verifying it and correcting it (always) seems to take longer than just doing the work myself

You should spend time as much as 50% on harness engineering for you and team to stop that and make the AI steer better.

Re: The AI Productivity Gap

#40

Post by himata4113 »

pydry wrote: post #23
himata4113 wrote: post #16

Earlier quoted context omitted.

Neither of these points feel true anymore. Models are very much predictable these days (except anthropic models). The real issue stems from letting them work on their own for far too long. Also we are not controlled by 2 companies anymore as kimi k3, deepseek flash (and soon pro) as the ultra-cheap variants, glm 5.2 especially is a direct replacement for opus 4.8. Models will only get better and cheaper I wouldn't fe…

I always found that if you are good enough at whittling down boilerplate that coding becomes something akin to pure architecture. I find that mediocre programmers and LLMs are bad at both. They're helpful if you want to shit out some repetitive boilerplate or perform a complex search of some kind but otherwise you're better off without.

ehh, they're pretty good at automated performance research and bug fixes, especially when spanned across hundreds of them.
Post reply on HN