Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Productivity Gap

bjorg.bjornroche.com

Post reply on HN
Print view View original

91–100 of 127 posts

Re: The AI Productivity Gap

#91

Post by tempfile »

K0balt wrote: post #67
tempfile wrote: post #42

> Reading and Debugging 1.5 1.0 > Code Reviews 0.75 0.75 Since these numbers are made up, I may as well throw my personal anecdote in the ring. I find reading and reviewing far harder with coworkers who are using AI. Tickets contain about 5x as much meaningless junk as they used to, and testing notes - while far more thorough - are often now multiple pages in length. Reviews also contain much more code, people try to…

Your process engineering is lacking. Just throwing AI at existing workflows seldom produces good results. Processes have to be reengineered to benefit from the strengths and cover for the weaknesses of AI systems, with observability at the right inflection points being fundamental to success.

Care to give an example?

Re: The AI Productivity Gap

#92

Post by aaron_m04 »

yoz-y wrote: post #31
PostOnce wrote: post #5

Pre AI and Post AI code review hours are both 0.75 in this made up example. I find that implausible. Even with the same amount of code, AI code is less trustworthy* and requires more attention... but we know it won't be the same amount, it will be more. This means it will take longer to review, or there will be unforeseen consequences of not spending that extra time. *meaning no human eyes have looked at it and said…

To me the biggest gotcha with AI code is that the bugs are not “normal”. When reviewing human code I focus on specific parts because I know that there are parts where a person will just not make a bug (unless very junior). AI on the other hand, will not do an off-by-one mistake, but it will happily just delete perfectly working code for no obvious reason. Or monkey patch a dependency because it missed a config flag.…

> Imo the future lies in a solid core programs with powerful plugin frameworks that expect all plugins to be code that was never read.

What makes you think that?

Re: The AI Productivity Gap

#93

Post by Yopolo »

matthorse wrote: post #17

Writing code is a small part of everyday's job of a software engineer. The article's table reflects this fairly well. AI compresses implementation time for an individual engineer, but architecture decisions, design reviews, integration, testing, deployment, and production validation remain largely serial activities. If code generation speeds up by 5x while those bottlenecks don't, you've mostly increased the team's w…

Besides that i still think its wayy to early to put AI in front of center of everything, the current progress at least makes it clear that is not going to stay as it is.

Noneetheless, if the blog article already determines a gain of 15% and you have to factor in the cost of tokens, the company should still reduce the team size by 10-20% or by 1-2 people per 5-10 person team.

Which is still a massive issues for everyone who writes code.

Btw. when i was writing code full time (not being an architect), i for sure wrote like 90% of my time code.

I'm doing an experiment though having a code review agent becoming better and better doing code reviews how i would do it. Which leads also to a skill which fixes issues before the code review agent even has to find them.

We can now automate cases which were unthinkable. We can even keep the code review quality if i leave (Its not perfect yet but its better than what we had before)

Re: The AI Productivity Gap

#94

Post by asdff »

swiftcoder wrote: post #55
crnkofe wrote: post #34

It doesn't really matter how much more productive a developer is if all other roles at the company don't follow suit. Before a developer picks up something to work on a series of roles had to set their eyes on work to be done. Project/product leads, tech leads, business people stamping and deciding on priorities. Then there's all the work that happens after a developer finishes work which tends to be manual as well.…

The exception here is probably in very small companies. I'm curious to see if LLMs can usher in a new golden age of the one-programmer-one-designer-one-sales indie teams that typified so much of the 80s and 90s...

If they can one shot that, so can you, and you can custom tailor that tool to your needs not wait for them to add features they think might fit their collective customer base’s needs.

Re: The AI Productivity Gap

#95

Post by himata4113 »

pydry wrote: post #87
himata4113 wrote: post #81

Earlier quoted context omitted.

feels like a harness issue, bug fixes work out fine with omp.sh as long as it can be tested properly. Untestable things tend to lead to hallucinated results though.

might be a symptom of writing complex code. I'm sure on basic CRUD they can figure out most bugs since most of them have happened 1000 times before.

I doubt it, a lot of what I use llms for is on the very edge of what these models are trained for.

Re: The AI Productivity Gap

#96

Post by yoz-y »

aaron_m04 wrote: post #92
yoz-y wrote: post #31

Earlier quoted context omitted.

To me the biggest gotcha with AI code is that the bugs are not “normal”. When reviewing human code I focus on specific parts because I know that there are parts where a person will just not make a bug (unless very junior). AI on the other hand, will not do an off-by-one mistake, but it will happily just delete perfectly working code for no obvious reason. Or monkey patch a dependency because it missed a config flag.…

> Imo the future lies in a solid core programs with powerful plugin frameworks that expect all plugins to be code that was never read. What makes you think that?

I think this is a good practice anyway. Putting as much code as possible into silos with guarded permissions.

Historically plugins have been kind of crappy because they were constantly breaking with updates. However if they only live as a spec, and are regenerated when needed they can easily survive API changes.

Bonus feature is that if all “installed” plugins are generated together, the llm can also find ways to avoid them being buggy due to weird interactions.

All this while keeping the main program from crashing.

Security wise the spec can also be inspected using a trusted LLM. It is trivial to hide exfiltration or malicious code in plugin/extension code (e.g.: honey). But it is much harder to hide it in a spec.

Re: The AI Productivity Gap

#97

Post by Supermancho »

tempfile wrote: post #42

> Reading and Debugging 1.5 1.0 > Code Reviews 0.75 0.75 Since these numbers are made up, I may as well throw my personal anecdote in the ring. I find reading and reviewing far harder with coworkers who are using AI. Tickets contain about 5x as much meaningless junk as they used to, and testing notes - while far more thorough - are often now multiple pages in length. Reviews also contain much more code, people try to…

The numbers are completely made up. Jr developer 2.5 vs 1.0 while "regular" is 1.0 and 1.0? The more senior the bigger the work. It's the same across both, worst case.

0.75 to 0.75? Rework from review is also much faster. Now you don't have to tell a peer to rework a bit here and there for obvious reasons and spend time on a new loop. The review process isn't atomic.

Our production pipeline is faster across our very large organization, after implementing AI processes.

> Tickets contain about 5x as much meaningless junk as they used to

This is a process problem. Developers should be able to answer questions about their PRs, or you reject it. It's not a daunting blanket issue.

Re: The AI Productivity Gap

#99

Post by ericmcer »

rightbyte wrote: post #20
PostOnce wrote: post #5

Pre AI and Post AI code review hours are both 0.75 in this made up example. I find that implausible. Even with the same amount of code, AI code is less trustworthy* and requires more attention... but we know it won't be the same amount, it will be more. This means it will take longer to review, or there will be unforeseen consequences of not spending that extra time. *meaning no human eyes have looked at it and said…

The hard part is that LLM code looks like there is some sort of flow. It is like a nice statistical smooth flow. It looks very convincing at a glance. No one would write code like that and not know what they are doing comments self assured and all.

My experience as well, it is too fond of abstractions and will constantly spin up functions like: isUserAdmin(){return user.isAdmin}

which look nice on a surface level but obfuscates real understanding of the code and the actual data structures being used. Your end result is pretty and reads nice, but is bloated and difficult to reason with code.

Re: The AI Productivity Gap

#100

Post by wannabe44 »

Diti wrote: post #50

How to get rid of every highly-skilled-but-unmedicated neuroatypical developer (could be people like Xe Iaso or Soatok): > hiring someone who is a good coder, but has trouble reasoning about systems, has no patience for working through hard problems with others, and can’t break down vague requirements into tangible action items. Why not hire the excellent developers for the highly-technical skills they bring, and mat…

> no patience for working through hard problems with others, and can’t break down vague requirements into tangible action items.

These are very convenient and vague enough excuses to single out whoever honestly says your architecture is stupid.

Post reply on HN