Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI Productivity Gap

bjorg.bjornroche.com

Post reply on HN
Print view View original

111–120 of 127 posts

Re: The AI Productivity Gap

#111

Post by zormino »

hahahaa wrote: post #36
laszlojamf wrote: post #2

What I have noticed in my own work that a lot of the time that used to be for coding is now just waiting. I have three agents working on three different features in parallel, and I'll go back and forth with all of them, correcting things and steering etc, but then I find myself with three busy agents and nothing to myself except stare at the screen while they code away. There is a mental budget for me where I can't h…

You need loops so they run longer and use less of your context and brain power. Then (and this is where WFH is a super power) do stuff like walk, daydream, come up with killer ideas like a Madman episode laying on the office couch.

My step count is crazy these last few months, I'm loving it!

Re: The AI Productivity Gap

#112

Post by k1e »

Its too early to measure. Agentic coding harness'es arent mature yet. People try different approaches (with sub-agents, or without etc). Productivity will likely increase in near future even if new LLMs will hit the ceiling.

Re: The AI Productivity Gap

#113

Post by andrekandre »

zombot wrote: post #75
mingusrude wrote: post #71

Earlier quoted context omitted.

> AI compresses implementation time for an individual engineer, but architecture decisions, design reviews, integration, testing, deployment, and production validation remain largely serial activities. Integration, testing, deployment and production validation activities are also, depending on you setup, available for AI to work on. It depends on the type of stack that you have and how your hosting is set up but with…

GP: > With the current capabilities, models still need constant babysitting and course correction. You'd be opening yourself up to even more babysitting. Would that really save time, or improve quality? I'm not convinced, to put it mildly.

well, that is a problem with automation in general i think; if you go too far instead of a simple process a complicated processes get encoded in the automation and if it hits an edge case or breaks for whatever reason the babysitting kills productivity (this is just my experience anyways)

also, as far as quality goes, once something is automated that potential breakage puts a cap on how much improvement/quality you can achieve later... i'd guess llms are a bit more malleable there, but idk, i still see this issue even with skills and such...

Re: The AI Productivity Gap

#115

Post by andrekandre »

himata4113 wrote: post #60
rimliu wrote: post #49

Earlier quoted context omitted.

How exactly will models get cheaper?

Compare the performance of a 980 and a 5050 and I am sure that will answer your question. Also models baked into the silicon are able to achieve efficiency that is simply impossible to achieve with programmable circuits, there is a general slowdown in the raw capabilities that transformers can achieve and agentic tool use is simply an amplifier that will reach a wall eventually. It wouldn't surprise me if we saw with…

  > accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s
how do you update that baked-in model for things that have happened in the last say 2 months?

if i'm a programmer for example, even being a couple months old is a huge annoyance because programming languages and frameworks are changing all the time...

Re: The AI Productivity Gap

#116

Post by moezd »

Writing code is cheap. Coming up with a plan to stay updated on specs and verification are not so cheap, and you need more attention there than ever, because an LLM will take your half-assed spec and claim victory as it prints linted code that compiles and passes a few token unit tests while being under (500 or 1000) LoC.

That's PoC-level, happy-path-only engineering and LLMs are very dangerous at inserting subtle hardcoded values and cheats along the way, which makes it really hard to spot them later on. You need a system to whack them before they infect your codebase terminally, be it hyper vigilance, more SCA, more TDD, SDD... Otherwise you'll wake up one day with a production incident at 3 am and wonder how that code was even allowed in the release bundle.

Re: The AI Productivity Gap

#117

Post by vivzkestrel »

k1e wrote: post #112

Its too early to measure. Agentic coding harness'es arent mature yet. People try different approaches (with sub-agents, or without etc). Productivity will likely increase in near future even if new LLMs will hit the ceiling.

law of diminishing returns son

Re: The AI Productivity Gap

#118

Post by Rury »

tempfile wrote: post #42

> Reading and Debugging 1.5 1.0 > Code Reviews 0.75 0.75 Since these numbers are made up, I may as well throw my personal anecdote in the ring. I find reading and reviewing far harder with coworkers who are using AI. Tickets contain about 5x as much meaningless junk as they used to, and testing notes - while far more thorough - are often now multiple pages in length. Reviews also contain much more code, people try to…

You don't get it? It's like this:

"Make me a picture of a house".

> AI proceeds to draw a house.

"No that's not right, it should be a red bricked house. Not a brown one."

> AI redraws a red bricked house.

"No. It should have a 2 car garage, sit on top of a hill. Also it should have a front porch, and have a tree right in front."

> AI then draws a red bricked house on top a hill with a 2 car garage with a front porch and a tree right in front.

...

The issue is that people think AI should automagically create some vague idea in of theirs, without having to do the work of spelling out all the exact details. So AI (like people) must make some assumptions about what wasn't specified. Like since you didn't specify a "red bricked house" in your initial prompt, but merely a "house"... it had to come up with something as to the color, and did as you otherwise asked, but it didn't know you actually wanted a "red bricked house", since you never specified that detail. Hence why it failed to do what you wanted, and you had to "review and correct it".

Again this isn't solely an intelligence problem, but an inherent problem in language/communication, of unsaid assumptions/specifications. Of being unaware of what you don't know, unaware of your own assumptions, sometimes even being unaware of what you even want. It's why AI can't fully get rid of jobs in software.

But sometimes people just don't care. They just want a picture of a house made. Anything remotely resembling a house will do, not necessarily a solid one, or one that can withstand a magnitude 8 earthquake. You know... like something people can put up in minutes so they don't have to do hard work... such as a shabby old tent. And AI is very good at generalizing, so it can in fact achieve this. So everyone reports they are far more productive now, putting up tent after tent. Meanwhile, the people responsible for the slop have a nightmare to review...

Re: The AI Productivity Gap

#119

Post by HDBaseT »

andrekandre wrote: post #115
himata4113 wrote: post #60

Earlier quoted context omitted.

Compare the performance of a 980 and a 5050 and I am sure that will answer your question. Also models baked into the silicon are able to achieve efficiency that is simply impossible to achieve with programmable circuits, there is a general slowdown in the raw capabilities that transformers can achieve and agentic tool use is simply an amplifier that will reach a wall eventually. It wouldn't surprise me if we saw with…

> accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s how do you update that baked-in model for things that have happened in the last say 2 months? if i'm a programmer for example, even being a couple months old is a huge annoyance because programming languages and frameworks are changing all the time...

When was the last time you heard about someone talking about "Knowledge Cutoff" dates? OpenAI used to make a huge deal about it every release, now its not even mentioned.

We give agents tools, the ability to read a man page, the ability to use web search. Knowledge cut-off is far less important than it used to be.

Post reply on HN