Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The quality of AI-assisted software depends on unit of work management

blog.nilenso.com

Post reply on HN
Print view View original

51–60 of 127 posts

Re: The quality of AI-assisted software depends on unit of work management

#51

Post by strogonoff »

OptionOfT wrote: post #38

And the value of AI as pushed to us by these companies is in doing larger units of work. But... reviewing code is harder than writing code. Expressing how I want something to be done in natural language is incredibly hard. So over time I'm spending a lot of energy in those things, and only getting it 80% right. Not to mention I'm constantly in this highly suspicious mode, trying to pierce through the veil of my own p…

I somewhat dread code reviews. In order to properly evaluate the solution, you must first know what is the right solution to begin with. You must analyse the problem and arrive at it yourself. This is the brunt of the work, and yet you are not allowed the pleasant part of it: knowing that your work is shipped and feeling pride in it.

Working with LLM-generated code is mostly the same. The more sophisticated the autocomplete, the more mental overhead spent on understanding its output. There is an advantage: you are spared having to argue with a possibly defensive peer about what you believe is best. There is also a disadvantage: you do not feel like you are helping someone grow, and instead you are an unpaid (you are not paid for that in particular) contributor to a product by Microsoft (or similar) intended generally in longer term to push you and/or your peers out of the job. Additionally, there is no single mind that you can build rapport with and learn to understand the approaches and vibes of after a while.

Re: The quality of AI-assisted software depends on unit of work management

#52

Post by azinman2 »

liszper wrote: post #13
abtinf wrote: post #8

Earlier quoted context omitted.

liszper: > most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today Also liszper: oh, you tried the current approach and don’t agree with me? Well you just don’t know what you are doing.

Yes, exactly. Learning new things is hard. Personally it took me about 200 hours to get started, and since then ~2500 hours to get familiar with the advanced techniques, and now I'm very happy with the results, managing extremely large codebases with LLM in production. For context before that I had ~15 years of experience coding the traditional way.

You should write a blog post about your learnings. If you could even give some high level highlights here that’d be really helpful.

Re: The quality of AI-assisted software depends on unit of work management

#53

Post by dragonwriter »

nicce wrote: post #50
OptionOfT wrote: post #38

And the value of AI as pushed to us by these companies is in doing larger units of work. But... reviewing code is harder than writing code. Expressing how I want something to be done in natural language is incredibly hard. So over time I'm spending a lot of energy in those things, and only getting it 80% right. Not to mention I'm constantly in this highly suspicious mode, trying to pierce through the veil of my own p…

> Expressing how I want something to be done in natural language is incredibly hard Surprise, surprise… that is why programming languages were created.

Programming languages don’t solve that problem, since someone still has to explain what needs to be done in natural language unless the end customer is also the programmer.

Programming languages were created because of the different problem of “its very hard to get computers to understand natural language even if you know how to express what you want in it”.

Re: The quality of AI-assisted software depends on unit of work management

#54

Post by nicce »

dragonwriter wrote: post #53
nicce wrote: post #50

Earlier quoted context omitted.

> Expressing how I want something to be done in natural language is incredibly hard Surprise, surprise… that is why programming languages were created.

Programming languages don’t solve that problem, since someone still has to explain what needs to be done in natural language unless the end customer is also the programmer. Programming languages were created because of the different problem of “its very hard to get computers to understand natural language even if you know how to express what you want in it”.

I don't see the difference? Natural language simply was lacking the level of precision. We see natural language words and symbols everywhere in programming languages. Natural language was fine-tuned with improved accuracy. And optimised to reduce the amount of needed words. The difference in the precision between natural languages and programming languages was simply just too big, so you needed "an interpreter" to translate the level of precision from customer to computer.

Re: The quality of AI-assisted software depends on unit of work management

#55

Post by dingnuts »

angusturner wrote: post #15
liszper wrote: post #3

most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today thanks for the article, it's a good one

I think most SWEs do have a good idea where I work. They know that its a significant, but not revolutionary improvement. If you supervise and manage your agents closely on well scoped (small) tasks they are pretty handy. If you need a prototype and don't care about code quality or maintenance, they are great. Anyone claiming 2x, 5x, 10x etc is absolutely kidding themselves for any non-trivial software.

if the benefit is less than 2x then we're talking about AI assisted coding as being a very, very expensive IntelliSense. 1.x improvement just isn't much. My mind goes back to that study showing engineers claimed a 20% improvement and measured 20% reduction in productivity -- this is all encouraging me to just keep using traditional tools.

Re: The quality of AI-assisted software depends on unit of work management

#56

Post by dragonwriter »

nicce wrote: post #54
dragonwriter wrote: post #53

Earlier quoted context omitted.

Programming languages don’t solve that problem, since someone still has to explain what needs to be done in natural language unless the end customer is also the programmer. Programming languages were created because of the different problem of “its very hard to get computers to understand natural language even if you know how to express what you want in it”.

I don't see the difference? Natural language simply was lacking the level of precision. We see natural language words and symbols everywhere in programming languages. Natural language was fine-tuned with improved accuracy. And optimised to reduce the amount of needed words. The difference in the precision between natural languages and programming languages was simply just too big, so you needed "an interpreter" to tr…

Programming languages solve that machine code is hard for humans to work with for large problems and natural language (even when the meaning is perfectly clear) is very difficult (to the point of complete intractability with the level of knowledge and available hardware at the time programming languages were originally invented) to parse mechanically, a necessary first step in translation into machine code a computer could run.

Any problem with the difficulty of clearly expressing things in natural language (a real thing for which there have long been solutions between humans that are different than programming languages) was a problem that was technically unreachable at the user->machine interface because of that more fundamental problem for most of the history of computing (its arguable that LLMs are at the level where now it is potentially an issue with computers, but it took decades of having programming languages to be able to get the technical capacity to even experience the problem, it is not the problem programming languages address.)

Re: The quality of AI-assisted software depends on unit of work management

#57

Post by jonstewart »

liszper wrote: post #5
jonstewart wrote: post #4

I first tried getting specific with Claude Code. I made the Claude.md, I detailed how to do TDD, what steps it should take, the commands it should run. It was imperfect. Then I had it plan (think hard) and write the plan to a file. I’d clear context, have it read the plan, ask me questions, and then have it decompose the plan into a detailed plan of discrete tasks. Have it work its way through that. It would inevitab…

With all due respect, you sound like someone who is just getting familiar with these tools. 100 more hours spent with AI coding and you will be much more productive. Coding with AI is a slightly different skill from coding, similar how managing software engineers is different from writing software.

I think it has more to do with the kind of software I write and their requirements then it has to do with spending more time with this current tool. For some things it's great, but it's been a net productivity loss for me on my core coding responsibilities.

Re: The quality of AI-assisted software depends on unit of work management

#58

Post by danparsonson »

I seem to be in a minority but I find user stories or features to be really awkward and unnatural units of work for building software. Sure these things help to define the expected result but they shouldn't directly drive the development process. Imagine building a house that way - you don't build the living room, then the kitchen, then the bathroom etc.; you build floors, walls, the roof... The 'features' or use cases for the building arise out of the combination of different elements that were put into it, and usually right near the end of the build. The same is true for basically anything else that we build or create - if you're making a sculpture, do you finish working on one leg first before you move onto some other part?

Features are vertical slices through the software cake, but the cake is actually made out of horizontal layers. Creating a bunch of servings of cake and then trying to stick them together just results in a fragile mess that's difficult to work with and easy to break.

Re: The quality of AI-assisted software depends on unit of work management

#59

Post by jonstewart »

ryandrake wrote: post #29
liszper wrote: post #5

Earlier quoted context omitted.

With all due respect, you sound like someone who is just getting familiar with these tools. 100 more hours spent with AI coding and you will be much more productive. Coding with AI is a slightly different skill from coding, similar how managing software engineers is different from writing software.

I'm starting to kind of dig C.C. but you're right, it definitely feels like a very confident, very ambitious high schooler level developer with infinite energy. You really have to give it very small tasks and be constantly steering its direction. At the end of the day, I'm not sure I'm really saving that much time coaching Claude to do the job right vs. just writing the code myself, but it's definitely a neat trick.…

The thing I am calling BS on is that there's much productivity gain in giving it very small tasks and constantly steering its direction. For 80% of code, I'm faster than it if that's what I have to do. For debugging? For telling it to integrate a new tool? Port my legacy build system to something better? It's great at that, removes some important barriers to entry.

Re: The quality of AI-assisted software depends on unit of work management

#60

Post by nemomarx »

danparsonson wrote: post #58

I seem to be in a minority but I find user stories or features to be really awkward and unnatural units of work for building software. Sure these things help to define the expected result but they shouldn't directly drive the development process. Imagine building a house that way - you don't build the living room, then the kitchen, then the bathroom etc.; you build floors, walls, the roof... The 'features' or use cas…

They make more sense if you think about adding to an existing house. "I want to open up this wall to serve X function" kinda work

they're very well adapted to legacy enterprise work

Post reply on HN