Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The quality of AI-assisted software depends on unit of work management

blog.nilenso.com

Post reply on HN
Print view View original

21–30 of 127 posts

Re: The quality of AI-assisted software depends on unit of work management

#21

Post by Filligree »

blibble wrote: post #9
liszper wrote: post #3

most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today thanks for the article, it's a good one

> most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today yes, just as was said each and every previous time OpenAI/anthropic shit out a new model "now it doesn't suck!"

Each and every new model expands the scope of what you can do. You notice that, get elated when things that didn’t work start working, then three weeks later the honeymoon period is over and you notice the remaining limits.

The hedonic treadmill ensures it feels the same way each time.

But that doesn’t mean the models aren’t improving, nor that the scope isn’t expanding. If you compare today’s tools to those a year ago, the difference is stark.

Re: The quality of AI-assisted software depends on unit of work management

#22

Post by liszper »

sarchertech wrote: post #18
liszper wrote: post #13

Earlier quoted context omitted.

Yes, exactly. Learning new things is hard. Personally it took me about 200 hours to get started, and since then ~2500 hours to get familiar with the advanced techniques, and now I'm very happy with the results, managing extremely large codebases with LLM in production. For context before that I had ~15 years of experience coding the traditional way.

How many users is production and how large is extremely large.

200k DAU, 7 million registered, ~50 microservices, large monorepo

Re: The quality of AI-assisted software depends on unit of work management

#23

Post by angusturner »

kibwen wrote: post #17
liszper wrote: post #3

most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today thanks for the article, it's a good one

Last week I wanted to generate some test data for some unit tests for a certain function in a C codebase. It's an audio codec library, so I could have modified the function to dump its inputs to disk and then run the library on any audio file and then hardcoded the input into the unit tests. Instead, I decided I wanted to save a few bytes and wanted to look at generating dummy data dynamically. I wanted to try out Cl…

I feel this. I've had a few tasks now where in honest retrospect I find myself asking "did that really speed me up". Its a bit demoralising cause not only do you waste time, you have a worse mental model of the resulting code and feel less sense of ownership over the result.

Brainstorming, ideation and small, well defined tasks where I can quickly vet the solution : these feel like the sweet spot for current frontier model capabilities.

(Unless you are pumping out some sloppy React SPA that you don't care about anything except get it working as fast as possible - fine, get Claude code to one shot it)

Re: The quality of AI-assisted software depends on unit of work management

#24

Post by Filligree »

kibwen wrote: post #17
liszper wrote: post #3

most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today thanks for the article, it's a good one

Last week I wanted to generate some test data for some unit tests for a certain function in a C codebase. It's an audio codec library, so I could have modified the function to dump its inputs to disk and then run the library on any audio file and then hardcoded the input into the unit tests. Instead, I decided I wanted to save a few bytes and wanted to look at generating dummy data dynamically. I wanted to try out Cl…

There’s been a lot of noise about Claude performance degradation, and the current best option is probably Codex, but this still surprises me. It sounds like it succeeded on the hard part, then stumbled on the easy bit.

Just two questions, if you don’t mind satisfying my curiosity.

- Did you tell it to write C? Or better yet, what was the prompt? You can use Claude --resume to easily find that.

- Which model? (Sooner or Opus)? Though I’d have expected either one to work.

Re: The quality of AI-assisted software depends on unit of work management

#25

Post by sarchertech »

liszper wrote: post #22
sarchertech wrote: post #18

Earlier quoted context omitted.

How many users is production and how large is extremely large.

200k DAU, 7 million registered, ~50 microservices, large monorepo

You have 50 microservices for 200k daily users?

Let me guess this has something to do with AI?

Re: The quality of AI-assisted software depends on unit of work management

#26

Post by rco8786 »

liszper wrote: post #3

most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today thanks for the article, it's a good one

I still use Claude Code and Cursor and tbh still run into a lot of the same issues. Hallucinating code, hallucinating requirements, even when scoped to a very simple "make this small change".

It's good enough that it helps, particularly in areas or languages that I'm unfamiliar with. But I'm constantly fighting with it.

Re: The quality of AI-assisted software depends on unit of work management

#27

Post by liszper »

dgfitz wrote: post #16
liszper wrote: post #11

Earlier quoted context omitted.

I agree with your point. I think this is the reason why most developers still don't get it, because AI coding ultimately requires a "higher level" methodology.

"Hacker culture never took root in the 'AI' gold rush because the LLM 'coders' saw themselves not as hackers and explorers, but as temporarily understaffed middle-managers." [0] This, this is you. This is the entire charade. It seems poetic somehow. [0] https://news.ycombinator.com/item?id=45123094

I see myself as a hacker.

Re: The quality of AI-assisted software depends on unit of work management

#28

Post by realusername »

liszper wrote: post #3

most SWE folks still have no idea how big the difference is between the coding agents they tried a year ago and declared as useless and chatgpt 5 paired with Codex or Cursor today thanks for the article, it's a good one

I tried again recently and I see absolutely no difference. If there's been some improvement, it's very subtle.

There's a big difference with their benchmarks and real world coding.

Re: The quality of AI-assisted software depends on unit of work management

#29

Post by ryandrake »

liszper wrote: post #5
jonstewart wrote: post #4

I first tried getting specific with Claude Code. I made the Claude.md, I detailed how to do TDD, what steps it should take, the commands it should run. It was imperfect. Then I had it plan (think hard) and write the plan to a file. I’d clear context, have it read the plan, ask me questions, and then have it decompose the plan into a detailed plan of discrete tasks. Have it work its way through that. It would inevitab…

With all due respect, you sound like someone who is just getting familiar with these tools. 100 more hours spent with AI coding and you will be much more productive. Coding with AI is a slightly different skill from coding, similar how managing software engineers is different from writing software.

I'm starting to kind of dig C.C. but you're right, it definitely feels like a very confident, very ambitious high schooler level developer with infinite energy. You really have to give it very small tasks and be constantly steering its direction. At the end of the day, I'm not sure I'm really saving that much time coaching Claude to do the job right vs. just writing the code myself, but it's definitely a neat trick.

The difference from an actual junior developer, of course, is that the human junior developer learns from his mistakes and gets better, but Claude seems to be stuck at the level of expertise of its model, and you have to wait for the model to improve before Claude improves.

Post reply on HN