Live data from Hacker News

Effects of Gen AI on High Skilled Work: Experiments with Software Developers

papers.ssrn.com

391–400 of 532 posts

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#391

I sometimes wonder about whether the decline in IT worker quality is down to companies trying to force more and more roles onto each worker to reduce headcount. Developers, Operations, and Security used to be dedicated roles. Then we made DevOps and some businesses took that to mean they only needed 2/3 of the headcount, rather than integrating those teams. Then we made DevSecOps, and some businesses took that to mea…

Remember 10x? "Are you a 10x developer?!" lol

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#392

I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-driven efforts. Because my personal experience has involved a lot of that in one of the companies listed in the study. Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them th…

Code quality is the hardest thing to measure. Seems like they were measuring commits, pull-requests, builds, and build success rate. This sort of gets at that, but is probably inadequate.

The few attempts I've made at using genAI to make large-scale changes to code have been failures, and left me in the dark about the changes that were made in ways that were not helpful. I needed suggestions to be in much smaller chunks. paragraph sized. Right now I limit myself to using the genAI line completion suggestions in Pycharm. It very often guesses my intentions and so actually is helpful, particularly when laboriously typing out lots of long literals, e.g. keys in a dictionary.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#393

Earlier quoted context omitted.

While you have a good point, I think the experts also branched off with some unreasonable requirements. I remember reading Yegge's blog post, years ago, saying that an engineer needed to know bitwise operators, otherwise they were not good enough. I don't know. Curiosity, passion, focus, creative problem solving seem to me much more important criteria for an engineer to have, rather than bitwise operations. An engine…

This is something I have some expertise in (I run a recruiting company that uses a standardized interview, and I used to run assessments for another). It's also something I've thought a lot about. There is absolutely truth to what you're saying. But like most obvious observations that aren't playing themselves out, there's more to it than that. ----- One: curiosity, passion, and focus matter a lot. Every company that…

This has been one of the most interesting and insightful replies I've ever got in a discussion. Thank you for taking the time and explaining your points. It really resonates with what I've seen in real life.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#394
post #29

I wonder if the study includes the technical debt that more experienced developers had to tackle after the less experienced devs have contributed their AI-driven efforts. Because my personal experience has involved a lot of that in one of the companies listed in the study. Also, I've personally seen more interest in AI in devs that have little interest in technology, but a big interest in delivering. PMs love them th…

I am curious about this also. I have now seen multiple PR's that I had to review that a method was clearly completely modified by AI with no good reason and when asked why something was changed we just got silence. Not exaggerating, literal silence and then trying to ignore the question and explain the thing we were first asking them to do. Clearly having no idea what is actually in this PR. This was done because we…

When using code assist, I've occasionally found some perplexing changes to my code I didn't remember making (and wouldn't have made). Can be pretty frustrating.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#395
post #99

Earlier quoted context omitted.

They're cutting headcount because they have no conception of how to make a good product, so reducing costs is the only of making the bottom line go up. That's across the board, from the startups whose business plan is to be acquired at all costs, to the giant tech companies, whose business plan is to get monopoly power first, then figure out how to extract money later.

> They're cutting headcount because they have no conception of how to make a good product, so reducing costs is the only of making the bottom line go up. The field is wide open for a startup to do it right. Why not start one?

I think there's a misunderstanding. They _are_ doing it right, it's just that the right way doesn't involve building a good product, because you aren't any more valuable. There's no way a startup can make more money than the guy speedrunning acquisition, making gonzo dollars fast >>> making (probably less) gonzo dollars in the long term.

If you think there's a problem with this model (and based on your wording of "doing it right", this seems to be the case), it's largely in the incentive structure, not the actors.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#396
I think that over time, AI will tend to make all programmers into average programmers.

For those who are beginners, it can bring their skills up and make them look like better developers than they are.

More insidiously, expert programmers who overuse of AI might also regress to the mean as their skills deteriorate.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#397
post #354

Earlier quoted context omitted.

For myself it's an incredible tool for learning. I learn both broader and deeper using chat tools. If anything it gives me a great "sounding board" for exploring a subject and finding additional resources. E.g last night I setup my first Linux raid. A task that isn't too hard, but following a tutorial or just "reading the docs" isn't particularly helpful given it takes a few different tools (mount, umount, fstab, blk…

You’ve given two examples for “broad”, but none for “deep”. I’ve also used LLMs for setting up my homelab, and they were really helpful since I was basically at beginner level in most linux admin topics (still am). But trying to eg setup automatic snapshot replications for my zfs pool had me go back to reading blog posts, as ChatGPT just couldn’t provide a solution that worked for me.

I think one of the catch-22s of LLMs is that using it as a fancy search index (which is the dev assistant use-case) is that the information it surfaces is hugely dependent on what words you use and it matches energy. If you don't know the words you'll get very beginner oriented content and you can get it to surface deeper knowledge iff you know the shibboleths, but it's annoying. One dumb trick that's been unreasonably useful is just copy-pasting barely-related source code, but at the level you're trying to understand and then just asking an unrelated question— just yank that search vector way over to the region where you think the good information lives.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#398

Earlier quoted context omitted.

Easy: exclude developers who try it. Learning--be it a new codebase, a new programming language, a new database--takes time. You're not going to be as productive until you learn how. That's fine! Cheating on your homework with an LLM should not be something we celebrate, though, because the learner will never become productive that way, and they won't understand the code they're submitting for review.

The truth is that the tools are actually quite good already. If you know what you are doing they will 10-20x your productivity. Ultimately not adopting them will religate you to the same fate as assembly programmers. Sure there are place for it, but you won't be able to get near as much functionally done in the same amount of time and there won't be as much demand for it.

We'll see. Get back to me sometime in 2034 and we can compare notes.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#399
post #368

Earlier quoted context omitted.

Without judgement, this feels like a switch up. It seems to me that the prior author did not suggest they were necessary, but instead ambient, available, and interesting. Indeed for many they are not useful. At the same time, that might be a good indication of passion: a useless but foundational thing you learn despite having zero economic pressure to do so. In the domains you have worked on, what are examples of suc…

It is not a switch up. The author said that an engineer would find these during their journey, and I am asking when. I don't have a strongly held opinion here, I literally want to know when. I am curious about it.

> I literally want to know when

When the curious engineer is reading a book about their choosen programing language. Most likely in the part which describes operators.

Or when they parse something stored in binary.

Re: Effects of Gen AI on High Skilled Work: Experiments with Software Developers

#400

Earlier quoted context omitted.

It is not a switch up. The author said that an engineer would find these during their journey, and I am asking when. I don't have a strongly held opinion here, I literally want to know when. I am curious about it.

Bitmasks, enumerating combinations (subsets), parsing binary data, ...

Cryptography, parity checking, sorting, image processing, optimization...
Post reply on HN