Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: SWEs how do you future-proof your career in light of LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 1001 posts

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#51

Post by paulyy_y »

rvz wrote: post #10

> My prediction is that junior to mid level software engineering will disappear mostly, while senior engineers will transition to be more of a guiding hand to LLMs output, until eventually LLMs will become so good, that senior people won't be needed any more. It is more like across the board beyond engineers, including both junior and senior roles. We have heard first hand from Sam Altman that in the future that Agen…

Have you checked out the reception to Devin last week? The only thing it's going after is being another notch on the leaderboard of scams, right next to the Rabbit R1.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#52

Post by tinthedev »

Real software engineering is as far from "only writing code", as construction workers are from civil engineering.

> So, fellow software engineers, how do you future-proof your career in light of, the inevitable, LLM take over?

I feel that software engineering being taken over by LLM is a pipe dream. Some other, higher, form of AI? Inevitably. LLMs, as current models exists and expand? They're facing a fair few hurdles that they cannot easily bypass.

To name a few: requirement gathering, scoping, distinguishing between different toolsets, comparing solutions objectively, keeping up with changes in software/libraries... etc. etc.

Personally? I see LLMs tapering off in new developments over the following few years, and I see salesmen trying to get a lot of early adopters to hold the bag. They're overpromising, and the eventual under-delivery will hurt. Much like the AI winter did.

But I also see a new paradigm coming down the road, once we've got a stateful "intelligent" model that can learn and adapt faster, and can perceive time more innately... but that might take decades (or a few years, you never know with these things). I genuinely don't think it'll be a direct evolution of LLMs we're working on now. It'll be a pivot.

So, I future-proof my career simply: I keep up with the tools and learn how to work around them. When planning my teams, I don't intend to hire 5 juniors to grind code, but 2 who'll utilize LLMs to teach them more.

I ask more of my junior peers for their LLM queries before I go and explain things directly. I also teach them to prompt better. A lot of stuff we've had to explain manually in the past can now be prompted well, and stuff that can't - I explain.

I also spend A LOT of time teaching people to take EVERYTHING AI-generated with generous skepticism. Unless you're writing toys and tiny scripts, hallucinations _will_ waste your time. Often the juice won't be worth the squeeze.

More than a few times I've spent a tedious hour navigating 4o's or Claude's hallucinated confident failures, instead of a pleasant and productive 45 minutes writing the code myself... and from peer discussions, I'm not alone.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#53

Post by m_ke »

angoragoats wrote: post #36
m_ke wrote: post #31

Earlier quoted context omitted.

Those models will be here within a year. Long context is practically a solved problem and there's a ton of work now on test time reasoning motivated by o1 showing that it's not that hard to RL a model into superhuman performance as long as the task is easy / cheap to validate (and there's works showing that if you can define the problem you can use an LLM to validate against your criteria).

I intentionally glossed over a lot in my first comment, but I should clarify that I don't believe that increased context size or RL is sufficient to solve the problem I'm talking about. Also "as long as the task is easy / cheap to validate" is a problematic statement if we're talking about the replacement of senior software engineers, because problem definition and development of validation criteria are core to the d…

Sorry I wasn't clear enough, the cheap to validate part is only needed to train a large base model that can handle writing individual functions / fix bugs. Planning a whole project, breaking it down into steps and executing each one is not something that current LLMs struggle at.

Here's a recipe for a human level LLM software engineer:

1. Pretrain an LLM on as much code and text as you can (done already)

2. Fine tune it on synthetic code specific tasks like: (a) given a function, hide the body, make the model implement it and validate that it's functionally equivalent to the target function (output matching), can also have an objective to optimize the runtime of the implementation (b) introduce bugs in existing code and make the LLM fix it, (c) make LLM make up problems, write tests / spec for it, then have it attempt to implement it many times until it comes up with a method that passes the tests, (d-z) a lot of other similar tasks that use linters, parsers, AST modifications, compilers, unit tests, specs validated by LLMs, profilers to check that the produced code is valid

3. Distill this success / failure criteria validator to a value function that can predict probability of success at each token to give immediate reward instead of requiring full roll out, then optimize the LLM on that.

4. At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations.

See papers like: https://arxiv.org/abs/2409.15254 or slides from NeurIPS that I mentioned here https://news.ycombinator.com/item?id=42431382

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#54

Post by m_ke »

ThrowawayR2 wrote: post #47
m_ke wrote: post #23

I've been thinking about this a bunch and here's what I think will happen as cost of writing software approaches 0: 1. There will be way more software 2. Most people / companies will be able to opt out of predatory VC funded software and just spin up their own custom versions that do exactly what they want without having to worry about being spied on or rug pulled. I already do this with chrome extensions, with the h…

What's the equivalent of @justsayinmice for NeurIPS papers? A lot of things in papers don't pan out in the real world.

There's a lot of work showing that we can reliably get to or above human level performance on tasks where it's easy to sample at scale and the solution is cheap to verify.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#56

Post by throwaway_43793 »

code_for_monkey wrote: post #16

Im hoping I can transition to some kind of product or management role since frankly Im not that good at coding anyways (I dont feel like I can pass a technical interview anymore, tbh.) I think a lot of engineers are in for some level of rude awakening. I think a lot of engineers havent applied some level of business/humanities thinking in this, and I think a lot of corporations care less about code quality than even…

What's your plan to transition into product/management?

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#57

Post by hnthrowaway6543 »

throwaway_43793 wrote: post #44
hnthrowaway6543 wrote: post #40

Earlier quoted context omitted.

> bust most engineers are writing code most of the time. the physical act of writing code is different than the process of developing software. 80%+ of the time working on a feature is designing, reading existing code, thinking about the best way to implement your feature in the existing codebase, etc. not to mention debugging, resolving oncall issues, and other software-related tasks which are not writing code GPT i…

Everything you have described, apart from on-call, I think LLMs can/will be able to do. Explaining code, reviewing code, writing code, writing test, writing tech docs. I think we are approaching a point where all these will be done by LLMs. You could argue about architecture/thinking about the correct/proper implementations, but I'd argue that for the past 7 decades of software engineering, we are not getting close t…

> Explaining code, reviewing code, writing code, writing test, writing tech docs.

people using GPT to write tech docs at real software companies get fired, full stop lol. good companies understand the value of concise & precise communication and slinging GPT-generated design docs at people is massively disrespectful to people's time, the same way that GPT-generated HN comments get downvoted to oblivion. if you're at a company where GPT-generated communication is the norm you're working for/with morons

as for everything else, no. GPT can explain a few thousand lines of code, sure, but it can't explain how every component in a 25-year-old legacy system with millions of lines and dozens/scores of services works together. "more context" doesn't help here

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#58

Post by angoragoats »

m_ke wrote: post #53
angoragoats wrote: post #36

Earlier quoted context omitted.

I intentionally glossed over a lot in my first comment, but I should clarify that I don't believe that increased context size or RL is sufficient to solve the problem I'm talking about. Also "as long as the task is easy / cheap to validate" is a problematic statement if we're talking about the replacement of senior software engineers, because problem definition and development of validation criteria are core to the d…

Sorry I wasn't clear enough, the cheap to validate part is only needed to train a large base model that can handle writing individual functions / fix bugs. Planning a whole project, breaking it down into steps and executing each one is not something that current LLMs struggle at. Here's a recipe for a human level LLM software engineer: 1. Pretrain an LLM on as much code and text as you can (done already) 2. Fine tune…

> At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations.

If you're saying that it takes one software engineer one hour to produce comprehensive criteria that would allow this whole pipeline to work for a non-trivial software engineering task, this is where we violently disagree.

For this reason, I don't believe I'll be convinced by any additional citations or research, only by an actual demonstration of this working end-to-end with minimal human involvement (or at least, meaningfully less human involvement than it would take to just have engineers do the work).

edit: Put another way, what you describe here looks to me to be throwing a huge number of "virtual" low-skilled junior developers at the task and optimizing until you can be confident that one of them will produce a good-enough result. My contention is that this is not a valid methodology for reproducing/replacing the work of senior software engineers.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#59

Post by polotics »

This question is so super weird, because:

Ask an LLM to generate you 100 more lines of code, no problem you will get something. Ask the same LLM to look at 10000 lines of code and intelligently remove 100... good luck with that!

seriously, I tried uploading some (but not all) source code of my company to our private Azure OpenAI GPT 4o for analysis, as a 48 MB cora-generated context file, and really the usefulness is not that great. And don't get me started about Copilot's suggestions.

Someone really has to know their way around the beast, and LLM's cover a very very small part of the story.

I fear that the main effect of LLMs will be that developers that have already for so long responded to their job-security fears with obfuscation and monstrosity... will be empowered to produce even more of that.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#60

Post by m_ke »

angoragoats wrote: post #58
m_ke wrote: post #53

Earlier quoted context omitted.

Sorry I wasn't clear enough, the cheap to validate part is only needed to train a large base model that can handle writing individual functions / fix bugs. Planning a whole project, breaking it down into steps and executing each one is not something that current LLMs struggle at. Here's a recipe for a human level LLM software engineer: 1. Pretrain an LLM on as much code and text as you can (done already) 2. Fine tune…

> At test time use this final LLM to produce multiple versions until one passes the criteria, for the cost of an hour of a software engineer you can have an LLM produce millions of different implementations. If you're saying that it takes one software engineer one hour to produce comprehensive criteria that would allow this whole pipeline to work for a non-trivial software engineering task, this is where we violently…

That's not what I'm saying at all. I'm saying that there's a trend showing that you can improve LLM performance significantly by having it generate multiple responses until it produces one that meets some criteria.

As an example, huggigface just posted an article showing this for math, where with some sampling you can get a 3B model to outperform a 70B one: https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling...

Formalizing the criteria is not as hard as you're making it out to be. You can have an LLM listen to a conversation with the "customer", ask follow up questions and define a clear spec just like a normal engineer. If you doubt it open up chatGPT, tell it you're working on X and ask it to ask you clarifying questions, then come up with a few proposal plans and then tell it which plan to follow.

Post reply on HN