Live data from Hacker News

Ask HN: SWEs how do you future-proof your career in light of LLMs?

news.ycombinator.com

131–140 of 1001 posts

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#131

Nothing because I’m a senior and LLM’s never provide code that pass my sniff test, and it remains a waste of time. I have a job at a place I love and get more people in my direct network and extended contacting me about work than ever before in my 20 year career. And finally I keep myself sharp by always making sure I challenge myself creatively. I’m not afraid to delve into areas to understand them that might look “…

> Nothing because I’m a senior and LLM’s never provide code that pass my sniff test, and it remains a waste of time. I am constantly surprised how prevalent this attitude is. ChatGPT was only just released in 2022. Is there some expectation that these things won't improve? > LLM’s never provide code that pass my sniff test This is ego speaking.

> Is there some expectation that these things won't improve?

Sure. But the expectation is quantitative improvement - qualitative improvement has not happened, and is unlikely to happen without major research breakthroughs.

LLMs are useful. They still need a lot of supervision & hand holding, and they'll continue to for a long while

And no, it's not "ego speaking". It's long experience. There is fundamentally no reason to believe LLMs will take a leap to "works reliably in subtle circumstances, and will elicit requirements as necessary". (Sure, if you think SWE work is typing keys and make some code, any code, appear, then LLM are a threat)

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#132
I think fully automated LLM code generation is an inherently flawed concept, unless the entire software ecosystem is automated and self-generating. I think if you carry out that line of thought to its extreme, you'd essentially need a single Skynet like AI that controls and manages all programming languages, packages, computer networks internally. And that's probably going to remain a sci-fi scenario.

Due to a training-lag, LLMs usually don't get the memo when a package gets updated. When these updates happen to patch security flaws and the like, people who uncritically push LLM-generated code are going to get burned. Software moves too fast for history-dependent AI.

The conceit of fully integrating all needed information in a single AI system is unrealistic. Serious SWE projects, that attempt to solve a novel problem or outperform existing solutions, require a sort of conjectural, visionary and experimental mindset that won't find existing answers in training data. So LLMs will get good at generating the billionth to-do app but nothing boundary pushing. We're going to need skilled people on the bleeding edge. Small comfort, because most people working in the industry are not geniuses, but there is also a reflexive property to the whole dynamic. LLMs open up a new space of application possibilities which are not represented in existing training data so I feel like you could position yourself comfortably by getting on board with startups that are actually applying these new technologies creatively. Ironically, LLMs are trained on last-gen code, so they obsolete yesterday's jobs. But you won't find any training data for solutions which have not been invented yet. So ironically AI will create a niche for new application development which is not served by AI.

Already if you try to use LLMs for help on some of the new LLM frameworks that came out recently like LangChain or Autogen etc, it is far less helpful than on something that has a long tailed distribution in the training data. (And these frameworks get updated constantly, which feeds into my last point about training-lag).

This entire deep learning paradigm of AI is not able solve problems creatively. When it tries to it "hallucinates".

Finally, I still think a knowledgable, articulate developer PLUS AI will consistently outperform an AI MINUS a knowledgable, articulate developer. More emphasis may shift onto "problem formulation", getting good at writing half natural language, half code pseudo-code prompts and working with the models conversationally.

There's a real problem too with model collapse, as AI generated code becomes more common, you remove the tails of the distribution, resulting in more generic code without a human touch. There's only so many cycles of retraining on this regurgitated data you can create before you start encountering not just diminishing returns, but damage the model. So I think LLMs will be self-limiting.

So all in all I think LLMs will make it harder to be a mediocre programmer who can just coast by doing highly standardized janitorial work, but it will create more value if you are trying to do something interesting. What that means for jobs is a mixed picture. Fewer boring, but still paying jobs, but maybe more work to tackle new problems.

I think only programmers understand the nuances of their field however and people on the business side are going to just look at their expense spreadsheets and charts, and will probably oversimplify and overestimate. But that could self-correct and they might eventually concede they're going to have to hire developers.

In summary, the idea that LLMs will completely take over coding logically entails an AI system that completely contains the entire software ecosystem within itself, and writes and maintains every endpoint. This is science fiction. Training lag is a real limitation since software moves too fast to constantly retrain on the latest updates. AI itself creates a new class of interesting applications that are not represented in the training data, which means there's room for human devs at the bleeding edge.

If you got into programming just because it promised to be a steady, well-paying job, but have no real interest in it, AI might come for you. But if you are actually interested in the subject and understand that not all problems have been solved, there's still work to be done. And unless we get a whole new paradigm of AI that is not data-dependent, and can generate new knowledge whole cloth, I wouldn't be too worried. And if that does happen, too, the whole economy might change and we won't care about dinky little jobs.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#133
I was advising this MBA student's nascent startup (with the idea I might technical cofound once they're graduating), and they asked about whether LLMs would help.

So I listed some ways that LLMs practically would and wouldn't fit into the workflow of the service they doing. And related it to a bunch of other stuff, including how to make the most of the precious customer real-world access they'd have, and generating a success in the narrow time window they have, and the special obligations of that application domain niche.

Later, I mentally replayed the conversation in my head (as I do), and realized they were actually probably asking about using an LLM to generate the startup's prototype/MVP for the software they imagined.

And also, "generating the prototype" is maybe the only value that an MBA student had been told a "technical" person could provide at this point. :)

That interpretation of the LLM question didn't even occur to me when I was responding. I could've easily whipped up the generic Web CRUD any developer could do and the bespoke scrape-y/protocol-y integrations that fewer developers could do, both to a correctness level necessarily higher than the norm (which was required by this particular application domain). In the moment, it didn't occur to me that anyone would think an LLM would help at all, rather than just be an unnecessary big pile of risk for the startup, and potential disaster in the application domain.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#134
post #41

Earlier quoted context omitted.

This is by far the best advice I've seen.

Except I suspect that Carmack would not be where he is today without a burning intellectual draw to programming in particular.

Exactly... I read "masters of doom" and Carmack didn't strike me as the product guy who cares about people needs. He was more like a coding machine.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#135
I think it's about evaluating the practical strengths and weaknesses of genAI for coding tasks, and trying to pair your skillset (or areas of potentially quick skill learning) with the weaknesses. Try using the tools and see what you like and dislike. For example I use a code copilot for autocomplete and it's saving my carpals; I'm not a true SWE more a code-y DS, but autocomplete on repetitive SQL or plotting cells is a godsend. It's like when I first learned vi macros, except so much simpler. Not sure what your domain is, but I'd wager there are areas that are similar for you; short recipes or utils that get reapplied in slightly different ways across lots of different areas. I would try and visualize what your job could look like if you just didn't have to manually type them; what types of things do you like doing in your work and how can you expand them to fill the open cycles?

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#137
I think the real world improvements will plateau and it'll take awhile for current enterprise just to adopt what is possible today but that is still going to cause quite a bit of change. You can imagine us going from AI Chat Bots with RAG on traditional datastores, to AI-enhanced but still human-engineered SaaS Products, to bespoke AI-generated and maintained products, to fully E2E AI Agentic products.

An example is do you tell the app to generate a python application to manage customer records or do you tell it "remember this customer record so other humans or agents can ask for it" and it knows how to efficiently and securely do that.

We'll probably see more 'AI Reliability Engineer' type roles which will likely be around building and maintain evaluation datasets, tracking and stomping out edge cases, figuring out human intervention/escalation, model routing, model distillation, Context-window vs Fine-tuning, and overall intelligence-cost management.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#138
post #121

Earlier quoted context omitted.

That’s short term thinking in my opinion. LLMs will not replace developers by writing better code: it’s the systems we work on that will start disappearing. Every SaaS, marketplace is at risk of extinction, superseded by AI agents communicating ad-hoc. Management and business software replaced by custom, one-off programs built by AI. The era of large teams painstakingly building specialized software for niche use cas…

you do realize that these so called "one-off" AI programs would need to be maintained? Most people paying for Saas are paying for the support/maintenance rather than features, which AI can't handle. No one will want to replace any Saas they depend on with a poorly generated variant that they want to maintain

Nah, you only write it and it runs by itself forever in the AI cloud.

Sometimes I wonder if people saying this stuff have actually worked in development at all.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#139

Nothing because I’m a senior and LLM’s never provide code that pass my sniff test, and it remains a waste of time. I have a job at a place I love and get more people in my direct network and extended contacting me about work than ever before in my 20 year career. And finally I keep myself sharp by always making sure I challenge myself creatively. I’m not afraid to delve into areas to understand them that might look “…

> If there’s ever a day where there’s an AI that can do these things, then I’ll gladly retire. But I think that’s generations away at best.

People really believe it will be generations before an AI will approach human level coding abilities? I don't know how a person could seriously consider that likely given the pace of progress in the field. This seems like burying your head in the sand. Even the whole package of translating high level ideas into robust deployed systems seems possible to solve within a decade.

I believe there will still be jobs for technical people even when AI is good at coding. And I think they will be enjoyable and extremely productive. But they will be different.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#140

Nothing because I’m a senior and LLM’s never provide code that pass my sniff test, and it remains a waste of time. I have a job at a place I love and get more people in my direct network and extended contacting me about work than ever before in my 20 year career. And finally I keep myself sharp by always making sure I challenge myself creatively. I’m not afraid to delve into areas to understand them that might look “…

> Nothing because I’m a senior and LLM’s never provide code that pass my sniff test, and it remains a waste of time. I am constantly surprised how prevalent this attitude is. ChatGPT was only just released in 2022. Is there some expectation that these things won't improve? > LLM’s never provide code that pass my sniff test This is ego speaking.

> Is there some expectation that these things won't improve?

Yes. The current technology is at a dead end. The costs for training and for scaling the network are not sustainable. This has been obvious since 2022 and is related to the way in which OpenAI created their product. There is no path described for moving from the current dead end technology to anything that could remotely be described as "AGI."

> This is ego speaking.

This is ignorance manifest.

Post reply on HN