Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: SWEs how do you future-proof your career in light of LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

581–590 of 1001 posts

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#581

Post by deathanatos »

paulcole wrote: post #571
deathanatos wrote: post #206

Earlier quoted context omitted.

> This is ego speaking. No, it really isn't. Repeatedly, the case is that people are trying to pass off GPT's work as good without actually verifying the output. I keep seeing "look at this wonderful script GPT made for me to do X", and it does not pass code review , and is generally extremely low quality. In one example, a bash script was generated to count number SLoC changed by author; it was extremely convoluted,…

> No, it really isn't It really is. Either that or you’re not thinking about what you’re saying. Imagine code passes your rigorous review. How do you know that it wasn’t from an LLM? If it’s because you know that you only let good code pass your review and you know that LLMs only generate bad code, think about that a bit.

> Imagine code passes your rigorous review. How do you know that it wasn’t from an LLM?

That's not what I'm saying (and it's a strawman; yes, presumably some LLM code would escape review and I wouldn't know it's from an LLM, though I find that unlikely, given…) — what I'm saying is of LLM generated code that is reviewed, what is the quality & correctness of the reviewed code? And it's resoundingly (easily >90%) crap.

Obviously we can't sample from unknown-authorship … nor am I; I'm sampling problems that I and others run through an LLM, and the output thereof.

The other facet of this point is that I believe a lot of the craze that users using the LLM have is driven by them not looking closely at the output; if you're just deriving code from the LLM, chucking it over the wall, and calling it a day (as was the case from one of the examples in the comment above) — you're perceiving the LLM as being useful, when it fact it is leaving bugs that you're either not attributing to it, someone else is cleaning up (again, that was the case in the above example), etc.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#582

Post by sagarpatil »

adamredwoods wrote: post #577

LLM token size will have to increase a lot to digest a large system of code. Then there's the companies that own LLMs that are large enough to do excellent coding. Consider "have" and "have nots", those that have the capital to incorporate these amazing LLMs and those that do not.

2 million token size is not enough?

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#583

Post by skydhash »

grugagag wrote: post #497
epolanski wrote: post #432

Earlier quoted context omitted.

I don't see it. Trading is about doing very specific math in a very specific scenario with known expectations. Software engineering is anything but like that.

Yes, software engineering is different in many areas but today a lot of it is CRUD and plumbing. While SW engineering will not die it will certainly transform a lot, quite possibly there will be fewer generalists than today and more specialized branches will pop out - or maybe being a generalist will require one to be familiar many new areas. Likely the code we write today will go the same way writing assembly code w…

> software engineering is different in many areas but today a lot of it is CRUD and plumbing

Which you can do away in a few days with frameworks and code reuse. The rest of the time is mostly spent on understanding the domain, writing custom components, and fixing bugs.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#584

Post by koe123 »

Are LLMs / AI attaining better results than the data they were trained on? For me, the answer is no: LLMs are always imperfectly modeling the underlying distribution of the training dataset.

Do we have sufficient data that spans the entire problem space that SWE deals with? Probably not, and even if we did it would still be imperfectly modeled.

Do we have sufficient data to span the space of many routine tasks in SWE? It seems so, and this is where the LLMs are really nice: e.g., scripting, regurgitating examples, etc.

So to me, much like previous innovation, it will just shift job focus away from the things the innovation can do well, rather than replacing the field as a whole.

One pet theory I have is that we currently suck at assessing model performance. Sure, vibes-based analysis of the outputs of the model make them look amazing. But is that not the literal point of RLHF? But how good are these outputs really?

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#585

Post by disconcision »

crystal_revenge wrote: post #564
manquer wrote: post #450

Earlier quoted context omitted.

> LLMs can pass CFA III. Everyone cites these kind of examples as LLM beating some test or other as some kind of validation. It isn’t . To me that just tells that the tests are poor, not the LLMs are good. Designing and curating a good test is hard and expensive. Certifying and examination bodies often use knowledge as a proxy to understanding or reasoning or any critical thinking skills.they just need to filter enou…

> To me that just tells that the tests are poor, not the LLMs are good. I mean the same is true of leetcode but I know plenty of mediocre engineers still making ~$500k because they learned how to grind leetcode. You can argue that the world is unjust till you're blue in the face, but it won't make it a just world.

sure, but if companies find that llm performance on tests is less correlated with actual job performance that human test performance, then that means the test might not be not a useful metric to inform automation decisions

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#586

Post by steve_adams_86 »

aprilthird2021 wrote: post #529
627467 wrote: post #436

Earlier quoted context omitted.

It's interesting you say this because in my current process to learn to build apps for myself I first try build mermaid diagrams aided by LLM. And when I'm happy, i then ask it to generate the code for me based on these diagrams. I'm no SWE and probably never will be. SWE probably don't consider what I do "building an app" but I don't really care

Diagramming out what needs to be built is often what some of the highest paid programmers do all day

This is what keeps crossing my mind.

Even trivial things like an ETL pipeline for processing some data at my work fall into this category. It seemed trivial on its surface, but when I spoke to everyone about what we were doing with it and why (and a huge amount of context regarding the past and future of the project), the reason the pipeline wasn’t working properly was both technically and contextually very complex.

I worked with LLMs on solving the problems (I always do, I guess I need to “stay sharp”), and they utterly failed. I tried working from state machine definitions, diagrams, plain English, etc. They couldn’t pick up the nuances at all.

Initially I thought I must be over complicating the pipeline, and there must be some way to step it back and approach it more thoughtfully. This utterly failed as well. LLMs tried to simplify it by pruning entire branches of critical logic, hallucinating bizarre solutions, or ignoring potential issues like race conditions, parallel access to locked resources, etc. entirely.

It has been a bit of an eye opener. Try as I might, I can’t get LLMs to use streams to conditionally parse, group, transform, and then write data efficiently and safely in a concurrent and parallel manner.

Had I done this with an LLM I think the result eventually could have worked, but the code would have been as bad as what we started with at best.

Most of my time on this project was spent planning and not programming. Most of my time spent programming was spent goofing around with LLM slop. It was fun, regardless.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#587

Post by adamredwoods »

sagarpatil wrote: post #582
adamredwoods wrote: post #577

LLM token size will have to increase a lot to digest a large system of code. Then there's the companies that own LLMs that are large enough to do excellent coding. Consider "have" and "have nots", those that have the capital to incorporate these amazing LLMs and those that do not.

2 million token size is not enough?

Probably not for a large monorepo, definitely not ours.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#588

Post by nosbo »

Volrath89 wrote: post #297

(10+ years of experience here) I will be starting training for commercial pilot license next year. The pay is much less than one of a software engineer but I think this job is already done for most of us, only the top 5% will survive. I don’t think I’m part of that top and don’t want to go to management or PO roles so I am done with tech

You do realise planes already fly themselves right. And at some airports even land and takeoff.

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#589

Post by richardw »

RealityVoid wrote: post #561
cle wrote: post #552

Earlier quoted context omitted.

This back and forth is so tiring. I have built web services used by many Fortune 100 companies, built and maintained and operated them for many years. But I'm not doing that anymore. Now I'm working on my own, building lots of prototypes and proof-of-concepts. For that I've founding LLMs to be extremely helpful and time-saving. Who the hell cares if it's not maintainable for years? I'll likely be throwing it out anyw…

I agree with you and I don't think OP disagrees either. The point if contention is the inevitable and immediate death of programming as a profession.

Surely nobody has that binary a view?

What are the likely impacts over the next 1, 5, 10, 20 years. People getting into development now have the most incredible technology to help them skill up, but also more risk than we had in decades past. There's a continuum of impact and it's not 0 or 100%, and it's not immediate.

What I consider inevitable: humans will keep trying to automate anything that looks repeatable. As long as there is a good chance of financial gain from adding automation, we'll try it. Coding is now potentially at risk of increasing automation, with wildcards on "how much" and "what will the impact be". I'm extremely happy to have nuanced discussions, but I balk at both extremes of "LLMs can scale to hard AGI, give up now" and "we're safe forever". We need shorthand for our assumptions and beliefs so we can discuss differences on the finer points without fixating on obviously incorrect straw men. (The latter aimed at the general tone of these discussions, not your comment.)

Re: Ask HN: SWEs how do you future-proof your career in light of LLMs?

#590

Post by ethbr1 »

9cb14c1ec0 wrote: post #568
wewtyflakes wrote: post #567

Earlier quoted context omitted.

You're fighting the tide with a broom.

No, I'm fighting for the software I sell my customers to be actually reliable.

That ship seems to have sailed at the same time boxed software went extinct.

How many companies still have dedicated QA orgs with skilled engineers? How many SaaS solutions have flat out broken features? Why is SRE now a critical function? How often do mobile apps ship updates? How many games ship with a day zero patch?

The industries that still have reliable software are because there are regulatory or profit advantages to reliability -- and that's not true for the majority of software.

Post reply on HN