Live data from Hacker News

A coder considers the waning days of the craft

newyorker.com

481–490 of 1001 posts

Re: A coder considers the waning days of the craft

#481

Earlier quoted context omitted.

How long ago would you have considered this discussion ridiculous? How long till GPT-N will be churning out solutions faster than you can read them? It's useless for me now as well, but I'm pretty sure I'll be doomed professionally in the future.

Not necessarily. Every hockey stick is just the beginning of an s-curve. It will saturate, probably sooner than you think.

Some parts of AI will necessarily asymptote to human-level intelligence because of a fixed corpus of training data. It's hard to think AI will become a better creative writer than the best human creative writers, because the AI is trained on their output and you can't go much further than that.

But in areas where there's self-play (e.g. Chess, and to a lesser extent, programming), there is no good reason to think it'll saturate, since there isn't a limit on the amount of training data.

Re: A coder considers the waning days of the craft

#482
post #169

Earlier quoted context omitted.

I’ve never found GPT-4 capable of producing a useful solution in my niche of engineering. When I’m stumped, it’s usually on a complex and very multi-faceted problem where the full scope doesn’t fit into the human brain very well. And for these problems, GPT will produce some borderline unworkable solutions. It’s like a jack of all trades and master of none in code. It’s knowledge seems a mile wide and an inch deep. G…

What’s your niche? I think much of using it well is understanding what it can and can’t do (though of course this is a moving target). It’s great when the limiting factor is knowledge of APIs, best practices, or common algorithms. When the limiting factor is architectural complexity or understanding how many different components of a system fit together, it’s less useful. Still, I find I can often save time on more d…

My niche is in video game programming, and I am very specialized in a specific area. So I might ask things like how would one architect a certain game system with a number of requirements, to meet certain player expectations, and be compatible with a number of things.

Unfortunately, it hasn’t been helpful once, and often due to the same reason - when the question gets specific enough, it hallucinates because it doesn’t know, just like in the early days.

Moreover, I am a domain expert in my area, so I only ask for help when the problem is really difficult. For example, when it would take me several days to come up with an answer and a few more weeks to refine it.

Game development has a lot of enthusiasts online sharing material, but most of this material is at junior to intermediate level. You very quickly run out of resources for questions at a principal level, even if you know the problems you have have been solved in other AAA companies.

You have to rely on your industry friends, paid support from middleware providers, rare textbooks, conferences, and, on the off-chance that anything useful got scooped up into the training data set - GPT. But GPT has been more like wishful thinking for me.

Re: A coder considers the waning days of the craft

#483

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

Just like the author suggests, sometimes you have to tailor your question to ChatGPT, for it to succeed.

As long as this is true, ChatGPT is going to be a programmer's tool, not a programmer's replacement. I know that my job as I know it will vanish before I enter retirement age, but I don't worry it will happen in the next few years because of this.

Re: A coder considers the waning days of the craft

#484

Earlier quoted context omitted.

The author isn't a professional programmer. He has a couple side projects and doesn't seem very good at coding.

That’s interesting. I think there’s a real shift going on in terms of what coding actually means I’ve never produced so much code so quickly and it’s disconcerting to the part of me that wants to feel like I earned the outcome on a pretty deep level because I can just generate shit that works so fast and then just edit it until I get exactly what I’m looking for and it feels like cheating in a weird way. I still don’…

Honestly, this reminds me of my early days when auto-complete and OOP was the best thing. It's why OOP proliferated, you type in the name of the object and get a list of completions in visual studio. No need to read docs or understand what you are doing, just find the method you need on that object. This new copilot coding reminds me of that, throwing shit and see what sticks, without understanding why and how it works. I can't help to shake the thought that we will see heaps of mediocre code cranked out that just about works, even though the problem maybe required a tenth of that to solve it.

Re: A coder considers the waning days of the craft

#485
post #263

I've noticed that over the years, coding has gotten easier. I've been able to express myself better with each year. I see AI as another higher-level way of expressing myself. Also, people don't write as much new code as they think they do. Still, I have many many... MANY projects that I have on my wishlist. Maybe AI will help, but my list won't go away.

I've noticed that over the years, coding has gotten easier.

That is the history of programming in a nutshell. I know I've told this story before, but my wife's uncle talks about the revolution he felt when FORTRAN IV was released. Finally you didn't have to know anything about computers or programming to write programs. You just told the computer what you want it to do and it sorted out all the hard stuff for you. Engineers like him could now just use computers on their own and didn't have to wait for the programmers to have time. He spent the rest of his career writing code without ever learning any of the 'craft' the veteran programmers at the time would have considered necessary.

Re: A coder considers the waning days of the craft

#486
post #463

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

I'm probably bad at writing prompts, but in my limited experience, I spend more time reviewing and correcting the generated code than it would have taken to write it myself. And that is just for simple tasks. I can't imagine thinking a llm could generate millions of lines of bug free code.

Personally, for me this flow works fine AI does the first version -> I heavily edit it & debug & write tests for it -> code does what I want -> I tell AI to refactor this -> tests pass and the ticket is done.

Re: A coder considers the waning days of the craft

#487

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

> It answers questions confidently but with subtle inaccuracies.

This is a valid challenge we are facing as well. However, remember that ChatGPT which many coders use, is likely training on interactions so you have some human reinforcement learning correcting its errors in real-time.

Re: A coder considers the waning days of the craft

#488

Do people seriously consider this the waning days of the craft? I don’t understand that. My view is that I am about to enter the quantum productivity period of coding. I am incredibly excited about AI assistance on my coding tasks, because it improves not only what I’m writing, but also helps me to learn as I go. I have never had a better time writing software than I have in the last year. I’ve been writing software…

I think what will happen is what happened to hardware. You used to come across people who could do things like solder components to a board and hook up various ICs as well. Heck, I did this at uni.

Now that layer of craftsmen is gone. You are either an uber-expert in how computer hardware works, or you just buy the hardware and treat it like a kind of magic.

Traditionally to become an expert you went through an intermediate stage, showed your interest, and got trained. Your hobby could turn into a profession, since the step up was gentle and interested people were needed to do some of the supporting work.

Nowadays if you're going to work in a chip fab, it's not because you were a soldering iron kid. You go through a quite long academic process that doesn't really recruit from the ranks of the hobbyists, though of course you'd expect there to be some natural interest. But the jump isn't gentle, you end up learning some pretty advanced things that make soldering seem like the stone age.

Software has this layer of craftsmen still, though it is rapidly dying, and not just from LLMs. There's a enormous number of people who can more or less hook up a website and make it work, without knowing everything about how exactly it does that. There's also plenty of excel people and python scripting people who use code in their day-to-day, but can't tell you advanced concepts. There's a lot of modern services that make this sort of thing easier (wordpress etc), and this level of skill is very learnable by people in the developing world. It's not like you can't become a real expert in those parts of the world, but economically it makes sense that there are a lot of intermediately skilled there.

What will happen with GPT and the like is the experts will not need as much help from the juniors. If you're a software architect in charge of a whole system, you won't just sketch out the skeleton of the system and then farm out the little tasks to junior coders. Maybe there are pieces that you'd farm out, but you will definitely save on the number of juniors because an LLM will give you what you need.

The result being that we'll get fewer people trained to the highest levels, but those who are will be much more productive. So those guys will indeed be entering the quantum age, but it strands a lot of people.

Re: A coder considers the waning days of the craft

#489

Do people seriously consider this the waning days of the craft? I don’t understand that. My view is that I am about to enter the quantum productivity period of coding. I am incredibly excited about AI assistance on my coding tasks, because it improves not only what I’m writing, but also helps me to learn as I go. I have never had a better time writing software than I have in the last year. I’ve been writing software…

Waning days of the career associated with it. Similar to how old craftsmanship has been superseded by machine/industrialisation.

The need for mastery of the craft will be lesser, and so mastery of it will wane, as people depend on AI instead. Then if you ever need someone to do it without AI, you might no longer be able to find anyone with the craftsmanship know-how to do it.

Also, it'll probably bring down how good coding is as a career. As productivity enters this quantum state, you'll need less engineers, and they'll need less qualifications, which might translate to less jobs, worse pay, and an increased expectation in productivity that makes you dependent on AI, as you can no longer meet these expectations and compete with others if you also dont fully leverage AI.

Re: A coder considers the waning days of the craft

#490
post #444

Earlier quoted context omitted.

I was on a team developing a critical public safety system on a tight deadline a few years ago, and i had to translate some wireframes for the admin back-end into CSS. I did a passable job but it wasn’t a perfect match. I was asked to redo it by the team-lead. It had zero business value, but such was the state of our team…being pixel perfect was a source of pride. It was one of the incidents that made me to stop fron…

[flagged]

Calling bullshit is maybe too harsh. There may be requirements matching the available training data and the right mood the LLM has been tuned for where it delivers acceptable, then considered flawless, results (extreme example: just try "create me hello world in language x" will mostly deliver flawless)... and by that amateurs (not judging, just mean less exposed to variety of problems and challenges) may end up with the feeling that LLMs could do it all.

But yes, any "serious" programmer working on harder problems can quickly derail an LLM and prove otherwise, with dozens of his simple problems each day (tried it *). It doesn't even need that, one can e.g. prove ChatGPT (also 4) quickly wrong and going in wrong circles on C++ language questions :D, though C++ is still hard, one can also do the same with questions on the not-ultracommon Python libs. It confidently outputs bullshit quick.

(*): Still can be helpful for templating, ideas, or getting into the direction or alternatives, no doubts on that!

Post reply on HN