Live data from Hacker News

A coder considers the waning days of the craft

newyorker.com

461–470 of 1001 posts

Re: A coder considers the waning days of the craft

#461

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

Try breaking down the problem. You don't have to do it yourself, you can tell ChatGPT to break down the problem for you then try to implement individual parts.

When you have something that kind of works, tell ChatGPT what the problems are and ask for refinement.

IMHO currently the weak point of LLMs is that they can't really tell what's adequate for human consumption. You have to act as a guide who knows what's good and what can be improved and how can be improved. ChatGPT will be able to handle the implementation.

In programming you don't have to worry too much about hallucinations because it won't work at all if it hallucinates.

Re: A coder considers the waning days of the craft

#462
post #444

Earlier quoted context omitted.

I was on a team developing a critical public safety system on a tight deadline a few years ago, and i had to translate some wireframes for the admin back-end into CSS. I did a passable job but it wasn’t a perfect match. I was asked to redo it by the team-lead. It had zero business value, but such was the state of our team…being pixel perfect was a source of pride. It was one of the incidents that made me to stop fron…

I’d need to see it. I can’t get ChatGPT to outperform a novice. And now I’m having candidates argue that they don’t need to learn the fundamentals because LLMs can do it for them.. Good luck HTML/CSS expert who couldn’t produce a valid HTML5 skeleton. Reminds me of the pre-LLM guy who said he was having trouble because usually uses React.. So I told him he could use React. I don’t mean to rag on novices but these guy…

Just out of curiosity: is the code generated by ChatGPT not what you expected or is it failing to produce the result that you wanted.

I suspect you mean the latter, but just wanted to confirm.

Re: A coder considers the waning days of the craft

#463

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

I'm probably bad at writing prompts, but in my limited experience, I spend more time reviewing and correcting the generated code than it would have taken to write it myself. And that is just for simple tasks. I can't imagine thinking a llm could generate millions of lines of bug free code.

Re: A coder considers the waning days of the craft

#465

Earlier quoted context omitted.

We’ll see - but given the gap between chatgpt 3 and 4, I think AIs will be competitive with mid level programmers by the end of the decade. I’d be surprised if they aren’t. The training systems we use for LLMs are still so crude. ChatGPT has never interacted with a compiler. Imagine learning to write code by only reading (quite small!) snippets on GitHub. That’s the state llms are in now. It’s only a matter of time b…

Why do you say the snippets are small? They don’t get trained on the full source files?

Nope. LLMs have a limited context window partly because that's the chunk size with which they're presented with data to learn during training (and partly for computational complexity reasons).

One of the reasons I'm feeling very bullish on LLMs is because if you look at the exact training process being used it's full of what feels like very obvious low hanging fruit. I suspect a part of the reason that training them is so expensive is that we do it in really dumb ways that would sound like a dystopian hell if you described it to any actual teacher. The fact that we can get such good results from such a terrible training procedure by just blasting through it with computational brute force, strongly suggests that much better results should be possible once some of that low hanging fruit starts being harvested.

Re: A coder considers the waning days of the craft

#466

Earlier quoted context omitted.

4th option. They're trying to enter the market and get a good job, but some companies consider them less experienced but more expensive than Copilot.

Fewer but more senior resources seems possible. I have already seen the impact of AI systems, reducing the number of junior coding positions and outsourcing. On the other hand, anyone who actually takes the time to get good with AI coding tools can be performing at senior levels, much more quickly, so there’s that too. I’m not convinced that it’s some kind of apocalypse as much as it is a massive shift that is going…

Agreed. The only thing to come true in the last decade about the death of software engineering from no/low-code systems is that there were more software jobs created to build no/low-code systems.

Also see "death of the sys admin by way of AWS and the like"

Re: A coder considers the waning days of the craft

#467

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

Haha, does it involve timers?

Re: A coder considers the waning days of the craft

#468
I wonder if there are parallels to be drawn here against the industrialization of other, physical, goods, and how their production evolved over time. If you look at something like, say, printing (I'll use the example of woodblock printing here, since I know a decent amount about it), it originated very much as a craft and discipline, as a means to fulfill a particular function. As printing technology improved and became better industrialized, the craft of printing was gradually replaced with other means that fulfilled the same function, but possessed a different form. i.e., lacking those characteristics of products of craft that we find desirable; artless.

We've been able to fulfill the function of printing very cheaply for what seems like ages now, and we've reached a point where some niche and particularly attuned segment of the population wants a bit more out of the actual form of the printed product, the depth of form that was once common. There is a growing community of people that deeply care about woodblock prints now, favoring their physical characteristics, despite such prints falling out of fashion for a period of time during the heights of industrialization. This group of people understands the value of such craft, and is willing to spend more for it, since the difference in the form of the end product from mass-produced stuff is so stark.

The key thing here, and with other categories of physical goods (e.g. pottery, glassware, furniture, etc.), is that there's an obvious and tangible difference in the form of products produced via traditional means, and those mass manufactured, despite them serving fundamentally the same function.

With software however, I worry that this isn't the case, and the sort of resurgence of interest we see now in products produced by traditional means won't ever translate, assuming that we do move in the direction of more and more software engineering being "automated" by AI assistance. To an end-user of a piece of software, I imagine that there will be very little visible difference in the observable characteristics between fully hand-written and AI-produced software. Indeed, given the same requirements, there ought not be a difference between these two things. It's exactly this delta, however, which drives the passionate and less cost-sensitive enthusiasts to prefer handmade physical goods over manufactured ones. If both the form and the function of AI-produced software is identical to those of traditional software, but the AI-produced software is cheaper, why would anyone go with the traditional stuff? I understand that there are other factors at play here as well (e.g., particularly principled consumers etc.) but really, some combination of form, function, and cost seem like the biggest levers to me, and they seem on the face of it to be pulled toward the direction of AI, for better or worse.

Re: A coder considers the waning days of the craft

#469
post #444

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

I was on a team developing a critical public safety system on a tight deadline a few years ago, and i had to translate some wireframes for the admin back-end into CSS. I did a passable job but it wasn’t a perfect match. I was asked to redo it by the team-lead. It had zero business value, but such was the state of our team…being pixel perfect was a source of pride. It was one of the incidents that made me to stop fron…

> I recently asked ChatGPT to produce similar CSS and it did so flawlessly.

I use ChatGPT every day for many tasks in my work and find it very helpful, but I simply do not believe this.

> The article points this out: middling generalists can now compete with specialists.

I'd say it might allow novices to compete with middling generalists, but even that is a stretch. On the contrary, ChatGPT is actually best suited to use by a specialist who has enough contextual knowledge to construct targeted prompts & can then verify & edit the responses into something optimal.

Re: A coder considers the waning days of the craft

#470
post #450

Earlier quoted context omitted.

Not gonna happen. I don’t want scrapeable answers out there, I want to see ChatGPT cross this little Rubicon on its own.

It's not that I don't believe you, but without sharing the specific prompt it's hard to say if it's actually GPT4 failing, or if it's actually being poorly-prompted, or if actually the task it is being given is more complex than GPT's capabilities or you are implying. GPT4 does fail (often!) but fails less with good prompts, simple requirements, it is better at some frameworks and languages than others, and there is…

I’ve gone through every permutation that I can think of. It’s a very basic question. If it understood the CSS spec it wouldn’t be difficult to answer the questions or perform the task.

At a certain point going down the rabbit hole of proompter engineering levels feels like an apologist’s hobby. I’m rooting for the tech but there’s a lot of hyperbole out there and the emperor might be naked for a few more years.

Post reply on HN