Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A coder considers the waning days of the craft

newyorker.com

Post reply on HN
Print view View original

701–710 of 1001 posts

Re: A coder considers the waning days of the craft

#701

Post by ern »

danielbln wrote: post #615
debok wrote: post #541

Earlier quoted context omitted.

I have the same experience with the test I give my back-end devs. ChatGPT can't even begin to decode an encoded string if you don't tell it which encoding was used. ChatGPT is great at some well defined, already solved problems. But once you get to the messy real world, the wheels come off.

Base64: https://chat.openai.com/share/3793da6d-4634-4e13-9cc4-ce9983... ROT13: https://chat.openai.com/share/ae7c311d-ab23-4425-bdfa-c2314e... HEX: https://chat.openai.com/share/4b0740b7-53c0-4776-bb00-ab65b4... What kind of encoded string did you use?

It’s pretty impressive that it was able to actually decode those strings. In March, I used GPT 3.5 to write code for validating a type of string which used a checksum algorithm.

It did the task well, and even wrote tests, but it failed when generating test case values. I wonder if it would perform better if I did it today.

Re: A coder considers the waning days of the craft

#702

Post by archiepeach »

miiiiiike wrote: post #438
Banditoz wrote: post #415

Earlier quoted context omitted.

What sort of questions do you ask out of curiosity?

I don’t want scrapeable answers out there, I want to see ChatGPT cross this little Rubicon on its own. Vaguely: Questions that most people think they know the correct answers to but, in my experience, don’t.

This does sound like a test that is almost "set up to fail" for an LLM. If the answer is something that most people think they know, but actually don't then it won't pass in an LLM which is essentially a distillation of the common view.

Re: A coder considers the waning days of the craft

#703

Post by nkozyra »

jorams wrote: post #613
lucideer wrote: post #469

Earlier quoted context omitted.

> I recently asked ChatGPT to produce similar CSS and it did so flawlessly. I use ChatGPT every day for many tasks in my work and find it very helpful, but I simply do not believe this. > The article points this out: middling generalists can now compete with specialists. I'd say it might allow novices to compete with middling generalists, but even that is a stretch. On the contrary, ChatGPT is actually best suited to…

I agree with this. There are cases where it produces good results, but there are also cases where it produces bs, and it's not always obvious. I find it to work fine for cases where I know what I want but could use a starting point, but it often invents or misunderstands all kinds of things. The most frustrating situations are those where it invents a function that would miraculously do what's necessary, I tell it th…

> There are cases where it produces good results, but there are also cases where it produces bs, and it's not always obvious.

Pessimistically, this is the medium term role I see for a lot of devs. Less actual development, more assembly of pieces and being good enough at cleaning up generated code.

If an LLM can get you even 25% there most of the time, that's a massive disruption of this industry.

Re: A coder considers the waning days of the craft

#704

Post by elif »

YeGoblynQueenne wrote: post #672
ern wrote: post #444

Earlier quoted context omitted.

I was on a team developing a critical public safety system on a tight deadline a few years ago, and i had to translate some wireframes for the admin back-end into CSS. I did a passable job but it wasn’t a perfect match. I was asked to redo it by the team-lead. It had zero business value, but such was the state of our team…being pixel perfect was a source of pride. It was one of the incidents that made me to stop fron…

>> The article points this out: middling generalists can now compete with specialists. They can't, and aren't even trying to. It's OpenAI that's competing with the specialists. If the specialists go out of business, the middling generalists obviously aren't going to survive either so in the long term it is not in the interest of the "middling generalists" to use ChatGPT for code generation. What is in their interest…

It REALLY depends on the task. For instance, if you provide GPT with a schema, it can produce a complex and efficient SQL query in I would also argue that not only are the models improving, we have less than a year practically interfacing with LLM's. OUR ability to communicate with them is in infancy, and a generation that is raised speaking with them will be more fluent and able to navigate some of the clear pitfalls better than we can.

Re: A coder considers the waning days of the craft

#705

Post by archiepeach »

gitgud wrote: post #624
fauigerzigerk wrote: post #477

Earlier quoted context omitted.

I agree it's amazing. But your comment doesn't touch on the key economic question that will decide for how many people it will be this amazing new dev experience. If AI makes developers twice as productive (maybe a few years down the road with GPT-6), will this additional supply of developer capacity get absorbed by existing and new demand? Or will there be half as many developers? Or will the same number of develope…

> “ If AI makes developers twice as productive (maybe a few years down the road with GPT-6), will this additional supply of developer capacity get absorbed by existing and new demand? Or will there be half as many developers? Or will the same number of developers get paid far less than today? ” Something to remember is that every new innovation in software development only raises the expectations of the people paying…

I think this hits the nail on the head. Obviously a lot of the participants in this discussion are programmers, so there is going to be a fair amount of bias where people feel like their self-worth is being attacked/devalued. That being said, from a company perspective, this should much more unlock "moving faster" than "let's rest on our laurels". Any company that has a leading position in a particular industry is currently at greater risk of upstarts achieving their feature set in a reduced amount of time. The incentive for all companies will be to find programmers who are skilled in directing and debugging AIs.

I am currently building an iOS app using GPT-4 (I don't know Swift), and am developing an awareness of what it can/can't do, and surprised that I'm moving at the speed I did when creating React Native apps. In a possibly more competitive future market for developers, it does work in one's favour if some developers resist the efficiency improvements of AI.

Re: A coder considers the waning days of the craft

#706

Post by ern »

gumballindie wrote: post #610
ern wrote: post #444

Earlier quoted context omitted.

I was on a team developing a critical public safety system on a tight deadline a few years ago, and i had to translate some wireframes for the admin back-end into CSS. I did a passable job but it wasn’t a perfect match. I was asked to redo it by the team-lead. It had zero business value, but such was the state of our team…being pixel perfect was a source of pride. It was one of the incidents that made me to stop fron…

But if you are a middling programmer when it comes to CSS how do you know the output was “flawless” and close to the quality that css “masters” produce?

It looked correct visually and it matched the techniques in the actual CSS that the team lead and I produced when we paired to get my layout to the standard he expected.

Re: A coder considers the waning days of the craft

#707

Post by amenghra »

cultureswitch wrote: post #662
amenghra wrote: post #532

Earlier quoted context omitted.

There's a Swiss town which had autonomous shuttles running for 5 years (2015-2021) [1]. There's at least two companies (Waymo and Cruise) running autonomous taxi services in US cities that you can ride today. There have been lots of incorrect promises in the world of self-driving trucks/cars/buses but companies have gotten there (under specific constraints) and will generalize over time. [1] https://www.saam.swiss/pr…

It should be noted that the Waymo and Cruise experiments in their cities are laughably unprepared for actual chaotic traffic, often fail in completely unpredictable ways and are universally hated by locals. Autonomous buses and trams are much more successful because the problem is much easier too.

Agree. We could have all had some fine autonomous trams/subways/trains which run 24/7 at short intervals instead of spending money on self-driving cars and car infrastructure in general.

Re: A coder considers the waning days of the craft

#709

Post by varelse »

Art9681 wrote: post #688
j45 wrote: post #602

Earlier quoted context omitted.

Sometimes it requires expert guidance to get something meaningful out.

This is the correct answer. I have 23 years of experience in datacenter ops and it has been a game changer for me. Just like any tool on one's arsenal, it's utility increases with practice and learning to use it correctly. ChatGPT is no different. You get out of it what you put in to it. This is the way of the world. I used to be puzzled as to why my peers are so dismissive of this tech. Same folks who would say "We…

[dead]

Re: A coder considers the waning days of the craft

#710

Post by fkyoureadthedoc »

FrustratedMonky wrote: post #687
barrkel wrote: post #632

Earlier quoted context omitted.

ChatGPT goes from zero to maybe 65th percentile? There or thereabouts. It's excellent if you know nothing. It's mediocre and super buggy if you're an expert. A big difference is that the expert asks different questions, off in the tails of the distribution, and that's where these LLMs are no good. If you want a canonical example of something, the median pattern, it's great. As the ask heads out of the input data dist…

I encountered this with particular problem in python. Seemed like GPT wanted to always answer with something that had a lot of examples on the web, even if most answers were not correct. So garbage in, garbage out problem. I'm bit worried that the LLM's will continue to degrade as the web has increasing amount of LLM generated content. Seems to already be occurring.

Why do people that hand wave away the entire concept of LLMs because of one instance of it doing one thing poorly that they could do better, and yet always seem to fail to just show us their concrete example?
Post reply on HN