Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A coder considers the waning days of the craft

newyorker.com

Post reply on HN
Print view View original

921–930 of 1001 posts

Re: A coder considers the waning days of the craft

#921

Post by dontupvoteme »

ryanjshaw wrote: post #918
dontupvoteme wrote: post #823

Earlier quoted context omitted.

>F# Barely exists in training data. Might as well ask it to code some microcontroller specifically assembly, watch it fail and claim victory.

> Barely exists in training data. Irrelevant - this is basic computer science. As far as I know, you can't create a bidirectional graph node structure without a mutable data structure or language magic that ultimately hides the same mutability. The fact that ChatGPT recognizes the mutability issue when I explain the bug tells you it has the knowledge, but it doesn't correctly infer the right answer and instead makes…

I would agree with you if it was a model trained to do computer science, rather than a model to basically do anything, which just happens to be able to do computer science as well.

Also code is probably one of the easiest use cases for detecting hallucinations since you can literally just see if it is valid or not the majority of the time.

It's much harder for cases where your validation involves wikipedia, or academic journals, etc.

Re: A coder considers the waning days of the craft

#922

Post by chiefalchemist »

miiiiiike wrote: post #409

I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…

I understand your concern, but isn't it apples v oranges.

Yes, ChatGPT can't pass a particular test of X to Y. But does that matter when ChatGPT is both the designer and the developer? How can it be wrong, when its answer meets the requirements of the prompt? Maybe it can't get from X to Y, but if its Z is as good as Y (to the prompter) then X to Y isn't relevant.

Sure there will be times when X to Y is required but there are plenty of other times where - for the price - ChatGPT's output of Z will be considered good enough.

"We've done the prototype (or MVP) with ChatGPT...here you finish it."

Re: A coder considers the waning days of the craft

#923

Post by tj-teej »

"At one point, we wanted a command that would print a hundred random lines from a dictionary file. I thought about the problem for a few minutes, and, when thinking failed, tried Googling. I made some false starts using what I could gather, and while I did my thing—programming—Ben told GPT-4 what he wanted and got code that ran perfectly."

Sure ChatGPT can do this, but let's not pretend this is a problem requiring much "craft".

Re: A coder considers the waning days of the craft

#924

Post by barrkel »

danielbln wrote: post #909
barrkel wrote: post #632

Earlier quoted context omitted.

ChatGPT goes from zero to maybe 65th percentile? There or thereabouts. It's excellent if you know nothing. It's mediocre and super buggy if you're an expert. A big difference is that the expert asks different questions, off in the tails of the distribution, and that's where these LLMs are no good. If you want a canonical example of something, the median pattern, it's great. As the ask heads out of the input data dist…

Why not give it a systemprompt that specifies some you're requirements: "you are a experienced senior ruby developer who writes robust maintainable code, follow these coding guidelines 《examples》"

If I thought it was worthwhile, maybe that would patch that specific hole.

The other problem I get when trying to make it write code is that it gets kinda slippery with iterated refinements. In the back and forth dialog, addressing issues 1 through n in sequence, it gets to a place where issue k I mean, I still use it to get a basic shape especially when I'm working with a command line tool I'm not an expert in, it's still useful. It's just not great code.

Re: A coder considers the waning days of the craft

#925

Post by dhkk »

Aerbil313 wrote: post #446
decasia wrote: post #52

I have been having the following debate with my friend who does AI and neural network stuff: Him: Coding will soon be obsolete, it will all be replaced by chatgpt-type code gen. Me: OK but the overwhelming majority of my job as a "senior engineer" is about communication, organizational leadership, and actually understanding all the product requirements and how they will interface with our systems. Yes, I write code,…

Wait before product-gen AI emerges. No, seriously. Do folks here not see it's possible even today with a complex system based on LLMs? It's a matter of time.

No. I think those of us that work on enterprise software within massive orgs know the level of AI needed to do any portion of our job is leaps and bounds ahead of what is currently available. I can see some distant future where maybe this is possible, but I doubt we'll be using AI based on transformers by that point...

Re: A coder considers the waning days of the craft

#926

Post by hn1986 »

cglan wrote: post #6

Maybe I’m in the minority. I’m definitely extremely impressed with GPT4, but coding to me was never really the point of software development. While GPT4 is incredible, it fails OFTEN. And it fails in ways that aren’t very clear. And it fails harder when there’s clearly not enough training resources on the subject matter. But even hypothetically if it was 20x better, wouldn’t that be a good thing? There’s so much of t…

I'll ask simple questions for SQL queries and it just hallucinates fields that don't exist in system/information_schema tables. It's mind boggling how bad it is sometimes

Re: A coder considers the waning days of the craft

#927

Post by gammalost »

cubefox wrote: post #873
skepticATX wrote: post #82

Am I the only one becoming less impressed by LLMs as time passes? I will admit, when Copilot first became a thing in 2021, I had my own “I’m about to become obsolete” moment. However, it’s become clear to me, both through my own experience and through research that has been conducted, that modern LLMs are fundamentally flawed and are not on the path to general intelligence. We are stuck with ancient (in AI terms) tec…

> We are stuck with ancient (in AI terms) technology. What are you talking about? ChatGPT came out only a year ago, GPT-4 less than a year ago. That's the opposite of ancient technology, it's extremely recent.

To be fair the theory behind it is old. Only that hardware wasn't up to the task yet. For example here's language prediction from 1991 https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog....

Re: A coder considers the waning days of the craft

#928

Post by cubefox »

gammalost wrote: post #927
cubefox wrote: post #873

Earlier quoted context omitted.

> We are stuck with ancient (in AI terms) technology. What are you talking about? ChatGPT came out only a year ago, GPT-4 less than a year ago. That's the opposite of ancient technology, it's extremely recent.

To be fair the theory behind it is old. Only that hardware wasn't up to the task yet. For example here's language prediction from 1991 https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog... .

That's not true. Modern language models use modern theory, such as the transformer architecture.

Re: A coder considers the waning days of the craft

#929

Post by tabtab »

> had no idea what a pain it is to make an iPhone app. I’d tried a few times and never got beyond something that half worked. I found Apple’s programming environment forbidding.

What's missing these days is domain-specific languages (DSL). Maybe domain-specific API's (libraries) can be made better, but I have not seen it done well yet.

Domain specific languages made it easy and DIRECT to do things you wanted done in that domain, no weird-ass reverse dependency factory visitor injection design patterns or whatnot, you only had domain-oriented commands and the basics of functions, loops, and conditionals to glue them together.

Sure, they didn't scale well to "enterprise", but if that's not what you want, so be it! Global variables didn't kill kittens at that scale like "real" software engineers warn against. I barely had to read the manual when learning VB classic, Microsoft Access (pre XML), Paradox, etc. it was so intuitive and almost everything related to the domain (desktop GUI's and small-ish CRUD).

It was almost like pseudo-code! That's the secret, the closer it is to domain-specific-pseudo-code, the easier it is to learn and the less bloated the code.

Wanna make a successful tool for non-giant orgs or groups? Focus on the domain and only the domain; don't let fads and buzzwords and mega-scaling distract you.

Re: A coder considers the waning days of the craft

#930

Post by stardom5761 »

Dave_Rosenthal wrote: post #71
alexmuro wrote: post #10

I don't feel like we are in the waning days of the craft at all. Most of the craft is creating an understanding between people and software and most human programmers are still bad at it. AI might replace some programmers but none who program as a craft.

"Chess engines might get better than some chess players, but none who play Chess as a craft." Do you think people in the 90s thought this? Probably... In the article, the author mentions that Chess centaurs (a human player consulting an engine) can still beat an engine alone. But the author is wrong. There was a brief period a while ago when that was true, but chess engines are so strong now that any human interventi…

Honest and sensible take, thank you.

Do you think that all areas of engineering will be impacted? I feel like the job of an EE or a ME is still inherently out of reach of LLMs.

Post reply on HN