Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 is phenomenal at Code

github.com

Post reply on HN
Print view View original

81–90 of 221 posts

Re: GPT-4 is phenomenal at Code

#81

Post by bob1029 »

My current take is that it's actually kind of bad at code in a lot of very important ways, but it is so good at getting to ~95% that you don't even feel the missing pieces once you are in flow. Having skills in the target area are important, but some quick runtime testing usually closes the gap.

The added confidence from being able to scream a horrible demand into the void and get back something nearly complete is beyond unbelievable to me. I'm making attempts at projects that I would never have considered otherwise, and I am becoming increasingly emboldened by the capability - "Without using 3rd party dependencies" is probably my #1 prompt nudge right now.

Re: GPT-4 is phenomenal at Code

#82

Post by yieldcrv »

zamnos wrote: post #61

Is it? According to OpenAI's paper GPT-4 GPT-4 (no vision) GPT-3.5 Leetcode (easy) 31 / 41 31 / 41 12 / 41 Leetcode (medium) 21 / 80 21 / 80 8 / 80 Leetcode (hard) 3 / 45 3 / 45 0 / 45 https://cdn.openai.com/papers/gpt-4.pdf Table 1; page 5. So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the…

people are impressed because you never need to do leetcode hard problems in any business anywhere in your career to make a few billion dollars. I would fire anybody using company time trying to reinvent some academic search thing instead of just using the hashmap.

Re: GPT-4 is phenomenal at Code

#83

Post by cypress66 »

wil421 wrote: post #65
ChuckNorris89 wrote: post #25

Earlier quoted context omitted.

I think my barber and my plumber have safer jobs from automation than average coders.

Let’s play this out. A company fires all the SE’s and GPT-4 takes over their place. Who will be there to ask the right questions? Middle management? Who can validate it spit out what you actually want? Project managers? Who will determine what questions to ask GPT-4? Product owners? Who will decide what type of tech (language etc…) to ask GPT-4 to write? College grads turned into corporate GPT-4 pushers? Who will be…

You don't need to fire all software engineers. You could let's say replace 75% of them with GPT.

Re: GPT-4 is phenomenal at Code

#84

Post by amelius »

LightMachine wrote: post #9

It managed to write, in one shot, a working λ-calculus parser, using a very specific programming style I asked it in JavaScript, and then translated it all to Python, including sarcastic, rhyming GLaDOS comments. https://twitter.com/VictorTaelin/status/1635726202231988225 It also seems to be extremely competent at writing Agda types and proofs. We need some tool that highly integrates it with entire codebases, allowi…

I don't think this is a good test. Every CS graduate student has to write a lambda-calculus parser. There must be thousands of implementations on the web. It really is not strange that GPT-4 can reproduce this.

Re: GPT-4 is phenomenal at Code

#85

Post by tracerbulletx »

beepbooptheory wrote: post #59

Honestly at this point I am just looking forward to us taking this stuff for granted. Its all very nice and useful, but I think my threshold for guffawing at it is smaller than most. I know we still got some time left in the cycle, but I just want people to calm down and just start using it. Yes, cool, you did xyz all at once, but what did you make with it? It's kinda akin to the whole "using rust" phenomenon. I only…

The model IS the thing that has been made, it is the impressive accomplishment. People are awed by that. The examples are just people testing the limits of its capabilities. It's like being present at the splitting of the atom and asking ya but what have you actually powered with it.

Re: GPT-4 is phenomenal at Code

#86

Post by DrBenCarson »

mjr00 wrote: post #66
recuter wrote: post #14

Remember the comments about artists and creatives when DALL-E came out? I remember.

I remember it being vastly overhyped about how artists were now obsolete, but it really hasn't happened. People have quickly became very attuned to the "AI art style"--humans are excellent at pattern recognition, after all--and if anything, it's made people appreciate how important an artists' individual style and attention to detail is.

People don’t make art for money (mostly)

People do write code for money (mostly)

See the difference? How would you feel if developers earned as much as artists do?

Re: GPT-4 is phenomenal at Code

#87

Post by riceart »

ChuckNorris89 wrote: post #51
swader999 wrote: post #39

Earlier quoted context omitted.

I think we'll all get more productive is all. There's a lot of projects that just weren't economical but now may become so. The bigger disruption will occur when Google, Facebook, Twitter, Reddit etc become useless from all the AI spam that's sure to overwhelm it all.

But here's the question. If these tools make existing devs 10x more productive, why bother hiring new people in the field?

The tools of today, even fucking Eclipse and gah Xcode from the early 2000s make programmers far far more than 10x more productive than machine code on punch cards. By your implied logic there should be a tiny fraction of the programmers there were in the 60s.

Anyway I think this 10x more productive claim is still a bunch of BS with the current crop of AI demos.

Re: GPT-4 is phenomenal at Code

#88

Post by sualehasif »

hot_gril wrote: post #18

I tried having it write some HTML and JS code for a chat app UI. It works amazingly well. I'm legitimately going to have ChatGPT help me write this side project website I've been working on.

We are trying to build this right in to the editor. The UI is in the tweet thread. Feel free to sign up at control.dev

Re: GPT-4 is phenomenal at Code

#89

Post by stuff20230314 »

yieldcrv wrote: post #22
stuff20230314 wrote: post #12

That's cool and all, but can it do "in this existing code, please amend this feature"? Also useful, "please cover this code with meaningful unit tests"?

> Also useful, "please cover this code with meaningful unit tests"? GPT-3.5 is already really good at that. If you aren't using ChatGPT for your unit tests I'd suspect you're on a Performance Improvement Plan by now

Yeah, I am sure the FAANG devs are all copy-pasting things into ChatGPT or requesting Copilot. Sure.

Re: GPT-4 is phenomenal at Code

#90

Post by dougmwne »

zamnos wrote: post #61

Is it? According to OpenAI's paper GPT-4 GPT-4 (no vision) GPT-3.5 Leetcode (easy) 31 / 41 31 / 41 12 / 41 Leetcode (medium) 21 / 80 21 / 80 8 / 80 Leetcode (hard) 3 / 45 3 / 45 0 / 45 https://cdn.openai.com/papers/gpt-4.pdf Table 1; page 5. So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the…

That’s better than a lot of professional engineers can do, but I agree that it is unlikely to get hired in a senior FAANG role.
Post reply on HN