Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 is phenomenal at Code

github.com

Post reply on HN
Print view View original

151–160 of 221 posts

Re: GPT-4 is phenomenal at Code

#151

Post by antibasilisk »

wil421 wrote: post #93
cypress66 wrote: post #83

Earlier quoted context omitted.

You don't need to fire all software engineers. You could let's say replace 75% of them with GPT.

Those software engineers are going to be very overloaded. Will they be troubleshooting 100% of the code with only 25% of the staff? GPT might become a tool in our arsenal but not a replacement. It needs to be able to say I don’t know before it spits out nonsense.

It takes fewer people to maintain a system than it does to build it. You do not need a team of 7 engineers to stick around to read logs and code, you need exactly one, and in all likelihood he's going to spend most of his time watching YouTube because he's so out of practice that if anything was wrong the AI would most likely find it before him, and when he does find something he's going to just ask the AI to fix it instead of exerting the mental effort to do it himself.

Re: GPT-4 is phenomenal at Code

#152

Post by LightMachine »

worrycue wrote: post #147
amelius wrote: post #84

Earlier quoted context omitted.

I don't think this is a good test. Every CS graduate student has to write a lambda-calculus parser. There must be thousands of implementations on the web. It really is not strange that GPT-4 can reproduce this.

Frankly, if ChatGPT is only good at reproducing commonly written code, I don’t think it will impact the profession much given that code reuse is already a thing and lots are distributed free on the internet.

Even if that was the case, it'd still be massively useful because it would allow code to be transposed between languages idiomatically. For example, certain things like game engines were only implemented in very messy C++ code. If GPT understands how these libraries work, could it recreate all these game engines as cleaned up, elegant Haskell code?

Re: GPT-4 is phenomenal at Code

#153

Post by gptgpp »

yieldcrv wrote: post #82
zamnos wrote: post #61

Is it? According to OpenAI's paper GPT-4 GPT-4 (no vision) GPT-3.5 Leetcode (easy) 31 / 41 31 / 41 12 / 41 Leetcode (medium) 21 / 80 21 / 80 8 / 80 Leetcode (hard) 3 / 45 3 / 45 0 / 45 https://cdn.openai.com/papers/gpt-4.pdf Table 1; page 5. So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the…

people are impressed because you never need to do leetcode hard problems in any business anywhere in your career to make a few billion dollars. I would fire anybody using company time trying to reinvent some academic search thing instead of just using the hashmap.

[dead]

Re: GPT-4 is phenomenal at Code

#154

Post by Jtsummers »

palijer wrote: post #123
capableweb wrote: post #95

Earlier quoted context omitted.

Let's invent a better test then. What could be an example task that isn't widely available on the internet already? I'm having a hard time coming up with anything that isn't reproduced in many places.

Take something from advent of code and change it up a bit seems like a good test

Give it Synacor Challenge, just the spec, and see if it can pull it off. Fewer of those solutions out there. It just went offline recently, but Aneurysm9 has preserved the problem spec, their binary, and the checksum of the codes for their binary to check against.

https://github.com/Aneurysm9/vm_challenge

The program will, likely, need to be amended to get the last code (last 2 codes? been a while) so you can see how it would handle updating for the new requirements.

Re: GPT-4 is phenomenal at Code

#155

Post by nottathrowaway3 »

dougmwne wrote: post #135
nottathrowaway3 wrote: post #120

Earlier quoted context omitted.

That would actually be an interesting product: spell-check but it edits your email or slack message to be more sycophantic. I'd use it.

I should probably start doing that. I’m a self-important jerk usually.

Me too.

(1) U.S. total wages were $9.7 trillion in 2021 [1].

(2) Most Americans are not inherent masochists and have too much self-esteem to be effective at sucking up.

(3) The average white-collar office worker would be willing to give up a portion of their salary to not have to stoop.

We have discovered a market.

[1] https://fred.stlouisfed.org/series/BA06RC1A027NBEA

Re: GPT-4 is phenomenal at Code

#156

Post by impulser_ »

sualehasif wrote: post #134
impulser_ wrote: post #114

Unless these models are updated frequently it not going to be better than a human. ChatGPT doesn't even know Go has generics yet. Now imagine is suggesting you a inferior way of doing something because it doesn't know that a bug as been fixed or a better feature/version has been implemented. It often recommends made up libraries, or libraries that don't exists anymore. It even recommended using a Go wiki page as an i…

GPT-4 certainly is aware of go generics. Do you have an example you would like to test?

It does know that Go has generics if you ask it because 1.17 had experimental generics. But it doesn't know about 1.18 which is the official release of generics.

For example it doesn't know that Go has the any type that can be used in place for interface{}.

Re: GPT-4 is phenomenal at Code

#157

Post by CapstanRoller »

worrycue wrote: post #147
amelius wrote: post #84

Earlier quoted context omitted.

I don't think this is a good test. Every CS graduate student has to write a lambda-calculus parser. There must be thousands of implementations on the web. It really is not strange that GPT-4 can reproduce this.

Frankly, if ChatGPT is only good at reproducing commonly written code, I don’t think it will impact the profession much given that code reuse is already a thing and lots are distributed free on the internet.

Most code produced is commonly-written code.

A sizable portion of software devs ultimately work on something that, at its core, follows the basic CRUD pattern. The day-to-day stuff also involves a lot of boilerplate -- "public static void main" has paid a lot of mortgages over the years.

Re: GPT-4 is phenomenal at Code

#158

Post by coenhyde »

stuff20230314 wrote: post #12

That's cool and all, but can it do "in this existing code, please amend this feature"? Also useful, "please cover this code with meaningful unit tests"?

Probably. GPT3.5 was really good at writing unit tests. I asked it to unit tests for some typescript code using jest and aws-sdk-mock. It did it as I would do. I really couldn't fault it.

Re: GPT-4 is phenomenal at Code

#159

Post by softwaredoug »

We're in a situation where junior side of the software job market has really dried up. My question is whether the junior side will ever come back? Will we ever have tech companies like we've had, with massive intern -> junior pipelines, gradually developing senior devs? Or will there by a narrower pipeline, with fewer senior devs working with AI tools?

An optimistic take is that by automating a lot of mundane work, our level of craftsmanship and creativity can increase. That we can have a much higher bar for quality and professionalism.

Unfortunately it will also mean a lot of junior folks may never get back into software jobs.

Re: GPT-4 is phenomenal at Code

#160

Post by worrycue »

DrBenCarson wrote: post #86
mjr00 wrote: post #66

Earlier quoted context omitted.

I remember it being vastly overhyped about how artists were now obsolete, but it really hasn't happened. People have quickly became very attuned to the "AI art style"--humans are excellent at pattern recognition, after all--and if anything, it's made people appreciate how important an artists' individual style and attention to detail is.

People don’t make art for money (mostly) People do write code for money (mostly) See the difference? How would you feel if developers earned as much as artists do?

If people don’t make art for money there wouldn’t be such a huge blowback from AI art.
Post reply on HN