Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

331–340 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#331

Post by diab0lic »

korijn wrote: post #314
IKLOL wrote: post #229

Earlier quoted context omitted.

I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.

If as GP says AI could automate 1% of a single programmer's job (the boring part where you write code), then how on earth can you derive that a single programmer could do the job of 5 with AI? It's completely illogical.

I suspect the post you’re replying to disagrees with the 1% figure.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#332

Post by coldtea »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

>Basically, 99% of the effort is NOT WRITING CODE. It's all about communication with people, and problem solving. If we use GPT-X in our company, it will help us with 1% of our workload

First, if you did have GPT-X (say GPT-10) in your company, there wouldn't be much back-and-forth communication either. Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements might be given by a GPT-X.

Second, even if that's not the case, the part of doing the communication can be handled by non-programmers. Then they can feed the result of the communication to GPT-X and had it churn out some program. Perhaps would keep a couple of developers to verify the programs (sort of like GPT-X operators and QA testers) and get rid of the rest.

As for the rest of the current team of developers? GPT-X and the people running the company could not care less about them!

Re: Testing GPT 4's code-writing capabilities with some real world problems

#333

Post by dmix »

From my own sampling of going through about 10 times I used GPT for real world Typescript code, some used in production, I can confirm that GPT-4 does a noticeably better job and produces code I actually want to use way more often.

GPT-3.5 always produced very verbose types and over engineered code. The GPT-4 outputs were consistently shorter and more focused. Kind of like how a junior dev has to think through all the smaller steps and makes functions for each, as he incrementally solves the problem slower and less intuitively, almost over explaining the basics, while a senior dev merges the simpler stuff into small concise functions. You can see it with the var names and type choices GPT-4 focused much more on what the code is trying to accomplish rather than what the code itself is doing. And these are all with the same prompts.

There’s still things like unused vars being included occasionally and some annoying syntax choices, if I could append prettier/eslint rules automatically to GPT output it’d be gold (I haven’t tried to do this myself).

But still very encouraging.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#334

Post by bumby »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

It's really about a misunderstanding on the value-stream mapping from concept to production. The claims that GPT-X will write code and thus cover the whole value-stream is conflating the very last steps with the whole process.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#335

Post by Jeff_Brown »

staminade wrote: post #211
nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

While I think there's truth to what you say, I'd also point our that workers in many pre-automated industries with an "artisan" approach also considered themselves irreplaceable because they figured, correctly, that nobody could build a machine with the capability of reproducing their workflow, with all its inherent uncertainty, flexibility and diverse physical and mental skills. What they failed to predict was that…

A deep insight. Thanks.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#336

Post by swexbe »

pixl97 wrote: post #114
photonbeam wrote: post #64

Earlier quoted context omitted.

They said the same thing about transistor scaling

Being that you and I are talking to each other by forming thoughts in meat that us causing other meat to move around tells me that digital thought still has a lot of scaling room ahead of it. Maybe superintelligence isn't possible for some reason, but the fact we exist should tell you that general intelligence is not magic.

Your brain is analog. Trying to emulate this with a digital computer is bound to come with some computational cost.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#337

Post by dukeofdoom »

A solo developer can now afford an assistant. It's liberating, since it makes it easier to get some things done. So you can do more, or have more free time. You can get by using Midjourney for art, and GPT-4 to answer questions and occasionally help to write code.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#338

Post by marcyb5st »

maxdoop wrote: post #109

It seems so many of you guys are extremely lucky to be working on novel problems require elegant new solutions each day. That must be the case, otherwise I don’t understand these comments shrugging off GPT-n capabilities around coding. “Psh, it’s just doing stuff it saw from its training data. It’s not thinking. It can’t make anything new.” In my 11 years as a professional software engineer (that is, being paid by co…

To be honest, I am using ChatGPT and now GPT4 as tools to speed up my workflow. Writing tests, writing boilerplate, parsing logic for deeply nested and messy JSON.

Most of the times it gets things quite well, and if you provide context in the form of other source code, it's really good, even at using classes or functions that you provide and hence are novel to it.

The hard logic bits imho (something elegant, maintainable, ...) Are still up to you.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#339

Post by zeroonetwothree »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

But once XXQ has their own ChatGPT they can just communicate with each other and hash this out 1000x faster.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#340

Post by dmix »

dmix wrote: post #333

From my own sampling of going through about 10 times I used GPT for real world Typescript code, some used in production, I can confirm that GPT-4 does a noticeably better job and produces code I actually want to use way more often. GPT-3.5 always produced very verbose types and over engineered code. The GPT-4 outputs were consistently shorter and more focused. Kind of like how a junior dev has to think through all th…

If someone can make an ESLint/prettier rules to GPT prompt script I’d love you. Or even the airbnb-base baseline one.
Post reply on HN