Testing GPT 4's code-writing capabilities with some real world problems • Hacker News Board

Re: Testing GPT 4's code-writing capabilities with some real world problems

#101

Post by z3c0 » Fri, Mar 17, 2023, 7:13 AM UTC

dragonwriter wrote: post #40

jacobedawson wrote: post #16

Earlier quoted context omitted.

I think most people see where the puck is going, and even where it is right now is very impressive. It's not hard to see that it will be likely less than 5 years before it will be able to do what you did on day 2, and much more, at a tiny fraction of the cost, with no downtime, no attitude problems, sick days, etc. The things you mentioned (taking on board feedback, testing across devices, iterating on solutions) doe…

> I think most people see where the puck is going Overinflated hype about “where the puck is going” being wrong is…not a new phenomenon. And the non-tech media (traditional and social, and particularly a whole lot of the elite commentariat that spans both more than the “hard” news side of traditional media, though that is influenced too) perspective on this is driven quite disproportionately by the marketing message…

Soon, a lot of people will realize that the puck doesn't travel in a straight line. Eventually, it will veer to the right in a logarithmic fashion. When it does is up for debate, but - in my experience - it always happens very short of where the hype claims it was heading.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#102

Post by anonzzzies » Fri, Mar 17, 2023, 7:14 AM UTC

pewpew2020 wrote: post #47

nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

I feel it's the average programmer who is enamoured with GPT. The programmers working on hard stuff know that AI writing programs is still far away.

It’s also programmers who work with average and bad programmers and finally have a way to not waste time with them anymore. I work on the hard stuff, gpt generate the rest (I don’t have to waste time and energy to try to pull dead horses) and my colleagues go do something else. It’s a reality today and will improve further. Sure it cannot do what I do, but that might improve; gpt4 is noticeably better than 3.5 at slightly harder things, especially if you work properly iterative with it and having it fix it’s own errors. But no, it cannot do hard things it never saw; however it can give me all the boilerplate around the hard things so I only have to do the hard things. My days are simply 3-5 in 1 now, as I also don’t have the overhead of explaining jira tasks to others who are never going to really grasp what they are doing anyway.

So I am not average and I am enamoured with gpt, simply because it presents high value to me now, more than some actual real humans. For me that’s enough revolutionary.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#103

Post by red75prime » Fri, Mar 17, 2023, 7:15 AM UTC

z3c0 wrote: post #81

red75prime wrote: post #42

Earlier quoted context omitted.

> ways (people on the Internet think) It reminds me of arguments that it's not the computer that plays chess, but its programmers. You can describe a GPT's response as a statistical average of responses on the internet (for quite a contrived definition of average), but at some point it will be easier to describe it as analyzing a snippet and forming an opinion (based on what people on the Internet think). Are we past…

> quite a contrived definition of average Sure, maybe an argmax mixed with an RNG isn't really an average, but some would say it's quite a contrived definition of forming an opinion.

argmax is doing all the heavy lifting there. Consider argmax is_proof_of_P_eq_NP(string)

You can describe solution of any well defined problem with argmax, it doesn't make it trivial.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#104

Post by pixl97 » Fri, Mar 17, 2023, 7:15 AM UTC

jamiek88 wrote: post #96

hnfong wrote: post #88

Earlier quoted context omitted.

And the next iteration of ChatGPT will likely generate a better contract because your interaction with it will be used as training data

Huh. There’s something scary about that in my lizard brain. Not sure what.

Its like feeding a baby polar bear. When it grows up it is going to eat you.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#105

Post by lxe » Fri, Mar 17, 2023, 7:15 AM UTC

It is far from being able to solve difficult or even annoyingly complicated problems that programmers solve on a regular basis just by a one-shot prompt.

Ask it to parse a PDF document and separate it into paragraphs, for example. The first solution isn't gonna work well, and by the time you get to solving yet another quirk while it apologizes to you making a mistake, it will lose context.

Best way to use this tool is to ask it short and precise questions that deal with a small piece of code.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#106

Post by theshrike79 » Fri, Mar 17, 2023, 7:16 AM UTC

nitwit005 wrote: post #58

maxdoop wrote: post #50

Earlier quoted context omitted.

I don’t think you’ve tried GPT-4. If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life). You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or C…

If all you're doing is very simple crud apps and transforming API responses, your coding is already replaceable with "no code" tools.

I'm still waiting for the no-code tools that can replace me, the API Glue Coder.

They've been promising those since the early 2000's, but I'm still employed.

Some stuff has been automated by Zapier, IFTTT and the like, but most of the systems I'm integrating with don't have public APIs.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#107

Post by aix1 » Fri, Mar 17, 2023, 7:16 AM UTC

derefr wrote: post #10

nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

Have you seen this video? https://www.youtube.com/watch?v=9QZlQMpNk-M I think the author is onto something – while AI might not be able to program per se, it can certainly be handed a code snippet and then use its huge corpus of Internet Learning™ to tell you things about it, code that looks like it, and ways (people on the Internet think) it might be solved better. In that sense, it isn't replacing the programmer; i…

I really enjoyed the video, thanks for sharing.

I think the author is operating in what I consider to be the sweet spot of current LLMs - where I can ask a question I don't know the answer to but can reliably spot bullshit (either because I know enough or through other means). I think there's a lot of value to be had when those conditions are met, and not just for coding.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#108

Post by noitpmeder » Fri, Mar 17, 2023, 7:16 AM UTC

Jensson wrote: post #97

noitpmeder wrote: post #87

Earlier quoted context omitted.

I think all it shows is that GP has never worked on complicated problems that require anything more than gluing libraries together.

If you do nontrivial coding then GPT wont help you, so that isn't relevant for the discussion.

I disagree. GP, by their statement and appeal "I've worked on adtech, crypto, fintech, gamedev, startup founder, ...", implies that he has worked on complicated software issues, and that he thinks GPT is a promising replacement for developers working on those problems.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#109

Post by maxdoop » Fri, Mar 17, 2023, 7:17 AM UTC

It seems so many of you guys are extremely lucky to be working on novel problems require elegant new solutions each day. That must be the case, otherwise I don’t understand these comments shrugging off GPT-n capabilities around coding.

“Psh, it’s just doing stuff it saw from its training data. It’s not thinking. It can’t make anything new.”

In my 11 years as a professional software engineer (that is, being paid by companies to write software), I don’t think I’ve once had come up with a truly original solution to any problem.

It’s CRUD; or it’s an API mapping some input data to a desired output; or it’s configuring some infra and then integrating different systems. It’s debugging given some exception message within a given context; or it’s taking some flow diagram and converting it to working code.

These are all things I do most days (and get paid quite well to do it).

And GPT-4 is able to do that all quite well. Even likely the flow diagrams, given it’s multi-modal abilities (sure, the image analysis might be subpar right now but what about in a few years?)

I’m not acutely worried by any means, as much of the output from the current LLMs is dependent on the quality of prompts you give it. And my prompts really only work well because I have deeper knowledge of what I need, what language to use, and how to describe my problem.

But good god the scoffing (maybe it’s hopium?) is getting ridiculous.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#110

Post by anonzzzies » Fri, Mar 17, 2023, 7:18 AM UTC

theK wrote: post #73

maxdoop wrote: post #50

Earlier quoted context omitted.

I don’t think you’ve tried GPT-4. If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life). You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or C…

On the flip side, I am having a great time charging enterprises exorbitant hourly rates to come in and help them fix their crappy crud programms. But I still don’t know if I am thrilled by the prospect of being called in to make their monkeypatched, GPT generated project work.

> their monkeypatched

You mean, like almost every outsourcing company pops over? So the type of code that infests companies who hired some sweatshop to do ‘some simple crud’? What’s the difference? Can you see the difference? Besides the gpt code will be far better commented as comments come for almost free with gpt while humans hate writing them.