Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

81–90 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#81

Post by z3c0 »

red75prime wrote: post #42
derefr wrote: post #10

Earlier quoted context omitted.

Have you seen this video? https://www.youtube.com/watch?v=9QZlQMpNk-M I think the author is onto something – while AI might not be able to program per se, it can certainly be handed a code snippet and then use its huge corpus of Internet Learning™ to tell you things about it, code that looks like it, and ways (people on the Internet think) it might be solved better. In that sense, it isn't replacing the programmer; i…

> ways (people on the Internet think) It reminds me of arguments that it's not the computer that plays chess, but its programmers. You can describe a GPT's response as a statistical average of responses on the internet (for quite a contrived definition of average), but at some point it will be easier to describe it as analyzing a snippet and forming an opinion (based on what people on the Internet think). Are we past…

> quite a contrived definition of average

Sure, maybe an argmax mixed with an RNG isn't really an average, but some would say it's quite a contrived definition of forming an opinion.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#82

Post by vbezhenar »

dr_dshiv wrote: post #45
refulgentis wrote: post #27

Earlier quoted context omitted.

Demonstrably untrue, it’s taken up semi-residence in everything from oped pages to boardrooms since the New Year. Welcome to the mid singularity, my friend.

You can’t say “singularity” on HN without a ton of downvotes. It’s not because people disagree (or they’d say so), but because they literally shit their pants when they think about it. Hard to blame them.

I'm saving my money, so I can live enough off savings to learn a new profession. I'm not sure if that will happen, but I consider this option as likely enough to affect my life decisions. The world is changing at alarming rate.

Though I think that it's more likely that I'll become shaman who talks to the spirits. Managers hate computers and they'd prefer to pay someone to deal with them.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#83

Post by hnfong »

theK wrote: post #73
maxdoop wrote: post #50

Earlier quoted context omitted.

I don’t think you’ve tried GPT-4. If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life). You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or C…

On the flip side, I am having a great time charging enterprises exorbitant hourly rates to come in and help them fix their crappy crud programms. But I still don’t know if I am thrilled by the prospect of being called in to make their monkeypatched, GPT generated project work.

"Hey GPT-X: Can you refactor the codebase in gpt@monkeypatched-crap.git for me? Preferably in the style of gpt@crappycrud.git that I worked on last year."

Re: Testing GPT 4's code-writing capabilities with some real world problems

#84

Post by hammyhavoc »

closetnerd wrote: post #26

What's most astounding is that it was expected to work when so much of the prompt just doesn't make sense: > Objects should only move if they will end up on an empty tile after their move > "An object is free to move onto a tile that another object moves onto if that other object is moving off of its tile"

Repeat the experiment in a better way and blog about it then rather than negging in the comments. I'm not the author, but I'll await your peer review.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#85

Post by neom »

I came up in the 90s, used a lot of dreamweaver to build sites, all my friends thought I was a wizard because I had a website and that required you to program interweb stuff. Then the net became pretty complex, I gave up around dhtml but always really appreciated and loved what folks could do with the DOM. I've been thinking a lot recently that GPT might allow me to build again, I have some of that 90s dreamweaver vibes using it.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#86

Post by whateveracct »

sabellito wrote: post #66
whateveracct wrote: post #55

Earlier quoted context omitted.

Programming takes little effort. It's at most 20% of my effort. Feels like GPT is programming with extra steps. Maybe it's for people who can never think programming is easy. Clearly there's a lot of such types. Explains a lot.

What I've seen in this thread so far is some people saying "this is really useful in these particular cases" and some others, like yourself, saying "I'm too smart for this, maybe it's useful for inexperienced/incompetent people".

I don't even think it's useful for the incompetent. I do think the incompetent think it's useful tho.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#87

Post by noitpmeder »

copperx wrote: post #71
whateveracct wrote: post #53

Earlier quoted context omitted.

the big thing to me is programming isn't hard I've worked on adtech, crypto, fintech, gamedev, startup founder, BigCo. Not once was programming something that was a time sink. Makes me feel like GPT is marketing to the incompetent or something.

> Not once was programming something that was a time sink. Can you please expound on that? Do you mean not hard as in comparison to something else?

I think all it shows is that GP has never worked on complicated problems that require anything more than gluing libraries together.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#88

Post by hnfong »

matwood wrote: post #67
JimmyRuska wrote: post #22

Earlier quoted context omitted.

This post feels like the people that go into linux forums and say linux sucks because I can't get it to do X but microsoft can, but then get 400 replies and several that show GPT has limited reasoning but given enough knowledge of the problem you can coerce it to do surprising things so long as you can relate it to something in else in the knowledge base. Given how big that knowledge base is, you can get lucky surpri…

> you can coerce it to do surprising things Yeah, I've gotten it to write some pretty decent contracts, but only because I have written said contracts and can ask it all the right questions/prod it into adding what I need.

And the next iteration of ChatGPT will likely generate a better contract because your interaction with it will be used as training data

Re: Testing GPT 4's code-writing capabilities with some real world problems

#90

Post by jacobedawson »

noitpmeder wrote: post #76
jacobedawson wrote: post #16

Earlier quoted context omitted.

I think most people see where the puck is going, and even where it is right now is very impressive. It's not hard to see that it will be likely less than 5 years before it will be able to do what you did on day 2, and much more, at a tiny fraction of the cost, with no downtime, no attitude problems, sick days, etc. The things you mentioned (taking on board feedback, testing across devices, iterating on solutions) doe…

Personally I think it's 99% hype. The current iteration and architecture of these systems means they will never be at the level where they can actually replace a programmer. The best they will ever get is barfing up snippets for a programmer (who still needs the industry-specific knowledge to VERIFY the snippets). Additionally, "the rate of increase in capabilities" is very much a false flag. Past performance (especi…

That's why I said it was unpredictable, in both directions.
Post reply on HN