Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

621–630 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#621

Post by CamperBob2 »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

In your scenario, what will happen is that there will never be a need to contract out whatever the customer is trying to do.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#622

Post by krainboltgreene »

stormfather wrote: post #184
whateveracct wrote: post #86

Earlier quoted context omitted.

I don't even think it's useful for the incompetent. I do think the incompetent think it's useful tho.

That's like saying a calculator isn't useful just because you're good at mental math. The ability to adapt and extract value from new tools is a mark of competence.

Imagine a calculator that crowdsources it's answers from the populace, how useful is that tool to the incompetent? How easily would the incompetent be able to tell it's bad?

That's what we're talking about: An autocomplete machine that has been trained on a million blog posts that contain some code, that maybe correct, incorrect, secure, insecure, outdated or uptodate and the machine can't tell the difference! It only knows what is and is not likely! So the more popular the wrong answer was replicated on the web, the more likely that's what you'll get!

Re: Testing GPT 4's code-writing capabilities with some real world problems

#623

Post by krainboltgreene »

legohead wrote: post #455

I was just complaining to my friend about how much trouble I'm having with it. I purchased the $20 GPT-Plus so I could use GPT-4 after reading someone on HN say that GPT-4 is "scary impressive" at writing code. I have two tasks I wanted it to try, both making use of public APIs, starting from scratch. In short, it was frustrating as hell. Never-ending import problems -- I'd tell it the error, it'd give me a different…

> I'd tell it the error, it'd give me a different way to import, only leading to a new import problem

It's dataset is thousands of blogs posts and stack overflow questions about this very thing, of course the autocomplete engine is going to predict the next response be "another way of doing x".

Re: Testing GPT 4's code-writing capabilities with some real world problems

#624

Post by DuckFeathers »

nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

It's a similar waste of my time to respond to comments like this... but I will at least try to give some pointers so that those who encounter this comment won't immediately fall into that line of thinking:

- a lot of programmers, including experienced ones, are absolutely infantile and they only have a job because there is a big shortage of programmers; not all of them get better with experience... hence a significant part of software development is dealing with problematic programmers and problems created by them.

- GPT is not that great a programmer but a great thing about it is that it is not a human... and one can get thousands of instances of them for the price of one human. You only need one of those instances to produce usable code.

- there have been many changes throughout the years which have definitely replaced a lot of programmers: library distribution services (pypi, npmjs), better software development tools and practices, SaaS delivery model, better programming languages etc.; so far, because the market need for programmers has continued to increase, most programmers continue to have jobs; this won't last forever.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#625

Post by gumballindie »

danparsonson wrote: post #321
iambateman wrote: post #242

The replies here are defensive and I think misguided. Yes, a programming job doesn’t solely require typing code. But the reason we have well-paid programming jobs is because there is a specialized skillset required to understand a body of syntax that takes several years to really grasp. The difference is that writing a well-formed prompt is massively easier to teach than writing the code itself, for similar results.…

The most challenging part of software development (and the reason we have well-paying jobs) is not understanding syntax, it's analysing and abstracting a problem domain into a set of cleanly separated modules that interact to solve those problems. That being the case then, actually none of us is getting replaced by GPT-n any time soon - 'prompt engineering' will just become the new Javascript, only more abstract; jus…

Correct. But once everyone has that tool in their toolbox everyone will become more productive, meaning skills scarcity will be greatly reduced. In turn that will lead to massive wage depression.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#626

Post by diedyesterday »

So this guy is basically complaining about GPT-4 not being a super-intelligence. Still that makes it more powerful and versatile thae the great majority of programmers out there.... And the game is only getting started. This is just the warmup.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#627

Post by kaoD »

jjkeddo199 wrote: post #508
mbesto wrote: post #378

Earlier quoted context omitted.

> imagine when they realize one overworked highly paid senior can replace 10 juniors This already happens, the market is just not very efficient about it, e.g. a highly paid senior dev is not working at a company that only needs 2-3 developers, they're working at Google with 100's of devs.

Disagree -- I think chatGPT will make juniors more palatable to hire. ChatGPT will basically give juniors a free pair programmer to baby-sit their work/progress. Why pay extra for senior devs when juniors can become much more efficient thanks to ChatGPT becoming stack-overflow on steroids. I think the wage gap between junior and senior will actually drop massively. I predict teams will keep 1-3 architect level positi…

I predict the wage gap will actually widen. StackOverflow-on-steroids can only give your so much, so seniors will still be in demand for what they do. It's just the competition between juniors will be fiercer so junior wages will drop.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#628

Post by illiarian »

CamperBob2 wrote: post #619
notJim wrote: post #534

Earlier quoted context omitted.

I dunno, I feel like we've been trying to find new places to put computers for a couple decades now, and the effort is kind of losing steam. Every week there are threads about how useless a lot of these efforts are.

You just don't hear about all the computers, that's all. There's one on your car's key fob, and likely 100 more in the car. Your coffee grinder has one. So does your dishwasher. And your electric blanket.

There are 100-150 CPUs in a modern car.

...And also over 100 bugs in the spec that governs communication between car systems: https://youtu.be/hXnS_Xjwk2Y?t=899 (around 15 minute mark)

Re: Testing GPT 4's code-writing capabilities with some real world problems

#629

Post by chpatrick »

zeroonetwothree wrote: post #350
Jevon23 wrote: post #279

Earlier quoted context omitted.

I don’t think anyone is claiming to not be impressed. Yes, we’re all impressed. This was unimaginable sci-fi just 5 years ago. You did it, we’re impressed! You won. The next step after that is for people to figure out how to justify their continued relevance. I think that’s a pretty natural reaction.

I’m not concerned. If AI replaces my job then ok I’ll find something else to do. Self driving cars seem to have stagnated so maybe I’ll be an Uber driver

I wouldn't count on that, you can call a self driving taxi right now in Phoenix.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#630

Post by chpatrick »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

The way I see it is that instead of hiring someone you'll be able to add a bot on slack that you can communicate with, laughs at your jokes and writes perfect PRs in milliseconds.
Post reply on HN