Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

521–530 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#521

Post by sumtechguy »

caddemon wrote: post #430
sumtechguy wrote: post #223

Earlier quoted context omitted.

I can see why educators are getting involved. They are seeing it change how their students do work in real time. I grew up and using a pocket calculator was verboten. You just did not do it. You better learn and memorize all of that stuff. Spin on 10 years after me and all kids have them now. If you have the right app on your phone the thing will OCR the problem and auto solve it and show you the steps. ChatGPT and i…

Yeah there is a broader issue here for sure. I don't think that means within a few weeks of the launch of the pocket calculator there should have been dozens of random talks from people that have no understanding of what a pocket calculator does trying to pitch a curriculum based around it. These posts ooze hype bullshit, not nuanced talk about pros and cons and how it will affect education. ChatGPT should be thought…

The thing was when pocket calculators came out they were expensive to get ahold of. Like 500-1000 bucks for one (in 1970s terms that is a lot). This sort of NLM is effectively free. All you need is an account and the cell phone you already have. This is disrupting classrooms right now. Everyone I talk to that has a kid is saying their kids are using it for coursework. I get why teachers are 'oh no...' as of right now it is exploding on them. They have not had time to digest it. Unfortunately at this point chatgpt is like an reporter summing up information it has no idea what it is talking about and putting it into nice prose that looks good. So while in your area of expertise you can spot out the issues. But you flip the page and it is on a different subject it will be tough to spot out the errors. Simply because you do not know and attribute expertise they do not really have onto it. This NLM is that but turned up to 11.

"where the puck is going" is something to consider and we will 100% get that wrong, but right now this thing is creating waves and making issues now.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#522

Post by lend000 »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

What if they replace "us" with GPT and one programmer to glue together the snippets it provides?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#523

Post by whateveracct »

noitpmeder wrote: post #108
Jensson wrote: post #97

Earlier quoted context omitted.

If you do nontrivial coding then GPT wont help you, so that isn't relevant for the discussion.

I disagree. GP, by their statement and appeal "I've worked on adtech, crypto, fintech, gamedev, startup founder, ...", implies that he has worked on complicated software issues, and that he thinks GPT is a promising replacement for developers working on those problems.

Opposite.

I did a startup for years and the backend was all boring Golang talking to Postgres. I am confident that if GPT had produced identical code it would've been worse. Because shit hit the fan many times due to misunderstandings or other things that cause bugs. Because I wrote the bugs, I was able to fix them. Making coding more of a blackbox and treating coding as menial labor would have definitely caused us to lose customers and miss important deadlines.

Maybe the next way to move the goalposts is "GPT will debug for you someday and be trained on the entire transitive closure of your dependencies."

That sort of thing could actually be useful as a sort of hint, but it isn't replacing devs any more than ripgrep did. To be honest, ripgrep through the source of my deps is already highly efficient and points me to things often.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#524

Post by whateveracct »

capableweb wrote: post #180
whateveracct wrote: post #55

Earlier quoted context omitted.

Programming takes little effort. It's at most 20% of my effort. Feels like GPT is programming with extra steps. Maybe it's for people who can never think programming is easy. Clearly there's a lot of such types. Explains a lot.

I think programming is very easy most of the time, most time I spend is just typing/moving code around, figuring out the solution is the easy part for me, only the computer brain coordination is slowing me down most of the time. But there are things that are harder for me, or more complex maybe. I struggle with math, and always had, so anything involving heavy math or algorithms is harder for me (I'm a hacker, not a…

So you write the unit tests yourself to confirm the code you say you can't understand is correct? That's an interesting approach. But you'd probably need more tests than usual to gain confidence, so you are losing efficiency there (although the ceiling-raising nature of it is interesting).

What happens in production when there's a bug in the complex code you punted to GPT? How do you debug?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#525

Post by serpix »

anon7725 wrote: post #427
coldtea wrote: post #332

Earlier quoted context omitted.

> Basically, 99% of the effort is NOT WRITING CODE. It's all about communication with people, and problem solving. If we use GPT-X in our company, it will help us with 1% of our workload First, if you did have GPT-X (say GPT-10) in your company, there wouldn't be much back-and-forth communication either. Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements…

> Second, even if that's not the case, the part of doing the communication can be handled by non-programmers. It can’t - only programmers will know which follow up questions to ask. GPT will be able to ask those questions before non-programmers will be able to. Half the work is nitpicking on date formats or where some id comes from or if a certain field is optional, etc.

Last paragraph is spot on. Would also add a vast amount is spent looking at the code and the system and just understanding what goes where.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#526

Post by sanderjd »

_fizz_buzz_ wrote: post #451
vages wrote: post #253

Earlier quoted context omitted.

If increased productivity equaled job loss, there would be two programmers alive today, doing the same job as the fewer than 10000 programmers using punch cards as we entered the year 1950. A lot of projects today are not even greenlit because they would be too expensive to make. For instance, there are a lot processes in almost every country that require you to file paper forms, even though we have had web forms and…

This is true for other domains also. My wife translates for an ecommerce business. A lot of what she does is very automated. The software she uses remembers all phrases she has translated and uses a word bank she manages for special parts and uses DeepL for new translations which are then only proofread and fine tuned. It's amazing how much stuff she can translate that way in a day (no idea how many thousands words).…

Yep! I think this is both the optimistic case, and the version of the future that seems most likely to me.

I'm still a bit bummed about it, because just writing and structuring code is something that often brings me joy (and maybe your wife feels similarly about the actual translation work...), but at the end of the day, I realized well before all this AI craze that it really isn't the most valuable thing I do in my job.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#527

Post by Planktonne »

Sunhold wrote: post #296
Planktonne wrote: post #258

Earlier quoted context omitted.

I think we're talking at cross-purposes here. I'm not underwhelmed by it being not as good as good humans at [task]; that's not a surprise. I'm saying that most people I see who are impressed by it for any given task are people who are not well-equipped to judge performance on that task.

I'm impressed by the talking dog because it's a dog that can talk, not because I think it's a great orator.

I trust that you're familiar enough with dogs to tell barking apart from talking.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#528

Post by risyachka »

IKLOL wrote: post #229
quitit wrote: post #200

Earlier quoted context omitted.

This is so common in many types of business, and usually a very difficult point to articulate so thank you for that. It's something to be shown to those ringing the death-knell for programmers, artists, and the like. Those death-knell types seemingly aren't aware of what day to day operations looks like and how AI makes a great tool, but doesn't necessarily deal with the very human factors of whims, uncertainty, reac…

I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.

Companies will continue to hire even more people. Their single point is growth.

If one can go the job of 5 - they revenue per employee will increase.

They will hire even more employees.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#529

Post by protomolecule »

WillPostForFood wrote: post #500
vages wrote: post #253

Earlier quoted context omitted.

If increased productivity equaled job loss, there would be two programmers alive today, doing the same job as the fewer than 10000 programmers using punch cards as we entered the year 1950. A lot of projects today are not even greenlit because they would be too expensive to make. For instance, there are a lot processes in almost every country that require you to file paper forms, even though we have had web forms and…

If increased productivity equaled job loss, there would be two programmers alive today Increased productivity doesn't necessarily lead to overall job loss, but it will eventually in the area where the productivity is realized. Agricultural employment is a very clear example. https://cdn2.vox-cdn.com/assets/4565243/Ag_workforce.png

Well, yes, it got down, but only from 12 mln to 5.5 mln people in agriculture.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#530

Post by jimbokun »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

How good is GPT-4 at writing emails to nail down requirements?
Post reply on HN