Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

541–550 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#541

Post by [deleted] »

jjkeddo199 wrote: post #508
mbesto wrote: post #378

Earlier quoted context omitted.

> imagine when they realize one overworked highly paid senior can replace 10 juniors This already happens, the market is just not very efficient about it, e.g. a highly paid senior dev is not working at a company that only needs 2-3 developers, they're working at Google with 100's of devs.

Disagree -- I think chatGPT will make juniors more palatable to hire. ChatGPT will basically give juniors a free pair programmer to baby-sit their work/progress. Why pay extra for senior devs when juniors can become much more efficient thanks to ChatGPT becoming stack-overflow on steroids. I think the wage gap between junior and senior will actually drop massively. I predict teams will keep 1-3 architect level positi…

[deleted]

Re: Testing GPT 4's code-writing capabilities with some real world problems

#542

Post by lolinder »

dgacmu wrote: post #433
dimitrios1 wrote: post #370

Earlier quoted context omitted.

I just adore the hubris on full display here to say what one of the world's best computer science educators has to say is irrelevant.

As a computer science educator, it's worth remembering that Dijkstra was incredibly full of himself and his specific formal view of the field. There are large parts of computer science that have nothing to do with the machine and there are other parts that do. There's no single cut and dried answer here, we're a pretty diverse field that spans everything from mathematics to engineering to a touch of art and human fac…

What you say about Dijkstra is true, but we're not actually talking about a Dijkstra quote here, we're talking about Harold Abelson.

https://en.m.wikipedia.org/wiki/Hal_Abelson

Re: Testing GPT 4's code-writing capabilities with some real world problems

#543

Post by riku_iki »

anonzzzies wrote: post #112
theK wrote: post #100

Earlier quoted context omitted.

Token cap will probably be the biggest problem here. After validation. After getting the changes to disk, documented, actually compiling, etc… But the biggest problem is that transferring the nuance that is external to the code base is typically really tiresome and lengthy grunt work and again token cap.

They went from 4K to 32k in a few months. Wait a few years.

we don't know what that jump means, did they increased model proportionally, or slapped some projection layer with quality degradation?..

Re: Testing GPT 4's code-writing capabilities with some real world problems

#544

Post by lolinder »

coldtea wrote: post #431
dimitrios1 wrote: post #370

Earlier quoted context omitted.

I just adore the hubris on full display here to say what one of the world's best computer science educators has to say is irrelevant.

First, this is an argument by authority. One of the "world's best computer science educators" can still say all kinds of wrong stuff. Especially someone as opinionated as Dijkstra, with many controversial opinions other world leading computer science educators disagree with. Second, relevance and applicability is depending on context. On the context of this discussion, about practical programming in the trenches, wha…

We're not talking about Dijkstra, OP is paraphrasing Harold Abelson.

https://en.m.wikipedia.org/wiki/Hal_Abelson

Re: Testing GPT 4's code-writing capabilities with some real world problems

#545

Post by spaceman_2020 »

blibble wrote: post #396
woeirua wrote: post #323

Earlier quoted context omitted.

Two points: GPT4 is significantly better in this regard, and you should be concerned about the rate of progress more than it’s actual capabilities today.

I suspect (and sort of hope) they're close to hitting a local maxima presumably they've already fed it all the code in the world (including a load they probably aren't licensed to)

Don't you think it's going to improve much more now that it has hundreds of billions of pointed queries being typed into it every month?

This is data at a scale OpenAI did not have access to before.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#546

Post by whateveracct »

capableweb wrote: post #539
whateveracct wrote: post #524

Earlier quoted context omitted.

So you write the unit tests yourself to confirm the code you say you can't understand is correct? That's an interesting approach. But you'd probably need more tests than usual to gain confidence, so you are losing efficiency there (although the ceiling-raising nature of it is interesting). What happens in production when there's a bug in the complex code you punted to GPT? How do you debug?

Basically. I'm not sure I'm losing much efficiency, my previous approach was contacting friends who know how to do math in programming, sometimes it could days before I could move past the issue. And the result is the same, ending up with a piece of code I mostly understand, but struggled to write myself so the knowledge is fleeting at best. Just to be clear, the context here is me writing games for fun, while strugg…

Has using it ended up teaching you anything? Like from reading the produced solutions?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#548

Post by esfandia »

shagie wrote: post #476
staminade wrote: post #211

Earlier quoted context omitted.

While I think there's truth to what you say, I'd also point our that workers in many pre-automated industries with an "artisan" approach also considered themselves irreplaceable because they figured, correctly, that nobody could build a machine with the capability of reproducing their workflow, with all its inherent uncertainty, flexibility and diverse physical and mental skills. What they failed to predict was that…

The artisan of old created one bowl at a time and made artisan pots. A machine that makes bowl can automate the artisan's job away. However, the next challenge is that the machine itself is now an "artisan" device. I'm sure the first bowl printing machine ( https://youtu.be/bD2DNSt8Wb4 ) was purely artisan... but now you can buy them on Alibaba for a few thousand dollars ( https://www.alibaba.com/product-detail/Print…

> There will always be an artisan making the first version of anything.

Until we reach the bootstrap point (the singularity?), i.e. when the GPT-making machine is GPT itself. Or maybe we're still one level behind, and the GPT-making machine will generate the GPT-making machine, as well as all the other machines that will generate everything else.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#549

Post by lionkor »

WinstonSmith84 wrote: post #472
lionkor wrote: post #240

Earlier quoted context omitted.

For me, its mostly that I have used GPT-3.5 a little for programming C++, and I wasnt impressed. For one, it made horrible, glaring mistakes (like defining extern functions which dont exist, using functions which are specific to a platform im not using, etc.), stuff beginners would do. It also decided to sneak in little issues, such as off-by-one errors (calling write() with a buffer and a size that is off by one in…

GPT4 is very different from 3.5. I've asked it today to write some unit tests given the code of the class (~200 lines) and the methods I wanted to cover and it did that just perfectly. It put asserts where it made sense (without me asking to do it), and the unit test code was better written than some code I've seen written by (lazy) humans. It's not perfect sure and it's easy to get a bad response but give OpenAI a f…

what kind of unit tests are these? is it `check_eq(add(1, 2), 3)` or "check for possible exceptions, cover edge cases, test extremes of this super important db function"

Re: Testing GPT 4's code-writing capabilities with some real world problems

#550

Post by race2tb »

I like how it ends with it can write code that is repetitive and has many examples of solutions floating around which is most of coding.

It will probably edge upward as well as time goes by till there are only very edge problems that it cannot solve. Even then I would use it to write the broken down version of the solution. It is going to be getting fed by pretty much every programmer,knowledge profession on the planet using copilots of some sort. Eventually it will have knowledge transferred everything humans can do into its model.

Post reply on HN