Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

391–400 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#391

Post by osigurdson »

osigurdson wrote: post #177
Jensson wrote: post #129

Earlier quoted context omitted.

Yeah, but the jump to 32k didn't take a few months, it was years in the making. Otherwise you could extrapolate with "yesterday we had 4k, today we have 32k, tomorrow we will have 256k", that isn't how we do it. If we follow the same exponential pace 256k would have to wait 3 years, and even that is unlikely.

I think it is better to assume that we are at the beginning of the s-curve, not the end.

Wow. I never expected to be downvoted for such a comment. Downvotes please elucidate as to why we are in the asymptomatic phase with LLMs.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#392

Post by antegamisou »

dimitrios1 wrote: post #370
coldtea wrote: post #354

Earlier quoted context omitted.

That's irrelevant though, because actual programming is very much about computers, and about writing things the majority of which have already been formalized by computer scientists, and gluing code and programs inti pipelines...

I just adore the hubris on full display here to say what one of the world's best computer science educators has to say is irrelevant.

That's the most humble HNer for you btw.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#393

Post by somenameforme »

jmuguy wrote: post #256
danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

This is exactly why I'm so tired of these "can AI write code" think pieces. I assume people writing this crap aren't actual developers. Maybe its management fan-fiction.

I suspect there's a strong invisible ideological undercurrent pushing a lot of this. When I was younger and enthusiastic about things like a universal basic income, I would often follow the latest murmurings, always ready to let my hype (and hyperbole) meter go to 11. I remember when I saw the first news about some drone delivering a pizza (in New Zealand?) I immediately jumped to it foretelling the imminent end of all delivery jobs, with broad customer service not especially far behind. There's even the fully automated Henn-na hotel in Japan, I mean omg!

In my naivete, the idea I had is that if the jobs disappeared en masse then a social solution to the economy would be forced to be enacted. So I was essentially hoping to see the destruction of normalcy and employment in any new technology. I would expect that view is not uncommon given the direction of contemporary education. It feels analogous to cows hoping for the end of beef/milk harvesting. My beleaguered bovine buddies, what awaits you there is something rather different than cowtopia.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#394

Post by jdminhbg »

lebuffon wrote: post #357
zeroonetwothree wrote: post #342

Earlier quoted context omitted.

How much more efficient is programming in Python with modern tools vs assembly? I would bet more than 5x

That's an interesting case and use of the word efficient. Developer time reduced by 5X CPU time increased by 500x When CPU cycles are cheap it's a logical tradeoff but it still grates against my sensibilities.

5X is a gigantic underestimate of how much developer time has improved. How long would it take a good assembly programmer to implement a CRUD web server connected to a database? Way more than 5x longer what a mediocre Python programmer would need.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#395

Post by namuol »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

> It's all about communication with people, and problem solving. [...] So, I couldn't care less about it.

These things seem very ripe for LLM exploitation...

Re: Testing GPT 4's code-writing capabilities with some real world problems

#396

Post by blibble »

woeirua wrote: post #323
lionkor wrote: post #240

Earlier quoted context omitted.

For me, its mostly that I have used GPT-3.5 a little for programming C++, and I wasnt impressed. For one, it made horrible, glaring mistakes (like defining extern functions which dont exist, using functions which are specific to a platform im not using, etc.), stuff beginners would do. It also decided to sneak in little issues, such as off-by-one errors (calling write() with a buffer and a size that is off by one in…

Two points: GPT4 is significantly better in this regard, and you should be concerned about the rate of progress more than it’s actual capabilities today.

I suspect (and sort of hope) they're close to hitting a local maxima

presumably they've already fed it all the code in the world (including a load they probably aren't licensed to)

Re: Testing GPT 4's code-writing capabilities with some real world problems

#397

Post by pydry »

IKLOL wrote: post #229
quitit wrote: post #200

Earlier quoted context omitted.

This is so common in many types of business, and usually a very difficult point to articulate so thank you for that. It's something to be shown to those ringing the death-knell for programmers, artists, and the like. Those death-knell types seemingly aren't aware of what day to day operations looks like and how AI makes a great tool, but doesn't necessarily deal with the very human factors of whims, uncertainty, reac…

I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.

Companies were bloated because of the profit potential of tech and organizational dysfunction.

AI wasnt needed to cut down on bloat - some combination of a few companies reaping the majority of the profit and reduced dysfunction is enough.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#398

Post by spdionis »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

The thing is, I think ChatGPT can help a lot with this as well, albeit not in its current form/implementation.

It just needs some knowledge repository centralization.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#399

Post by Kostchei »

Maxion wrote: post #99
shinycode wrote: post #93

Earlier quoted context omitted.

Also if all the company is doing is boring crud app as a service then the whole company is going to disappear once anyone can ask ChatGPT to create a version of the service for themselves

Most CRUD apps in enterprise settings relate to something in the real world. E.g. ordering systems, warehouse management and so on. The hard part here isn’t really the coding - but understanding the business logic and getting it out of the customer what they need. Often the customer hasn’t even got a good enough grasp on their own business to be able to formulate what they need. In the short term this ai tools will m…

Some folks grew up without practicing code- did business / operational /technical things for 20+ years. For someone like that, chatGPT is manna from heaven. 0>0>

I point chatgpt at the general problem and ask about the tech. Check it's answers, drill into specifics. As more questions, get it to write a prototype. (obviously with different parameters to prod- I don't trust open ai not to harvest from my interactions)- ok, now I have proof of concept, if it does the thing I want to do, then go to the engineers and say - hey I want to "X", here is RFC and rough code, .. any problems? if not, how long to push a localised version?

I guess you might call this scripting or prototyping not "real coding" but, damn it's useful not have to fill my head with python etc. Or bore/waste a guy earning 150k for a half day plus to get the basics, then never get my prototype.. because, priority, resources, etc

Post reply on HN