Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

431–440 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#431

Post by coldtea »

dimitrios1 wrote: post #370
coldtea wrote: post #354

Earlier quoted context omitted.

That's irrelevant though, because actual programming is very much about computers, and about writing things the majority of which have already been formalized by computer scientists, and gluing code and programs inti pipelines...

I just adore the hubris on full display here to say what one of the world's best computer science educators has to say is irrelevant.

First, this is an argument by authority. One of the "world's best computer science educators" can still say all kinds of wrong stuff. Especially someone as opinionated as Dijkstra, with many controversial opinions other world leading computer science educators disagree with.

Second, relevance and applicability is depending on context. On the context of this discussion, about practical programming in the trenches, what Dijkstra said is irrelevant.

For starters, he was talking about computer science.

Programmers in the trenches (what the thread is discussing) don't do computer science. They merely apply stuff developed by computer scientists (but also by non-scientific practioners in the field), in informal (and often haphazard) manner, to solve business problems (often the same problems again and again, which is something totally beneath a scientist).

So, yes, something Dijkstra had said can still be wrong, regardless of his achievements. And it can also far more easily be irrelevant, as this just needs his saying to be unrelated to the discussion.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#433

Post by dgacmu »

dimitrios1 wrote: post #370
coldtea wrote: post #354

Earlier quoted context omitted.

That's irrelevant though, because actual programming is very much about computers, and about writing things the majority of which have already been formalized by computer scientists, and gluing code and programs inti pipelines...

I just adore the hubris on full display here to say what one of the world's best computer science educators has to say is irrelevant.

As a computer science educator, it's worth remembering that Dijkstra was incredibly full of himself and his specific formal view of the field. There are large parts of computer science that have nothing to do with the machine and there are other parts that do. There's no single cut and dried answer here, we're a pretty diverse field that spans everything from mathematics to engineering to a touch of art and human factors.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#434

Post by nicholasjarnold »

IKLOL wrote: post #229
quitit wrote: post #200

Earlier quoted context omitted.

This is so common in many types of business, and usually a very difficult point to articulate so thank you for that. It's something to be shown to those ringing the death-knell for programmers, artists, and the like. Those death-knell types seemingly aren't aware of what day to day operations looks like and how AI makes a great tool, but doesn't necessarily deal with the very human factors of whims, uncertainty, reac…

I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.

> Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.

Yep. This is where I'm at in terms of personal armchair predictions of the future.

I expect the labor market will be tough for more junior software engineers in the coming years. This might indeed cause backpressure in the supply of new grads/new labor force entrants in this family of fields ("software development").

However, the "highly paid senior" is only around for so long before achieving financial independence and just not working anymore. Then what? The company didn't hire juniors because the "highly paid senior" did all the work. Whoops. Now the company lacks the pipeline of people to replace that senior.

It'll sort itself out in time, but the next decade will be interesting I think. Some companies will realize that they must make investments into the future of the labor force and will do better in the longer term. Other companies might indeed "fire the juniors" for some short term gains and find themselves lacking replacement staff later.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#435

Post by logifail »

coldtea wrote: post #332
danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

> Basically, 99% of the effort is NOT WRITING CODE. It's all about communication with people, and problem solving. If we use GPT-X in our company, it will help us with 1% of our workload First, if you did have GPT-X (say GPT-10) in your company, there wouldn't be much back-and-forth communication either. Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements…

> Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements might be given by a GPT-X.

What happens if one (or more) of the GPT-Xs starts having hallucinations while they're busy working on this project?

> Second, even if that's not the case, the part of doing the communication can be handled by non-programmers.

I was in a meeting with a sales manager and tech team a few days ago. Sales manager had been talking to a potential customer about a product that neither he nor the customer properly understood. They both thought it would be an excellent fit for a particular (new) purpose, one for which it was not designed.

As it turned out, everyone on the tech team knew that both sales manager and customer were utterly and catastophically wrong, but it took the best part of an hour to finally convince him of this.

It's quite hard to have useful conversations about stuff if you don't actually understand it.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#436

Post by InCityDreams »

jokoon wrote: post #426

People need to understand that AI doesn't think, doesn't have insight or intuition. AI just repeat patterns it saw in a huge database, but are not able to understand what is going on. Nobody can really understand what's inside a trained neural network, and nobody is really looking. No psychologist or neuro-scientist can really understand how a human brain, a mouse brain or even an ant brain or a fly brain even works,…

I introduced my colleagues to chatgpt this morning and they're knocked out. We deal with people, and the people answers are considerably more thoughtful than 'improved search'. Not sure where btc and and pseudo-scam come into it.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#437

Post by yc-kraln »

Quarrelsome wrote: post #228
staminade wrote: post #211

Earlier quoted context omitted.

While I think there's truth to what you say, I'd also point our that workers in many pre-automated industries with an "artisan" approach also considered themselves irreplaceable because they figured, correctly, that nobody could build a machine with the capability of reproducing their workflow, with all its inherent uncertainty, flexibility and diverse physical and mental skills. What they failed to predict was that…

That's an extremely interesting thought. Perhaps we will see organisations in the future structure themselves more like a suite of unit tests. Instead of getting a developer or software house to plug a specific need in their org and being entirely outside of the development process: they will reflect the development process organisationally to ensure they catch any problems with the current output and just feed the b…

This is already how automotive companies work. They are huge organizations which do four things: marketing, financing, design, and requirements. The supplier management and project management all fall under requirements management and enforcement.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#438

Post by wlesieutre »

ryandrake wrote: post #402
lebuffon wrote: post #357

Earlier quoted context omitted.

That's an interesting case and use of the word efficient. Developer time reduced by 5X CPU time increased by 500x When CPU cycles are cheap it's a logical tradeoff but it still grates against my sensibilities.

We should stop thinking about the trade-off as between developer time and CPU time. The CPU is an inanimate object which doesn't mind pushing around more electrons. What we're really throwing under the bus when we optimize for developer time is the customer's time . If an inefficiency saves a developer 10 days of work, but results in an operation taking 100 milliseconds longer, and you have 50M customers who do that…

Don't forget customer's battery

Re: Testing GPT 4's code-writing capabilities with some real world problems

#439

Post by cguess »

coldtea wrote: post #344
vages wrote: post #253

Earlier quoted context omitted.

If increased productivity equaled job loss, there would be two programmers alive today, doing the same job as the fewer than 10000 programmers using punch cards as we entered the year 1950. A lot of projects today are not even greenlit because they would be too expensive to make. For instance, there are a lot processes in almost every country that require you to file paper forms, even though we have had web forms and…

> If increased productivity equaled job loss there would be two programmers alive today, doing the same job as the fewer than 10000 programmers using punch cards as we entered the year 1950. The only reason it's not the case in this example is because computers at the time were a tiny early adopter niche, which massively multiplied and expanded to other areas. Like, only 1 in 10,000 businesses would have one in 1950,…

Have no doubt, we will find new places to put computers. In the 80's and even the 90's everyone said the same thing, "Why do I need a computer? I can do everything I do already without a problem?" Well, turns out with computers you could do 12 more things you can never considered. Consider the interoffice memo: it'd take what, 1-2 hours to get a document from one floor to another through the system? Cool, you can work on 2-3 projects at a time maybe, because that's all the bandwidth allowed. Along comes email and ups that to 5-6 because now the communications can be pretty consistent. It's still not perfect, because what if you're at lunch or the gym? Then came Blackberries and all of a sudden its 12-15 projects at once. Then Slack because you don't even have to think. Now add this.

Notice that during that time there weren't all of a sudden less programmers, or managers or sysadmins, if anything there's more. If anything everyone is even more stressed with more to do because of the context switching and 24/7. That's why this will do, I'd bet money on it.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#440

Post by anon7725 »

somenameforme wrote: post #393
jmuguy wrote: post #256

Earlier quoted context omitted.

This is exactly why I'm so tired of these "can AI write code" think pieces. I assume people writing this crap aren't actual developers. Maybe its management fan-fiction.

I suspect there's a strong invisible ideological undercurrent pushing a lot of this. When I was younger and enthusiastic about things like a universal basic income, I would often follow the latest murmurings, always ready to let my hype (and hyperbole) meter go to 11. I remember when I saw the first news about some drone delivering a pizza (in New Zealand?) I immediately jumped to it foretelling the imminent end of a…

I think the difference here is that LLMs are powerful, general purpose and suited to take on many common office tasks as well as accelerate programming work. I think “we need 10% fewer white collar workers” is a massively destabilizing scenario that is not very far fetched.

I also completely agree that society/politics will not have caught up with these developments in time to mitigate them.

Post reply on HN