Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

301–310 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#301

Post by franl »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

Just wait until XXQ adopts the same AI technology to keep the AI-using company’s business. Then the AI can simply coordinate with one another, and make the appropriate changes faster than currently possible. Microsoft is well positioned to do something like this and already working toward this end to end collaborative AI.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#302

Post by ArchitectAnon »

Here's my perspective on this as an Architect: Most construction details have been done before, they could be easily reproduced by an AI surely? There's usually just a few things that are different from the last time I have drawn it. A few 3D interactions with other components that need to be reasoned about. They are not that complicated individually.

But yet I see this problem as well just using old fashioned automation let along AI to save time. I find that if you haven't drawn the 2D section through all the different edge cases of a particular thing you are trying to design, you haven't done the analysis and you don't really understand what's happening. I've made mistakes where I've been working in 3D on something complicated and I've had to hide some element to be able to view what I'm working on, only to find later that when I turn everything on again I've created a clash or something impossible to build. That's why we still do 2D drawings because they are an analysis tool that we've developed for solving these problems and we need to do the analysis, which is to draw section cuts through things, as well as building 3D models. After all, if models were such a good way to describe buildings, then why weren't we just building physical scale models and giving them to the builders 100 years ago; it's because you can't see the build-up of the layers and you can't reason about them.

Reading this article I get the same sense about software engineering, if you haven't solved the problem, you don't really understand the code the AI is generating and so you don't really know if it is going to do what you've tried to describe in your prompt. You still have to read the code it's generated and understand what it is doing to be able to tell if it is going to do what you expect.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#303

Post by hgsgm »

muzani wrote: post #294
superasn wrote: post #219

Earlier quoted context omitted.

If anything it will be much better as it won't mind waiting and asking for answers nor get frustrated with incompetence.

Honestly one of my favorite early applications of AI was as customer service. The CS people I talk to always seem pissed with something . One conversation: Me: "Hello, I'd like to ask why all the loyalty points from my MILES card is gone." CS: "Sir, it says here that you upgraded your MILES card to a SMILES card. If you did not request a points transfer, they are gone." Me: "Okay, how do I do that?" CS: audible eyero…

Customer service is reading from a scipt. Why would AI help you? Customer service exists to deflect you from getting back what the corpo stole from you. The question is it is easier for you to social engineer the CSR into helping you, or prompt engineer the AI into helping you.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#304

Post by hgsgm »

theshrike79 wrote: post #230
danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

I once had to implement a Swedish standard for energy usage reporting. EVERY FIELD IN THE STANDARD WAS OPTIONAL. That was one of the most not-fun times I've had at work :D Every single field was either there or it was not, depending whether the data producer wanted to add them or not, so the whole thing was just a collection of special cases.

So, like Probocol Buffers?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#305

Post by netfortius »

franl wrote: post #301
danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

Just wait until XXQ adopts the same AI technology to keep the AI-using company’s business. Then the AI can simply coordinate with one another, and make the appropriate changes faster than currently possible. Microsoft is well positioned to do something like this and already working toward this end to end collaborative AI.

Absolutely this. I still don't understand why people stop seeing the AI at one end of the business/computing/requirements gathering model. You should have it at both ends, "converging" towards the target.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#306

Post by HervalFreire »

Guys all of this happened within a couple months.

If you guys are confident about the entity as it is right now not taking over your job, what if I double the accuracy of gpt output?

What if I double it again? Then again? And again? And again?

You guys realize this is what's coming right? This thing literally is a baby as of now.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#307

Post by breakingrules »

i was trying to get it to make a document scanner last night, it apologized to me like 10 times and we eventually got running code but the result was way off. this thing can write code but you're not gonna rely on it and nobody is gonna know it enough to edit it. it is not there yet, still very helpful for small things or extremely simple things. if you tell it to give you an express server with socket.io and your db, it will probably set that up for you perfectly.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#308

Post by [deleted] »

maxdoop wrote: post #109

It seems so many of you guys are extremely lucky to be working on novel problems require elegant new solutions each day. That must be the case, otherwise I don’t understand these comments shrugging off GPT-n capabilities around coding. “Psh, it’s just doing stuff it saw from its training data. It’s not thinking. It can’t make anything new.” In my 11 years as a professional software engineer (that is, being paid by co…

[deleted]

Re: Testing GPT 4's code-writing capabilities with some real world problems

#309

Post by anonzzzies »

davewritescode wrote: post #274
anonzzzies wrote: post #164

Earlier quoted context omitted.

> Please elaborate. I simply know firsthand (i'm old, i have manager, cto, ceo friends who I go golf and play squash with) that people in data entry and programming have been let go in the past weeks because 1 person could take over their work using the gtp/chatgpt api's and do their work faster with less errors. I am recommending the same in my company as a lot of my colleagues are doing nothing anymore as the skill…

> simply know firsthand (i'm old, i have manager, cto, ceo friends who I go golf and play squash with) that people in data entry and programming have been let go in the past weeks because 1 person could take over their work using the gtp/chatgpt api's and do their work faster with less errors. I am recommending the same in my company as a lot of my colleagues are doing nothing anymore as the skilled seniors are doing…

> This isn't software engineering work, this is 21st century data entry with some code. This is exactly the type of menial work that should be automated by AI.

Who said it was? This is what most programmers do all day, that's the point. These people can be replaced now without writing specialised software for the case. It is programming, not software engineering and it is what people are doing all day long (and longer) who are called programmers / software engineers in most companies all over the world. You can disagree with it, but that doesn't change much.

So you are now putting the bar higher which is not fair; the fact is that people who have the title 'programmer' and even 'software engineer' are now readily replaced by AI. If you don't agree with the title; I don't either, but reality is what it is.

I would say basically the point is ; there are way way too many people being 'programmers' (but not limited to this field) who can be replaced; only a few % should remain as the rest does what you wouldn't call programming, but the rest of the world does. Search twitter for 'html programmer' and such. Millions and millions will never be programmers and your definition, but have a high paying job (for their country) working as a programmer.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#310

Post by agos »

jmuguy wrote: post #256
danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

This is exactly why I'm so tired of these "can AI write code" think pieces. I assume people writing this crap aren't actual developers. Maybe its management fan-fiction.

see also "the future is no code!" fad of a couple of years ago
Post reply on HN