I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…
Testing GPT 4's code-writing capabilities with some real world problems
481–490 of 677 posts
Re: Testing GPT 4's code-writing capabilities with some real world problems
#482Earlier quoted context omitted.
But obviously increasing avg and sr programmer productivity by many x implies many fewer jobs for programmers. That’s the point we’re interested in.
https://en.wikipedia.org/wiki/Lump_of_labour_fallacy > In economics, the lump of labour fallacy is the misconception that there is a fixed amount of work—a lump of labour—to be done within an economy which can be distributed to create more or fewer jobs.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#483The productivity gains will not leave people unemployed, but will give managers the opportunity to start more projects.
The role of a developer will change. We'll be looking at generated and regenerated code. But we'll still be demand by those with ideas and never-decreasing human demand.
This assumes that GPT-X won't end up being used by end-users--bypassing both the C-level, the managers and the developers.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#484So, for a bit of fun, I signed up to GPT-4 thingy plus and I picked a fairly common web application and built it from scratch, only by talking to GPT-4 and copy pasting the code bits. I'm actually taken back by how well it's doing; including providing me some refreshers on stuff I forgot how it should work. I can see it failing at solving complex problems, but like the blog post mentions, most programming isn't new o…
Re: Testing GPT 4's code-writing capabilities with some real world problems
#485Earlier quoted context omitted.
I suspect (and sort of hope) they're close to hitting a local maxima presumably they've already fed it all the code in the world (including a load they probably aren't licensed to)
If you tried GPT4, you probably understood it's not about feeding all the code in the world. GPT4 analyzes and "understands" your code and will answer based on this. Clearly, it will read the variable names and make deductions based on this. It will actually read the comments, the function names and make decisions based on this. And it knows the rules of the language. I mean, I'm writing this because this is what I'v…
it doesn't understand anything, it doesn't make deductions
it's a probability model, and I understand how it's implemented perfectly well, thank you
Re: Testing GPT 4's code-writing capabilities with some real world problems
#486Earlier quoted context omitted.
> Basically, 99% of the effort is NOT WRITING CODE. It's all about communication with people, and problem solving. If we use GPT-X in our company, it will help us with 1% of our workload First, if you did have GPT-X (say GPT-10) in your company, there wouldn't be much back-and-forth communication either. Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements…
> Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements might be given by a GPT-X. What happens if one (or more) of the GPT-Xs starts having hallucinations while they're busy working on this project? > Second, even if that's not the case, the part of doing the communication can be handled by non-programmers. I was in a meeting with a sales manager and tech…
Also, having conversations about things you don't understand with a machine, where you don't have to keep up social decorum and can ask the dumbest questions should help a lot with improving the decision making of non-technical personel
Re: Testing GPT 4's code-writing capabilities with some real world problems
#487Earlier quoted context omitted.
How much more efficient is programming in Python with modern tools vs assembly? I would bet more than 5x
Good question. Why don't we pit one of today's Python programmers against an IBM VS assembler programmer on an IBM 360 system circa 1960 and see how far each gets in a week?
Re: Testing GPT 4's code-writing capabilities with some real world problems
#488Guys all of this happened within a couple months. If you guys are confident about the entity as it is right now not taking over your job, what if I double the accuracy of gpt output? What if I double it again? Then again? And again? And again? You guys realize this is what's coming right? This thing literally is a baby as of now.
Is it true that GPT4 failure rate is .5 of GPT3?
You can get away with a random sample. But there's a lot of bias in that sample and it's hard to control it. Out of the infinite possibilities there definitely exists sets of inputs and outputs where both GPT3 and GPT4 are always wrong.
On the other side of the coin there are also sets where GPT4 is always right and GPT3 is always wrong and vice versa.
Given that there's no methodology to control what is "random" in these sets it's hard to come up with a good metric.
So the 0.5 thing is a bit of anecdotal number from the gut.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#489I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…
So it's perfectly possible to have it eventually iterate back and forth with the client and not only output the code but also the conversation with the client leading up to it
Re: Testing GPT 4's code-writing capabilities with some real world problems
#490Earlier quoted context omitted.
Two points: GPT4 is significantly better in this regard, and you should be concerned about the rate of progress more than it’s actual capabilities today.
> you should be concerned about the rate of progress Kind of agree? On the one hand we don't even have a roadmap toward reliable AI. On the other, if we ever plug an LLM into something that has memory, acquires experiences, does experiments, observes the outcome and adjusts its worldview in response, consciousness might fall out of that. And writing good code might not even require consciousness.
- as a function of an independent human soul
- as the fundamental substrate on which the rest of the universe is built
- as a byproduct/secondary phenomenon of physical processes
In the latter two cases I believe that the question of whether GPT is conscious is immaterial. In either case it is functioning in the same medium we all are when we talk, think, write. In the first case it is not, and the question is thornier.