Earlier quoted context omitted.
That's irrelevant though, because actual programming is very much about computers, and about writing things the majority of which have already been formalized by computer scientists, and gluing code and programs inti pipelines...
I just adore the hubris on full display here to say what one of the world's best computer science educators has to say is irrelevant.
Testing GPT 4's code-writing capabilities with some real world problems
421–430 of 677 posts
Re: Testing GPT 4's code-writing capabilities with some real world problems
#422It starts at the first character, works forward one “token” at a time, and ends at the last character. Never moving back.
It feels like it knows where it’s going at the first character, even though it doesn’t.
It’s like it starts speaking a sentence and, by the time it’s done speaking, it’s written a syntactically correct Node.js application.
The way GPT communicates in English does seem similar to how humans communicate. The way GPT writes code doesn’t seem to come close to approximating how humans do - it’s an entirely different mechanism. Humans generally can’t write code without a cursor and backspace.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#423I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…
sorry, is the debate here whether gpt can engage in a conversation with someone and respond using previous context? why would any of this present a challenge given its known abilities?
Re: Testing GPT 4's code-writing capabilities with some real world problems
#424Earlier quoted context omitted.
I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.
> Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors. That is already possible without AI and has been the case for a long time... the issue is nobody will stay to be that highly paid senior running entire projects because at that point you can just run your own shop and pocket the full profits.
I'm not disagreeing with you. Im thinking of going solo myself.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#425Earlier quoted context omitted.
This is so common in many types of business, and usually a very difficult point to articulate so thank you for that. It's something to be shown to those ringing the death-knell for programmers, artists, and the like. Those death-knell types seemingly aren't aware of what day to day operations looks like and how AI makes a great tool, but doesn't necessarily deal with the very human factors of whims, uncertainty, reac…
I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#426Nobody can really understand what's inside a trained neural network, and nobody is really looking.
No psychologist or neuro-scientist can really understand how a human brain, a mouse brain or even an ant brain or a fly brain even works, so don't expect computer scientists to have any insight about doing something relevant with just a small collection of sophisticated statistical methods.
AI is soon going to become the pseudo-scam status that bitcoin experienced.
ChatGPT is an improved search engine at best.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#427I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…
> Basically, 99% of the effort is NOT WRITING CODE. It's all about communication with people, and problem solving. If we use GPT-X in our company, it will help us with 1% of our workload First, if you did have GPT-X (say GPT-10) in your company, there wouldn't be much back-and-forth communication either. Those parts would still be handled with GPT-X talking to another GPT-X in the other company. Even the requirements…
It can’t - only programmers will know which follow up questions to ask. GPT will be able to ask those questions before non-programmers will be able to.
Half the work is nitpicking on date formats or where some id comes from or if a certain field is optional, etc.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#428As a human programmer I didn't quite understand the problem statement until I read the whole article and the tests. I believe the goal is to find a path with the fewest possible "fire" cells and the minimum cost as a tie breaker. The cost of a path is the sum of its cells' cost and it can't be greater than 5. If I understood the assignment correctly, I don't think the problem statement is equivalent to what's include…
Looking at the two examples in the paragraph after "And there’s a lot of complication to it beyond the simple cases too", I can't figure out how the movement value is defined, as I can only see 10 and 8 moves respectively, not the 14 and 10 movement value claimed in the following text (and only one water tile on each path.)
Re: Testing GPT 4's code-writing capabilities with some real world problems
#429How are people generating multiple files for larger applications? I gave it a prompt and asked it to respond with a list of file names required to build the app. Then when I prompted a file name it should print the code for that file along with a list of ungenerated file names. It got through two before it got confused. I’m stuck with having it write one function at a time.
Because thats the most it can do. Claims that it can write code are getting quieter. People made wild claims on reddit but when prompted to share their code they either went mute or the code was hilariously amateurish and limited.
Re: Testing GPT 4's code-writing capabilities with some real world problems
#430Earlier quoted context omitted.
You know the shoeshine boy indicator story [1]? That's the vibe I've been getting with ChatGPT lately, I have multiple old high school teachers with 0 computational background who hardly ever spoke about ML before posting regularly on LinkedIn about ChatGPT the last couple months. The part that gets me is they aren't just aimlessly making posts, they're getting involved with webinars targeted at educators in their re…
I can see why educators are getting involved. They are seeing it change how their students do work in real time. I grew up and using a pocket calculator was verboten. You just did not do it. You better learn and memorize all of that stuff. Spin on 10 years after me and all kids have them now. If you have the right app on your phone the thing will OCR the problem and auto solve it and show you the steps. ChatGPT and i…
These posts ooze hype bullshit, not nuanced talk about pros and cons and how it will affect education. ChatGPT should be thought of as a writing tool and perhaps an information source on very surface level topics. Trying to teach an advanced research course at a top school with ChatGPT heavily involved is a terrible idea on the other hand.
I have tested asking it about a number of specific topics in biology research/asking questions about particular papers, and it gives horrible answers the majority of the time. If someone submitted that as a paper to me I'd give it a bad grade because it is dumb, I wouldn't need to know if it were ChatGPT or not. I would be alarmed if my kid's teacher went from organically teaching how to dissect a scientific paper to suggesting that a major part of the curriculum can be replaced with talking to GPT.
I've seen articles about teachers taking the other extreme against ChatGPT too, but I haven't personally seen anything that was a realistic take on what LLMs can do. Maybe it boils down again to disagreement on "where the puck is going" but to me most of the hype is making ridiculous assumptions about what is imminent meanwhile ignoring the things worth discussing now.
Which sounds a lot like bubbles. The dot com crash didn't mean the internet was a bad idea or that it wasn't worth discussing at that time.