Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

181–190 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#181

Post by tinco »

scg wrote: post #19

As a human programmer I didn't quite understand the problem statement until I read the whole article and the tests. I believe the goal is to find a path with the fewest possible "fire" cells and the minimum cost as a tie breaker. The cost of a path is the sum of its cells' cost and it can't be greater than 5. If I understood the assignment correctly, I don't think the problem statement is equivalent to what's include…

By the time you've formulated the problem as: "Give me the shortest route with a cost of 5 or lower that doesn't go through fire, and if that doesn't exist, the shortest route with a cost or 5 or lower that goes through fire." Then you've basically formulated the algorithm as well.

That's also precisely where one of the programmer's greatest challenges lies, to carefully translate and delineate the problem. I agree it's a bit steep to ask the GPT to come up with a precise solution to an imprecise question, but it's also fair to say that that's basically what the job of a programmer entails, and if you can't do that you're not really able to program.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#182

Post by SuoDuanDao »

nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

I think we're seeing the early phases a prediction I made in my first book come true: That computers will be more suitable than humans for most of the functions humans currently use their left brain half for.

Best case, that will have a whole lot more humans using their right brain halves on things like defining the problem. I like the thought of that, it's more pleasant work. But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks. I think you're pointing out there's more to programming than left-brain activities, and I think you're right that silicon will never eclipse carbon at those challenges, but I think a lot of people are also feeling threatened by the fact that chatgpt is getting better and better at the thing they used to be better than all humans at.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#183

Post by danwee »

I want to see GPT-4 dealing with this situation:

- they: we need a new basic POST endpoint

- us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code?

- they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db

- us: ok, cool. Let me get in contact with them

- ... one week later...

- company XXQ: we got this contract here:

- us: thanks! We'll work on this

- ... 2 days later...

- us: umm, there's something not specified in . What about this part here that says that...

- ... 2 days later...

- company XXQ: ah sure, sorry we missed that part. It's like this...

- ...and so on...

Basically, 99% of the effort is NOT WRITING CODE. It's all about communication with people, and problem solving. If we use GPT-X in our company, it will help us with 1% of our workload. So, I couldn't care less about it.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#184

Post by stormfather »

whateveracct wrote: post #86
sabellito wrote: post #66

Earlier quoted context omitted.

What I've seen in this thread so far is some people saying "this is really useful in these particular cases" and some others, like yourself, saying "I'm too smart for this, maybe it's useful for inexperienced/incompetent people".

I don't even think it's useful for the incompetent. I do think the incompetent think it's useful tho.

That's like saying a calculator isn't useful just because you're good at mental math. The ability to adapt and extract value from new tools is a mark of competence.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#185

Post by brookst »

malborodog wrote: post #125
noitpmeder wrote: post #76

Earlier quoted context omitted.

Personally I think it's 99% hype. The current iteration and architecture of these systems means they will never be at the level where they can actually replace a programmer. The best they will ever get is barfing up snippets for a programmer (who still needs the industry-specific knowledge to VERIFY the snippets). Additionally, "the rate of increase in capabilities" is very much a false flag. Past performance (especi…

But obviously increasing avg and sr programmer productivity by many x implies many fewer jobs for programmers. That’s the point we’re interested in.

You seem to be saying that there is fixed demand for programmers and oversupply means less hiring.

But the history of technology says that demand is recursive; the more tech produced, the more demand there is for producers.

There may be a time when we hit the old “the whole world only needs 5 computers”[0] limit, but I don’t think we’re anywhere close. AI is providing leverage to create net more programming; it is not replacing programmers in a zero sum game.

0. https://www.theguardian.com/technology/2008/feb/21/computing...

Re: Testing GPT 4's code-writing capabilities with some real world problems

#186

Post by pbhjpbhj »

ashwagary wrote: post #155
pixl97 wrote: post #104

Earlier quoted context omitted.

Its like feeding a baby polar bear. When it grows up it is going to eat you.

Extending this analogy into a question: Could the polar bear community eat enough expert food providers that their quality of food and eventually overall health declines?

Will GPTs eat themselves, or more correctly, will they dilute their training corpuses until there are no gains to make? Seems possible.

Like how we've relied on fossil fuels that are a limited resources, to get further we have to go beyond such fuels. It seems like we're in the process of locking in our highest attainable level of text-or-image based advancement.

We mine the expertise in industries (programming, music production, graphic design, ...) and simultaneously cause the replacement of many future experts. Thus leading to a lack of expert output that we can mine?

Now, currently, the problem seems to be "we're putting people out of work" -- but that's progress. The real problem is that we are unevenly distributing the benefits.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#187

Post by brookst »

dr_dshiv wrote: post #147
Ygg2 wrote: post #94

Earlier quoted context omitted.

Who's not disagreeing? Singularity feels like "man facing existential dread confuses saturation curve for exponential".

When I say singularity, I mean “changes the nature of human life irreversibly and to a greater degree than any prior technology through a rapid progression of self-reinforcing growth.” That rapid progression can be modeled with an exponential growth curve, like Moore’s law, even though nothing material can have a sustained exponential growth. Regardless of how the curve is modeled, the steepness is such that it serve…

I think you just declared radio, television, the internet, the macaroon, and mobile phones to be singularities.

People generally use the term to mean the point at which progress is automatic, unbounded, and instant. E.g. AIs that can produce smarter AIs themselves, in short order.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#188

Post by anileated »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

Why aren’t you thinking rather that instead of talking to you, “they” would already be talking to the LLM (likely trained on your code, among other data)—while you get 0 total billable workload in the first place?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#189

Post by guax »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

I have the same feeling, people are very concentrated on the ability of this AI generators to create working code from super specific and well formed prompts. When in reality, figuring out what the prompt should be accounts for 80% of the job.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#190

Post by brookst »

Planktonne wrote: post #153
whateveracct wrote: post #53

Earlier quoted context omitted.

the big thing to me is programming isn't hard I've worked on adtech, crypto, fintech, gamedev, startup founder, BigCo. Not once was programming something that was a time sink. Makes me feel like GPT is marketing to the incompetent or something.

> Makes me feel like GPT is marketing to the incompetent or something. Absolutely. The common constant I can see in people who are really blown away by GPT's performance at [task] is that they are bad at [task]. Programmers who describe their job as copying from StackOverflow think it's great at coding. People who don't read fiction think it's great at writing fiction, and so on.

Many people are blown away by GPT for the leverage it provides, not because it makes the impossible possible.

Programming is generally not hard. Producing 5x - 10x as much programming output at the same quality is quite hard. And keeping the most interesting 10% for yourself while offloading the 90% that is basically just typing? That’s what people are excited about.

Post reply on HN