Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

201–210 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#201

Post by munchbunny »

nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

These statements can definitely be simultaneously true:

* ChatGPT is revolutionary - honestly, it's genuinely impressive how much of a leap ChatGPT is compared to the attempts that came before it.

* Programmers write a lot of simple code that has been written before - there are genuinely tons of cases of "write a web endpoint that takes an ID, looks it up in a database table, pulls an object through an ORM, and returns a JSON serialization of it." Most programmers? Doubt it. But tons of programmers write CRUD stuff and tons of IT admins do light scripting, and a lot of it is repeated code.

Could ChatGPT do my job? Not even close. But it's still really impressive to me.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#202

Post by davemp »

SuoDuanDao wrote: post #182
nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

I think we're seeing the early phases a prediction I made in my first book come true: That computers will be more suitable than humans for most of the functions humans currently use their left brain half for. Best case, that will have a whole lot more humans using their right brain halves on things like defining the problem. I like the thought of that, it's more pleasant work. But a lot of intelligent people define t…

> But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks.

I think 'left brain' work also has a lot more predictability (knowing you can churn out ~X widgets/hr) so having only uncertain 'right brain' work can be uncomfortable for people to build their livelihoods upon.

That being said. 'right brain' work is certainly more fulfilling for me.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#203

Post by danielbln »

noitpmeder wrote: post #65
whateveracct wrote: post #55

Earlier quoted context omitted.

Programming takes little effort. It's at most 20% of my effort. Feels like GPT is programming with extra steps. Maybe it's for people who can never think programming is easy. Clearly there's a lot of such types. Explains a lot.

/r/iamverysmart I'm reminded of the old "Handyman's Invoice" trope. Actually implementing a solution is not the hard part. What _is_ hard is determining what the solution is in the first place. https://www.snopes.com/fact-check/know-where-man/ Once you have a rough idea of the solution, sure maybe GPT-4 can barf snippets to get you there. But it's lightyears away from translating business problems into actionable sol…

> But it's lightyears away from translating business problems into actionable solutions.

Is it though? Have you tried feeding it business problems and working through to possible solutions paths? Have you roped in additional external information (via agents, tools, vector search, etc.) during that process? I don't see why the model wouldn't be able to translate a lot of business problems into solutions paths.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#204

Post by eggsmediumrare »

naillo wrote: post #170

The biggest thing here is that it's semi capable and improving. I feel safe about my job right now but it is worrying to invest time to compete with a machine that will continue to get better over the years where previously I felt safe that the effort of my labour would bear fruit for decades to come. Now I'm not so sure.

This is what most people making "I'm not worried" arguments don't understand. Right now, it makes you way more productive. Even if its capabilities stalled right there, it will over time reduce the value of your labour. But it won't stall.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#205

Post by kykeonaut »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

To paraphrase Harold Abelson:

Computer Science is not really very much about computers. And it’s not about computers in the same sense that physics isn’t really about particle accelerators, and biology is not really about microscopes and petri dishes. It is about formalizing intuitions about process: how to do things [0].

[0]: https://www.driverlesscrocodile.com/technology/the-wizard-1-....

Re: Testing GPT 4's code-writing capabilities with some real world problems

#207

Post by danielbln »

theshrike79 wrote: post #136
pixl97 wrote: post #126

Earlier quoted context omitted.

>Then I asked it to write unit tests for the code, and it confidently started writing correct-looking code that would take the same input and expect the correct output, but just stopped in the middle. One of two things. First ask it to continue. Sometimes it just stops half way thru code foe whatever reason. The other possibility is you filled up the token context window. Not much you can do but wait for the 32k mode…

I asked it to continue twice after the first failure. Every time it failed in about the same point. Might've filled up some mysterious limit in the model. I didn't really need the unit tests anyway, but I wanted to try if it could do it :)

You can also ask it to "continue at this line XZY" and it will not give you the whole thing again, only the new continuuation.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#208

Post by z3c0 »

eggsmediumrare wrote: post #194
z3c0 wrote: post #101

Earlier quoted context omitted.

Soon, a lot of people will realize that the puck doesn't travel in a straight line. Eventually, it will veer to the right in a logarithmic fashion. When it does is up for debate, but - in my experience - it always happens very short of where the hype claims it was heading.

The puck can also hit you in the face and break your jaw.

Certainly a likelihood for everyone too lazy to write their own code.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#209

Post by gpderetta »

kykeonaut wrote: post #205
danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

To paraphrase Harold Abelson: Computer Science is not really very much about computers. And it’s not about computers in the same sense that physics isn’t really about particle accelerators, and biology is not really about microscopes and petri dishes. It is about formalizing intuitions about process: how to do things [0]. [0]: https://www.driverlesscrocodile.com/technology/the-wizard-1-... .

“Computer Science is no more about computers than astronomy is about telescopes”

- Edsger Dijkstra

Re: Testing GPT 4's code-writing capabilities with some real world problems

#210

Post by flippinburgers »

brookst wrote: post #185
malborodog wrote: post #125

Earlier quoted context omitted.

But obviously increasing avg and sr programmer productivity by many x implies many fewer jobs for programmers. That’s the point we’re interested in.

You seem to be saying that there is fixed demand for programmers and oversupply means less hiring. But the history of technology says that demand is recursive; the more tech produced, the more demand there is for producers. There may be a time when we hit the old “the whole world only needs 5 computers”[0] limit, but I don’t think we’re anywhere close. AI is providing leverage to create net more programming; it is no…

Sounds a bit too much like a perpetual motion machine to me.
Post reply on HN