Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

671–677 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#671

Post by Palpatineli »

danwee wrote: post #183

I want to see GPT-4 dealing with this situation: - they: we need a new basic POST endpoint - us: cool, what does the api contract look like? URL? Query params? Payload? Response? Status code? - they: Not sure. Third-party company XXQ will let you know the details. They will be the ones calling this new endpoint. But in essence it should be very simple: just grab whatever they pass and save it in our db - us: ok, cool…

Not sure about that. Theoretically, you can talk to GPT-X pretending to be your manager, and the your manager can talk to GPT-X pretending to be you. Then the two instances exchange information in a format much more efficient than human conversation. Sounds like an efficiency boost and if expanded this system avoid a bunch of office politics and helps with everybody's mental health.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#672

Post by pms »

theK wrote: post #132
anonzzzies wrote: post #110

Earlier quoted context omitted.

> their monkeypatched You mean, like almost every outsourcing company pops over? So the type of code that infests companies who hired some sweatshop to do ‘some simple crud’? What’s the difference? Can you see the difference? Besides the gpt code will be far better commented as comments come for almost free with gpt while humans hate writing them.

I think you can trust the code of both those worlds exactly the same, not at all. I’ve seen GPT 3 and 4 hallucinate the most amazing commentary about their own output. Maybe we will get dependable, out of process, guidance at some point about how factual the model thinks it is on an output per output basis but until that point you should trust every LOC and comment exactly the same as code gifted to you by an adversa…

Exactly... and then the question becomes: why do I interact with an adversary?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#673

Post by pms »

kevinwang wrote: post #32
nwienert wrote: post #8

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer.…

Eh, it's still impressive that these systems can write such good code despite pretty much just predicting the next word. I guess it's a matter of perspective. You can either be astounded by how much it can do relative to your expectations from 2018 or you can be skeptical relative to the volume of excitement. I think some excitement is also people extrapolating to the future: if predicting the next word gets you this…

> what happens when you actually try to make it good?

1. Do you think that companies paying millions of dollars to ML researchers aren't already trying to make it good?

2. I think it will take a real revolution in AI/ML to do what people here are extrapolating into the future. That revolution will eventually come, but I doubt it'll be as quick as people think. Just think about the excitement people had about Siri 10+ years ago, or about Full Self Driving 5+ years ago. In my opinion, in 5-10 years from now GPT will be in the same place where Siri and Full Self Driving currently are. Eventually we will make the leap we're dreaming of, but that leap isn't happening yet.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#674

Post by skeaker »

gpderetta wrote: post #209
kykeonaut wrote: post #205

Earlier quoted context omitted.

To paraphrase Harold Abelson: Computer Science is not really very much about computers. And it’s not about computers in the same sense that physics isn’t really about particle accelerators, and biology is not really about microscopes and petri dishes. It is about formalizing intuitions about process: how to do things [0]. [0]: https://www.driverlesscrocodile.com/technology/the-wizard-1-... .

“Computer Science is no more about computers than astronomy is about telescopes” - Edsger Dijkstra

This rant from a little while ago soured me on this quote: https://news.ycombinator.com/item?id=34940148

Re: Testing GPT 4's code-writing capabilities with some real world problems

#675

Post by iakgk »

WinstonSmith84 wrote: post #472
lionkor wrote: post #240

Earlier quoted context omitted.

For me, its mostly that I have used GPT-3.5 a little for programming C++, and I wasnt impressed. For one, it made horrible, glaring mistakes (like defining extern functions which dont exist, using functions which are specific to a platform im not using, etc.), stuff beginners would do. It also decided to sneak in little issues, such as off-by-one errors (calling write() with a buffer and a size that is off by one in…

GPT4 is very different from 3.5. I've asked it today to write some unit tests given the code of the class (~200 lines) and the methods I wanted to cover and it did that just perfectly. It put asserts where it made sense (without me asking to do it), and the unit test code was better written than some code I've seen written by (lazy) humans. It's not perfect sure and it's easy to get a bad response but give OpenAI a f…

write addition python code

Re: Testing GPT 4's code-writing capabilities with some real world problems

#676

Post by gpderetta »

skeaker wrote: post #674
gpderetta wrote: post #209

Earlier quoted context omitted.

“Computer Science is no more about computers than astronomy is about telescopes” - Edsger Dijkstra

This rant from a little while ago soured me on this quote: https://news.ycombinator.com/item?id=34940148

FWIW, that's actually a good rant.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#677

Post by lamp987 »

yucky wrote: post #654
lamp987 wrote: post #646

Earlier quoted context omitted.

Then youre wrong. There is no market incentive for layoffs due to GPT-like technology, as I have demonstrated above. Similar breakthroughs "replacing jobs" have happened before in the field of software engineering, this is nothing new or unprecedented. Its merely another tool that will become in widespread use to increase production. What i think will lead to mass layoffs is the current recession rather.

> this is nothing new or unprecedented This is both new and unprecedented.

Huh? I literally just gave you an example of tech that automated away what 90% of programmers were doing.
Post reply on HN