Live data from Hacker News

GPT-4 is phenomenal at Code

github.com

111–120 of 221 posts

Re: GPT-4 is phenomenal at Code

#111

Honestly at this point I am just looking forward to us taking this stuff for granted. Its all very nice and useful, but I think my threshold for guffawing at it is smaller than most. I know we still got some time left in the cycle, but I just want people to calm down and just start using it. Yes, cool, you did xyz all at once, but what did you make with it? It's kinda akin to the whole "using rust" phenomenon. I only…

The model IS the thing that has been made, it is the impressive accomplishment. People are awed by that. The examples are just people testing the limits of its capabilities. It's like being present at the splitting of the atom and asking ya but what have you actually powered with it.

Yes, I get it! Its amazing, impressive, all those things.

Its just been like six months of this kind of discourse if you count DALLE and stable diffusion,

I'm personally just tired of the orbit around it. I want to see the amazing future it can make is all. The future that doesnt just contain itself!

I'm not saying this is normal for me to feel, maybe I'm wrong to not stop and smell the flowers more..

Insofar as this is a tool, I want to see the houses not a continuing parade of hammers. I want to see a nuclear reactor!

Re: GPT-4 is phenomenal at Code

#112
post #84

It managed to write, in one shot, a working λ-calculus parser, using a very specific programming style I asked it in JavaScript, and then translated it all to Python, including sarcastic, rhyming GLaDOS comments. https://twitter.com/VictorTaelin/status/1635726202231988225 It also seems to be extremely competent at writing Agda types and proofs. We need some tool that highly integrates it with entire codebases, allowi…

I don't think this is a good test. Every CS graduate student has to write a lambda-calculus parser. There must be thousands of implementations on the web. It really is not strange that GPT-4 can reproduce this.

[deleted]

Re: GPT-4 is phenomenal at Code

#113

It managed to write, in one shot, a working λ-calculus parser, using a very specific programming style I asked it in JavaScript, and then translated it all to Python, including sarcastic, rhyming GLaDOS comments. https://twitter.com/VictorTaelin/status/1635726202231988225 It also seems to be extremely competent at writing Agda types and proofs. We need some tool that highly integrates it with entire codebases, allowi…

it looks like the lam parser should throw, as is it will consume input then fall through to app.

whitespace handling is also not correct.

I don't believe I've seen one correct program written by GPT yet.

edit: try parsing "λ(x x)" for example, or "λ x. x"

Re: GPT-4 is phenomenal at Code

#114
Unless these models are updated frequently it not going to be better than a human.

ChatGPT doesn't even know Go has generics yet. Now imagine is suggesting you a inferior way of doing something because it doesn't know that a bug as been fixed or a better feature/version has been implemented.

It often recommends made up libraries, or libraries that don't exists anymore.

It even recommended using a Go wiki page as an import for an experimental Go library because it doesn't know the the experimental library has been merged into Go std lib.

Another big problem I had was with algorithms. It would suggestion working code for an algorithm but would contain bugs that would completely defeat using the algorithm in the first place.

So if you don't know what your doing you would think it was giving you working code because it compiles, but it in fact it gave you bad code.

Yeah, it might be good at doing very basic and boring things, but I would never trust it for anything complex. You won't know you shot yourself in the foot until later on when you have to spend hours debugging a bug because you C+P code from ChatGPT.

Re: GPT-4 is phenomenal at Code

#116
post #65

Earlier quoted context omitted.

Let’s play this out. A company fires all the SE’s and GPT-4 takes over their place. Who will be there to ask the right questions? Middle management? Who can validate it spit out what you actually want? Project managers? Who will determine what questions to ask GPT-4? Product owners? Who will decide what type of tech (language etc…) to ask GPT-4 to write? College grads turned into corporate GPT-4 pushers? Who will be…

You don't need to fire all software engineers. You could let's say replace 75% of them with GPT.

https://en.wikipedia.org/wiki/Jevons_paradox

Re: GPT-4 is phenomenal at Code

#117
post #61

Is it? According to OpenAI's paper GPT-4 GPT-4 (no vision) GPT-3.5 Leetcode (easy) 31 / 41 31 / 41 12 / 41 Leetcode (medium) 21 / 80 21 / 80 8 / 80 Leetcode (hard) 3 / 45 3 / 45 0 / 45 https://cdn.openai.com/papers/gpt-4.pdf Table 1; page 5. So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the…

What is "Leetcode (hard)" specifically? Searching for it brought me to a website (https://leetcode.com/) but I'm assuming that is referring to some special list of problems that are meant to be harder than others, but I cannot find that.

Re: GPT-4 is phenomenal at Code

#118
post #66
post #14

Remember the comments about artists and creatives when DALL-E came out? I remember.

I remember it being vastly overhyped about how artists were now obsolete, but it really hasn't happened. People have quickly became very attuned to the "AI art style"--humans are excellent at pattern recognition, after all--and if anything, it's made people appreciate how important an artists' individual style and attention to detail is.

It absolutely has happened.

I've seen intense rage and despair for even top tier 2D artists, Japanese, Chinese, Western all alike. Like top of industry, massive fanbase, they all understand there is no sugarcoating it anymore.

AI improved from 5-year-old scribble to crushing 90% of artists in technical execution within 9 months. So they had no time to process it at all, and had to cope with shock and denial.

Now virtually all traditional 2d artist jobs are on the chopping block, its just inertia keeping them employed for another year or two. At top Chinese game studios like Tencent and Mihoyo, they are already intensely experimenting with AI workflows. And many indie game studios are pretty much already using AI for most concept art.

Artist jobs won't disappear, but they'll become hybrids of 3d modelling, SD prompting, python scripting, model training, and photoshopping. For many artists this means the workflows they spent decades honing is now gone. Also, most of them love drawing, so art jobs that don't involve drawing are despairing to them.

Those who fare best, are those who don't care much about drawing, but storytelling (Say Mangaka), they will be freed from drawing 14 hours a day, and cut their workloads in half.

Note, I think artist jobs will increase and pay better. The true industries in threat are non-virtual entertainment, such as tourism, concerts. They'll be competing with AI augmented artists that can produce awe-inspiring works in weeks not years.

Universal World's hogwarts parks are admirable. But they'll soon be competing against fully realized hogwarts (already modelled in Hogwarts legacy, without AI), that are then filled with fully responsive AIs whom you can actually talk to (Courtesy of GPT5,6,7).

AI will create massive value, but it'll also bring massive, extremely uncomfortable dislocations. Better try to adapt AI as soon as possible, and analyze how your role will transform.

Re: GPT-4 is phenomenal at Code

#119
post #61

Is it? According to OpenAI's paper GPT-4 GPT-4 (no vision) GPT-3.5 Leetcode (easy) 31 / 41 31 / 41 12 / 41 Leetcode (medium) 21 / 80 21 / 80 8 / 80 Leetcode (hard) 3 / 45 3 / 45 0 / 45 https://cdn.openai.com/papers/gpt-4.pdf Table 1; page 5. So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the…

My dude, putting aside the fact that leetcode problem solving isn't factoring in the majority of programming use cases... you do realize this is better than a lot of professional programmers right ?

It's truly amazing seeing the posts shift in real time.

Re: GPT-4 is phenomenal at Code

#120

> is phenomenal at Code Can it - attend bullshit meetings - lick my manager's boot - tell them that their ideas are good but in the masochistic, self-deprecating way they like to hear it from me in Not yet? Darn it, guess my job is safe for now.

I think it can actually do all those things reasonably well. I’d like to start getting meeting summaries that tell me my action items based on a transcript. And it can generate infinite boot-licking butt kissing if you just ask it. Generate and click send to your boss!

That would actually be an interesting product: spell-check but it edits your email or slack message to be more sycophantic. I'd use it.
Post reply on HN