Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT Isn't as Good at Coding as We Thought

pcmag.com

Post reply on HN
Print view View original

21–30 of 48 posts

Re: ChatGPT Isn't as Good at Coding as We Thought

#21

Post by tbiehn »

soultrees wrote: post #12

Lately it’s even worse. And any AI agent that relies on open AI has gone down in quality too (phind). I find it’s more of a struggle to have the AI write code in the first pass, even when I explicitly say something like “I want you to write a function that does this {action}” and on average it takes about 3 messages back and forth before it actually writes it. Whereas before it would start writing it immediately, eve…

A tip for your context window problem: prefer ‘editing’ a previous message to add whatever clarification is required - rather than having a ‘chat.’ Definitely helps keep it on the rails.

Re: ChatGPT Isn't as Good at Coding as We Thought

#22

Post by tasogare »

HWR_14 wrote: post #17
jiggawatts wrote: post #4

Yet another garbage study where “researchers” wasted months of their lives evaluating GPT 3.5 instead of paying $20 to evaluate GPT 4. It’s like drinking the complementary table water at a restaurant and then leaving a one-star review saying “tasteless and flat”. I seriously can’t believe this is maybe the tenth such “paper” I’ve seen making headlines. It brings shame on not only the institution that produced it, but…

It takes time to create and publish a study.

[dead]

Re: ChatGPT Isn't as Good at Coding as We Thought

#23

Post by bheadmaster »

SanderNL wrote: post #16
omarfarooq wrote: post #9

Earlier quoted context omitted.

Would you be so kind as to demonstrate how generated language and code does not transport meaning between different people.

I'd say the symbols themselves are meaningless without their ground truths, whatever that is. We can call this "experience" or "thinking". The results of these opaque processes are somehow amenable to be encoded into symbols which are themselves deprived of any significance. Which I guess makes them usable as such in the first place. Extreme example: if a random algorithm picked out a few words from a dictionary and…

> I'd say the symbols themselves are meaningless without their ground truths, whatever that is. We can call this "experience" or "thinking".

So you're basically dismissing the whole field of genetic algorithms?

Re: ChatGPT Isn't as Good at Coding as We Thought

#24

Post by nonameiguess »

jiggawatts wrote: post #4

Yet another garbage study where “researchers” wasted months of their lives evaluating GPT 3.5 instead of paying $20 to evaluate GPT 4. It’s like drinking the complementary table water at a restaurant and then leaving a one-star review saying “tasteless and flat”. I seriously can’t believe this is maybe the tenth such “paper” I’ve seen making headlines. It brings shame on not only the institution that produced it, but…

Jesus Christ, dude. To know they used 3.5, you must have read the methods section, in which case you can also see they conducted the data collection in March 2023, the same month GPT-4 was released. Bad timing, I guess, but your rant reduces to telling researchers to just scrap whatever work they did and not bother if an update happens to be released to whatever they were studying before they publish.

Re: ChatGPT Isn't as Good at Coding as We Thought

#25

Post by SanderNL »

bheadmaster wrote: post #23
SanderNL wrote: post #16

Earlier quoted context omitted.

I'd say the symbols themselves are meaningless without their ground truths, whatever that is. We can call this "experience" or "thinking". The results of these opaque processes are somehow amenable to be encoded into symbols which are themselves deprived of any significance. Which I guess makes them usable as such in the first place. Extreme example: if a random algorithm picked out a few words from a dictionary and…

> I'd say the symbols themselves are meaningless without their ground truths, whatever that is. We can call this "experience" or "thinking". So you're basically dismissing the whole field of genetic algorithms?

If said genetic algorithms were used to directly spew forth symbols that represent or point towards human experience (“language”) as if the underlying processes that ultimately form the foundation of said symbols could be approximated by slowly changing random walks, then, yes, I’d question the usefulness of that approach.

If not, what do you mean?

Re: ChatGPT Isn't as Good at Coding as We Thought

#26

Post by jimmychoozyx »

ChrisRR wrote: post #7
namaria wrote: post #6

I don't understand why generating language, or code, would be a good thing. Language, and code, is a tool for communication. If you treat it as fodder or merchandise, you loose the main use: transport meaning between different people. I fail to understand why we would want automatically generated language at all, except for the fact that you can sometimes make money off of it. Considering we're where we are as a civi…

You can't understand why software that can do in a fraction of a section what a human would do in hours would be valuable?

ChrisRR--

I would counter this with:

- What value is it to your company to know that the developers who churn into and out of your company... while they're there, they actually know a section of your codebase quite well?

versus the alternative: the degree to which your devs know where your code lives or what it does is low, since they do not engage with it enough.

Sure, one could say: "AI will navigate our codebase for them".

But I think it becomes a bit of a slippery-slope, regarding the question of: "When/to what degree can really we take the developer out of the picture?"

Imagine being a developer, and no one on your team really knows how your product works.

Is AI building your product at that point then? Is that product actually going to exist & be funcitonal?

Re: ChatGPT Isn't as Good at Coding as We Thought

#27

Post by ChatGTP »

jiggawatts wrote: post #19
ChatGTP wrote: post #15

Earlier quoted context omitted.

I'd like to see the stats on how many people use 3.5 over 4 though, I bet you that number is much higher than you think. The price of ChatGPT is being what I'm personally willing to pay for it so if I do use it, it's 3.5 for me. Maybe the logic is to study the most popular product? Would make sense to me.

Many more people drink tap water than eat dinner at a MICHELIN Starred restaurant, but if you're a food critic writing about the latest fancy restaurant, you had better pay for some food and write about that. This is especially painful considering the ludicrous pace of advancement in AI. You really have to aim for where the puck will be, not where the puck is. Within a year GPT4 will be out-of-date. Fundamentally, th…

This analogy should be in a gallery, good day to you sir.

Re: ChatGPT Isn't as Good at Coding as We Thought

#28

Post by bheadmaster »

SanderNL wrote: post #25
bheadmaster wrote: post #23

Earlier quoted context omitted.

> I'd say the symbols themselves are meaningless without their ground truths, whatever that is. We can call this "experience" or "thinking". So you're basically dismissing the whole field of genetic algorithms?

If said genetic algorithms were used to directly spew forth symbols that represent or point towards human experience (“language”) as if the underlying processes that ultimately form the foundation of said symbols could be approximated by slowly changing random walks, then, yes, I’d question the usefulness of that approach. If not, what do you mean?

I mean, just because an array of symbols is generated mechanically doesn't inherently make it meaningless, because meaning arises from the execution of those symbols (in case of computer programs; or interpretation, in case of human language).

If you read a sentence (whose source is unknown to you) and it has meaning to you and affects you, what difference does it make if the sentence was written by a human or a machine?

Re: ChatGPT Isn't as Good at Coding as We Thought

#29

Post by nwalizer »

selfhoster11 wrote: post #11
namaria wrote: post #6

I don't understand why generating language, or code, would be a good thing. Language, and code, is a tool for communication. If you treat it as fodder or merchandise, you loose the main use: transport meaning between different people. I fail to understand why we would want automatically generated language at all, except for the fact that you can sometimes make money off of it. Considering we're where we are as a civi…

The purpose of code is first and foremost to make a computer do things. Legibility is, strictly speaking, secondary.

I see you've never had to maintain code.
Post reply on HN