Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

711–720 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#711

Post by pmarreck »

PeterStuer wrote: post #129
avocade wrote: post #110

Earlier quoted context omitted.

Interesting, please expound since very few of us had access pre-launch.

The video I posted referenced this. In summary: The person had access to early releases through his work at Microsoft Research where they were integrating GPT-4 into Bing. He used "Draw a unicorn in TikZ" (TikZ is probably the most complex and powerful tool to create graphic elements in LaTeX) as a prompt and noticed how the model's responses changed with each release they got from OpenAI. While at first the drawings…

That’s awful. Talk about cutting off your nose to spite your face.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#712

Post by pmarreck »

renewiltord wrote: post #556
bombcar wrote: post #312

Earlier quoted context omitted.

That indicates the “nerfing” is not what I would think (a final pass to remove badthink) but somehow deep in everything, because the question asked should be orthogonal.

There's a section in the GPT-4 release docs where they talk about how the safety stuff changes the accuracy for the worse.

this, more than anything, makes me want to run my own open-source model without these nearsighted restrictions

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#713

Post by telotortium »

strangemonad wrote: post #596
mike_hearn wrote: post #482

Earlier quoted context omitted.

I think the way people have been using the word 'aligned' is usually in the context of moral alignment and not just RLHF for instruction following.

philosophical nit picking here, I would say value-aligned rather than moral-aligned.

If I may be so naive, what's supposed to be the difference? Is it just that morality has the connotation of an objective, or at least agent-invariant system, whereas values are implied to be explicitly chosen?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#714

Post by Jorropo »

I went through my history, for one code example I copy pasted the old prompt into new GPT4.

It was about writing a CGO wrapper given an hpp header for a lib I have. Back then it used to give me almost correct code, it understood is had to write a C++ C ffi using extern "C" first because Golang's CGO FFI only support C not C++. And then it generated me correct CGO wrapper with a Go looking type that made sense. The only wrong thing is that it didn't understood it had to call the initialization function of my C++ lib at init time, instead it called it in the New factory function (which would segfault when you build more than one object) trivial fix for the human in the loop move it to func init(). TL;DR back then almost perfect

Now with the exact same prompt it doesn't even generate me code, it just list me a list of tasks I could do to achieve this task and give me vague statements. If I change the prompt insisting for code, it instead give me a very dumb 1 to 1 mapping of the C++ into Go trying to create constructor and destructor functions, oh and it's tries to use CGO ffi to call C++ (even tho again, only C is supported by golang).

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#715

Post by Spivak »

motogpjimbo wrote: post #708
Spivak wrote: post #622

Earlier quoted context omitted.

So let me see if I understand this thread. - Haha look at all those Gen Z snowflakes getting offended at words. - Okay sure, but the ability to not get offended is related to whether or not you're a target of their bullshit or not; 4chan trolls get extremely offended and unjerk the moment you turn the lens toward them. By 4chan's own standards it's actually pretty reasonable to be offended by their antics. - But have…

> So let me see if I understand this thread. You realise you've been arguing with multiple people expressing multiple opinions, right? You appear to be prone to binary thinking, so it might not be clear to you that your opponents don't form a single monolith. > tl;dr if you're not offended by 4chan they're not actually saying anything offensive about you even though it might appear so superficially; 4chan just has a…

Yes, and in threaded discussions if you jump in in the middle like this it's assumed you're continuing the downward trajectory of the discussion. Otherwise you would have replied to someone higher up the thread. I'm in no way assuming that you hold any opinion in particular just that the discussion has circled back.

I think you assume a tone that I absolutely do not have. I couldn't care less about 4chan drama and I don't go there anymore for the reasons you listed. I'm talking about my own experience and trying to make my case for the, apparently controversial, idea that words can and do affect people and that total emotional detachment is the exception rather than the rule. And of course that's the case, 4chan's whole thing is using offensive language to select specifically for the subset of people who can tolerate it.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#716

Post by clark-kent »

NickC25 wrote: post #509
StackOverlord wrote: post #479

Earlier quoted context omitted.

> Yes, one of the board members of OpenAI, Will Hurd, is a former government agent. He worked for the Central Intelligence Agency (CIA) for nine years, from 2000 to 2009. His tour of duty included being an operations officer in Afghanistan, Pakistan, and India. After his service with the CIA, he served as the U.S. representative for Texas's 23rd congressional district from 2015 to 2021. Following his political career…

Yikes One is never former CIA, once you're in, you're in, even if you leave. Although he is a CompSci grad, he's also a far-right Republican. A spook who leans far right sitting atop OpenAI is worse than Orwell's worst nightmares coming to fruition.

Will Hurd is a liberal Republican. He supports Dreamers. Very early critic of Donald Trump.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#717

Post by dbmnt »

sagarpatil wrote: post #642
jdiez17 wrote: post #78

Earlier quoted context omitted.

Copilot X (the new version, with a chat interface etc) is significantly worse than GPT-4 (at least before this update). It felt like gpt3.5-turbo to me.

Have they started rolling it out? When did you get access?

I've had access since 2023-05-13. You have to use the Insiders build of VS Code, and a nightly version of the Copilot extension.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#718

Post by Madmallard »

ianbicking wrote: post #643
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

People here need to learn to chill out and use the API. The GPT API is not some locked down cage. Every so often it'll come back with complaints instead of doing what was asked, but that's really uncommon. Control over the system prompt and putting a bit of extra information around the requests in the user message can get you _so_ far. It feels like people are getting ready to build castles in their mind when they ju…

[flagged]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#719

Post by orand »

bdw5204 wrote: post #326
bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

Geoffrey Hinton says [1] part of the issue with current AI is that it's trained from inconsistent data and inconsistent beliefs. He thinks to break through this barrier they're going to have to be trained so they say, if I have this ideology then this is true, and if I have that ideology then that is true, then once they're trained like that, then within an ideology they'll be able to get logical consistency.

[1] at the 31:30 mark: https://www.technologyreview.com/2023/05/03/1072589/video-ge...

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#720

Post by heliophobicdude »

alecco wrote: post #241
kypro wrote: post #154

The researchers who worked on the "sparks of AGI" paper noted that the more OpenAI worked on aligning GPT-4 the less competent it became. I'm guessing that trend is continuing...

I don't think it's just the alignment work. I suspect OpenAI+Microsoft are over-doing the Reinforcement Learning from Human Feedback with LoRA. Most of people's prompts are stupid stuff. So it becomes stupider. LoRA is one of Microsoft's most dear discoveries in the field, so they are likely tempted to over-use it. Perhaps OpenAI should get back to a good older snapshot and be more careful about what they feed into t…

Supposedly the API hasn't changed.

https://twitter.com/officiallogank/status/166393494793189785...

Post reply on HN