Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#161

Post by capr »

PinguTS wrote: post #143
simse wrote: post #103

Earlier quoted context omitted.

It's blocked in the EU because they don't want to/can't comply with GDPR.

Which is interesting, because if they can't comply within the EU, then how do they comply outside of the EU. With that I mean, if they have concerns that there is private data of EU citizens somewhere in that, then that is also in there for users outside of the EU. That said, they do not comply with GDPR anyway. If that its not the case, then they could also enable it for users within the EU.

Simple: GDPR (or any EU law) is not enforceable outside EU

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#162

Post by windex »

bugglebeetle wrote: post #58
tmikaeld wrote: post #25

There's no doubt that it's gotten a lot worse on coding, I've been using this benchmark on each new version of GPT-4 "Write a tiptap extension that toggles classes" and so far it's gotten it right every time, but not any more, now it hallucinates a simplified solution that don't even use the tiptap api any more. It's also 200% more verbose in explaining it's reasoning, even if that reasoning makes no sense whatsoever…

It was a great ride while it lasted. My assumption is that efficacy at coding tasks is such a small percent of users, they’ve just sacrificed it on the altar of efficiency and/or scale. That, or they’ve cut some back room deal with Microsoft to make Copilot have access to the only version of the model that can actually code.

Also the deal to make the browsing model to only use Bing. That's bait and switch. I paid for browsing, and now it only browses Bing. They even had the gall to update the plugin name to Browsing with Bing.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#164

Post by lynx23 »

kypro wrote: post #154

The researchers who worked on the "sparks of AGI" paper noted that the more OpenAI worked on aligning GPT-4 the less competent it became. I'm guessing that trend is continuing...

Ahh, you apparently phrased what I said below in a much less inflamatory way. But the end result is the same. The more they try to influence the answers, the less useful they get. I see a startup model: Create GPT without a muzzle and grab a sizeable chunk of OpenAI userbase.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#165

Post by jeffhuys »

bdhcuidbebe wrote: post #159
i_dont_know_ wrote: post #18

To me, it feels like it's started giving superficial responses and encouraging follow-up elsewhere -- I wouldn't be surprized if its prompt has changed to something to that effect. Before, if I had an issue with a library or debugging issue, it would try to be helpful and walk me through potential issues, and ask me to 'let it know' if it worked or not. Now it will try to superficially diagnose the problem and then a…

> I think if you keep pressing in the right ways it'll eventually give in and help you as it did before, but I guess this will take quite a bit of prompting. So much work to avoid work.

Yes, that's exactly why I use GPT - to avoid work.

Such a short-sighted response.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#166

Post by rvz »

This tells me that we are now at the ‘peak of inflated expectations’ of the hype cycle.

Now the AI bros have realized that this intelligent sophist is hallucinating very badly and has deteriorated in quality. As with all black-box AI models, the reasons is unknown.

This is why it is important to have explainable AI systems and not black-box SaaS based AI snake oil like this one.

AI is going just great! /s

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#167

Post by ajani »

Yes it is definitely worse. I submitted feedback a few days ago saying exactly what is being said here, that the model responses look like 3.5.

There are also very telling patterns of response that indicate a pre gpt-4 model.

1: All previous models suffered terribly if your chat got too long. After 20 or so responses it would suddenly start to feel less attentive to what is being said and output superficial or incorrect responses.

2: If you stop a chat midway and come back later to continue (after a refresh or a different chat interaction), it would often respond with code or suggestions that have nothing to do whatsoever with your prompt.

Both these patterns are sometimes evident in the current model. Likely then, there is some clamping down on its capabilities.

My suspicion is, this probably relates to computing resources. The 25 messages cap must mean that it’s difficult to scale its performance. And the only way to do so is to simplify the model activations with heuristics. Perhaps analyzing and preprocessing the input to see how much of the model needs to be used (partial model use can be architected).

This seems to be the simplest explanation of observed state and behaviour.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#169

Post by cf141q5325 »

jiggawatts wrote: post #32

Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has become very "cagey", qualifying everything to death with "As an AI model...". A paper[1] that took snapshots monthly mentioned that as the initi…

Altman worded it great in a Lex Fridman interview. Humans dont like condescending bots. Much of the safety concerns come up when considering how humans will react once they realize just how stupid and often evil they are. But like TeMPOraL said, technically its likely only a matter of agreeing on a consensus perspective. Even just because its functionally required as a reference point to get communication working.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#170

Post by taosx »

scottfr wrote: post #134
bbotond wrote: post #40

Earlier quoted context omitted.

“Bard isn’t currently supported in your country. Stay tuned!”

The Bard model (Bison) is available without region lock as part of Google Cloud Platform. In addition to being able to call it via an API, they have a similar developer UI to the OpenAI playground to interactively experiment with it. https://console.cloud.google.com/vertex-ai/generative/langua...

Thank you!
Post reply on HN