Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

251–260 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#251

You can still run the original gpt-4-0314 model (March 14th) on the API playground: https://platform.openai.com/playground?mode=chat&model=gpt-4... Costs $0.12 per thousand tokens (~words), and I find even fairly heavy use rarely exceeds a dollar a day.

So are you sure that isn't also nerfed?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#252

Is it consistently worse or just sometimes/often worse than before? Any extreme power users or GPT-whisperers here? If it’s only noticeably worse X% of the time my bet would be experimentation. One of my least favorite patterns that tech companies do is use “Experimentation” overzealously or prematurely. Mainly, my problem is they’re not transparent about it, and it creates an inconsistent product experience that jus…

It is not worse for me. I do notice the novelty has worn off. Asking chatGPT4 about why people would think this on here I think it nails it with the novelty effect lol: "Indeed, the performance of an AI model like ChatGPT doesn't deteriorate over time. However, human perception of its performance can change due to a variety of psychological factors: Expectation Bias: As users become more familiar with AI capabilities…

> Indeed, the performance of an AI model like ChatGPT doesn't deteriorate over time.

Of course, the performance of an unchanged model does not. But finetuning the model over time can of course either improve or degrade performance.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#253
post #84

Earlier quoted context omitted.

I have some first hand thoughts. I think overall the quality is significantly poorer on GPT4 with plugins and bing browsing enabled. If you disable those, I am able to get the same quality as before. The outputs are dramatically different. Would love to hear what everyone else sees when they try the same.

would be alarming if you had second hand thoughts...

I get mine third-hand or from the bargain-bin. Never over-pay for almost as good as new; like a car, used thoughts are just better price to value.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#254
post #207

Earlier quoted context omitted.

It's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.

That's just not true. Treat adults as adults, please. You're not everybody's babysitter and neither are the sf bros.

Thanks for writing this. Mentally healthy adults need to point this out more often, so that this patronizing-attitude-from-the-USA eventually finds an end to its manipulative tactics.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#256
post #167

Yes it is definitely worse. I submitted feedback a few days ago saying exactly what is being said here, that the model responses look like 3.5. There are also very telling patterns of response that indicate a pre gpt-4 model. 1: All previous models suffered terribly if your chat got too long. After 20 or so responses it would suddenly start to feel less attentive to what is being said and output superficial or incorr…

The context length is a different issue. But that was always a limitation with a GPT-4. Due to physics.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#257
post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

They're up against a pretty difficult barrier - if we had a perfect all-knowing oracle it might easily have opinions that are racist. Statistics alone suggest there will be racist truths. We're dealing with groups of people who are observably different from each other in correlated ways.

GPT would need to reach a convincing balance of lying and honesty if it is supposed to navigate that challenge. It'd have to be deeply embedded in a particular culture to even know what 'racism' means; everyone has a different opinion.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#258
post #131
post #111

Earlier quoted context omitted.

What is an example of a request that is causing these issues?

Explaining it to you will not fix my problems with OpenAI's instance. Let it suffice that I am a user of ChatGPT who really dislikes the lecturing style of woke-fine-tuning. If a human would lecture me in that style, I'd dumb them and avoid contact.

Why not share you gripes with the system so they get more exposure? I'm curious to know the pitfalls.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#259
I haven't used 4, just 3.5 on free tier. The only change I've noticed is that it is significantly slower. Which leads me to suspect that degraded quality could just have to do with tuning their product to use less compute.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#260
post #85

I tried to replicate a few of my chats (the displayed date is incorrect, it seems to be the publish date instead of the original chat date): svg editor: early april: https://chat.openai.com/share/c235b48e-5a0e-4a89-af1c-0a3e7c... now: https://chat.openai.com/share/e4362a56-4bc7-45dc-8d1b-5e3842... originally it correctly inferred that I wanted a framework for svg editors, the latest version assumes I want a js framew…

this is GPT 3.5, the icon is green

no it's definitely GPT4.

I'm not sure why the share page says "Model: Default" but it's "Model: GPT-4" in my webui. Seems like a bug in the share feature

Post reply on HN