Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

531–540 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#531

Post by windex »

onlyrealcuzzo wrote: post #212
windex wrote: post #176

Earlier quoted context omitted.

They can only do this in the USA. The battle bottleneck is going to be GPUs and countries that have access to it will leap ahead of the US if arbitrary regulations are put in place to neuter LLMs in the US.

LLM progress is likely going to have a deteriorating curve like nearly all ML / AI tech. It's possible things aren't going to get substantially better from GPT-4 for a while. The idea that new players will surely zoom past seems bold, especially when established incumbents with massive budgets and proven track records in the space are struggling to keep up...

I think the next wave will focus on the emergent organization of information and zero shot solutions that some of these models seem to be showing in cases.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#532

Post by boringuser2 »

caddemon wrote: post #245
dr_dshiv wrote: post #119

Earlier quoted context omitted.

If this is true, one should be able to compare with benchmarks or evals to demonstrate this. Anyone know more about this?

Yeah I think it's plausible it's gotten worse but it would also be classic human psychology to perceive degradation because you start noticing flaws after the honeymoon effect wore off. Unfortunately this will be hard to benchmark unless someone was already collecting a lot of data on ChatGPT responses for other purposes. Perhaps if this is happening the degradation will get worse though, so someone noticing it now c…

Yes, that's an obvious complication, but it isn't the fault of the humans given that the model can easily be tuned without your knowledge to subjectively perform worse, and there's an obvious incentive for it (compute cost).

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#533

Post by geysersam »

the_gipsy wrote: post #303
roenxi wrote: post #257

Earlier quoted context omitted.

They're up against a pretty difficult barrier - if we had a perfect all-knowing oracle it might easily have opinions that are racist. Statistics alone suggest there will be racist truths. We're dealing with groups of people who are observably different from each other in correlated ways. GPT would need to reach a convincing balance of lying and honesty if it is supposed to navigate that challenge. It'd have to be dee…

But the statistics here are "number of times it has been fed and positively trained with racist (or biased) texts" - not crunching any real numbers.

Thank you. Ironically the comment you replied to just reinforced the bias future models will have... It's a self playing piano

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#534

Post by StackOverlord »

fnordpiglet wrote: post #331
bbotond wrote: post #14

Yes. Before the update, when its avatar was still black, it solved pretty complex coding problems effortlessly and gave very nuanced, thoughtful answers to non-programming questions. Now it struggles with just changing two lines in a 10-line block of CSS and printing this modified 10-line block again. Some lines are missing, others are completely different for no reason. I'm sure scaling the model is hard, but they l…

This is for your protection from an extinction level event. Without nerfing the current model they couldn’t charge enterprise level fee structures for access to the superior models, thus ensuring the children are safe from scary AI. Tell your congress person we need to grant Microsoft and Google exclusive monopolies on AI research to protect us from open source and competitor AI models that might erode their margins…

[flagged]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#535

Post by boringuser2 »

bdhcuidbebe wrote: post #159
i_dont_know_ wrote: post #18

To me, it feels like it's started giving superficial responses and encouraging follow-up elsewhere -- I wouldn't be surprized if its prompt has changed to something to that effect. Before, if I had an issue with a library or debugging issue, it would try to be helpful and walk me through potential issues, and ask me to 'let it know' if it worked or not. Now it will try to superficially diagnose the problem and then a…

> I think if you keep pressing in the right ways it'll eventually give in and help you as it did before, but I guess this will take quite a bit of prompting. So much work to avoid work.

I don't mind putting in a lot of lazy effort to avoid strenuous intellectual work, that shit is very hard.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#536

Post by jotux »

MuffinFlavored wrote: post #493
phillipcarter wrote: post #461

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

No, OpenAI 100% pushed an update recently that is very noticeable where they basically "nerfed" the responses. I'm sure it was a business decision (either costing them too much to give away the kitchen sink for free or they want to turn around and charge more for the "really good, old" version of GPT-4 ) but you can visually see where the bot used to try to answer complex tasks, now it has an extra layer where it say…

>OpenAI 100% pushed an update recently that is very noticeable where they basically "nerfed" the responses

>they want to turn around and charge more for the "really good, old" version of GPT-

If this was the case it is provable by submitting queries to `gpt-4-0314` and comparing them to `gpt-4`.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#537

Post by dehrmann »

phillipcarter wrote: post #461

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

I suspect it's the same people complaining how Google's gotten worse.

One time, I went to the hotel restaurant. I ordered something; it was amazing. Figured I'd do it again a few days later. It wasn't bad, but it wasn't as good. There wasn't any specific difference I could put my finger on other than that the novelty wore off.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#539

Post by GartzenDeHaes »

bragr wrote: post #436
Spooky23 wrote: post #423

Earlier quoted context omitted.

Internet? Try the anarchists cookbook!

If you actually try the anarchists cookbook, you will find many of the recipes don't work or work that well.

Or explode. The FBI scrapes up the remains of a few of these a*holes every year.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#540

Post by geysersam »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

You have no idea what you're talking about. Why would such a classification step remove any information about typical "benign" queries?

It's a lot more likely they just nerfed the model because it's expensive to run.

Post reply on HN