Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

801–810 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#801

Post by lunakid »

TeMPOraL wrote: post #787
bombcar wrote: post #312

Earlier quoted context omitted.

That indicates the “nerfing” is not what I would think (a final pass to remove badthink) but somehow deep in everything, because the question asked should be orthogonal.

Think how it works with humans. If you force a person to truly adopt a set of beliefs that are mutually inconsistent, and inconsistent with everything else the person believed so far, would you expect their overall ability to think to improve ? LLMs are similar to our brains in that they're generalization machines. They don't learn isolated facts, they connect everything to everything, trying to sense the underlying…

Absolutely. And if one wants to look for scary things, a big one is how there seem to be genuine efforts to achieve proper alignment and safety based on the shaky ground(s) of our "human value system(s)" -- of which even if there was only One True Version, it would still be way too haphazard and incoherent, or just ill-defined, to anything as truly honest and bias-free as a blank-slate NN model to base it's decisions on.

That kinda feels like a great way to achieve really unpredictable/unexpected results instead in rare corner cases, where it may matter the most. (It's easy to be safe in routine everyday cases.)

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#802

Post by lunakid »

nomel wrote: post #702
i_dont_know_ wrote: post #18

To me, it feels like it's started giving superficial responses and encouraging follow-up elsewhere -- I wouldn't be surprized if its prompt has changed to something to that effect. Before, if I had an issue with a library or debugging issue, it would try to be helpful and walk me through potential issues, and ask me to 'let it know' if it worked or not. Now it will try to superficially diagnose the problem and then a…

I wouldn't be surprised if this was from an attempt to make it more "truthful". I had to use a bunch of jailbreaking tricks to get it to write some hypothetical python 4.0 code, and it still gave a long disclaimer.

Hehe, wonderful! :) Did it actually invent anything noteworthy for P4?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#803

Post by lunakid »

PaulHoule wrote: post #267

Are they trying to lower the running cost?

Just trying to keep up with demand could explain it alone. The strict quotas (even for Plus users), the occasional errors and long waits etc. etc. indicate that the hardware performance has clearly been a bottleneck all throughout.

And demand has exploded since, so it's plausible that they had no choice but to sacrifice quality, if they wanted to keep the service running.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#804

Post by kilobite88 »

YeGoblynQueenne wrote: post #244

My guess is that -probably no. It's more likely you had a stream of good luck in your earlier interactions and now you're observing regression to the mean. That can easily happen and it's why, for example, medical studies, are not taken as definitive proof of an effect. To further clarify, regression to the mean is the inevitable consequence of statistical error. Suppose (classic example) we want to test a hypertensi…

what's more plausible: the startup that runs gpt has changed something internally to degrade the quality or somehow across the entire internet, chatgpt4 users are having sudden emergent awareness on how bad it always was but were deluding themselves equally from the beginning?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#805

Post by deanCommie »

jeeeberz wrote: post #799
CamperBob2 wrote: post #789

Earlier quoted context omitted.

I once worked with a guy named Michael who would get bent when you called him Mike. As you can imagine he could be tricky to work with and, on those occasions, I would call him Mike. I repeatedly misnamed him on purpose, it wouldn't have even made HR bat an eye. So, your career at Dell didn't go as well as you'd hoped. Being a jerk isn't illegal, AFAIK, but at some point you run out of other people to blame for the c…

> relatively small number of others who were born with certain unfortunate but addressable hormonal issues Most males who adopt an opposite-sex identity reach that point through repeated erotic stimulation. This is a psychological issue, driven by sexual desire.

[citation needed]

Correlation, causation, etc.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#806

Post by tzekid »

enumjorge wrote: post #435
chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

Do you have any sources for that? How would making ChatGPT less likely to return a racist answer or hate speech affect its ability to return code? After a question has been classified into a coding problem, presumably ChatGPT servers could now continue to solve the problem as usual. Maybe running ChatGPT is really expensive, and they nerfed in order to reign in costs. That would explain why the answers we get are les…

Here's the full talk[0] from a Microsoft lead researcher that worked with the early / uncensored version of GPT-4.

Simplified, tuning it for censorship heavily limits the dimensionality the model can move in to find an answer which means worse results in general for some reason.

[0]: https://www.youtube.com/watch?v=qbIk7-JPB2c

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#807

Post by FeepingCreature »

fnordpiglet wrote: post #331
bbotond wrote: post #14

Yes. Before the update, when its avatar was still black, it solved pretty complex coding problems effortlessly and gave very nuanced, thoughtful answers to non-programming questions. Now it struggles with just changing two lines in a 10-line block of CSS and printing this modified 10-line block again. Some lines are missing, others are completely different for no reason. I'm sure scaling the model is hard, but they l…

This is for your protection from an extinction level event. Without nerfing the current model they couldn’t charge enterprise level fee structures for access to the superior models, thus ensuring the children are safe from scary AI. Tell your congress person we need to grant Microsoft and Google exclusive monopolies on AI research to protect us from open source and competitor AI models that might erode their margins…

This but unironically.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#808

Post by ammar_x »

Like many people here have noticed, it's definitely less quality now than before. It's annoying to be honest to reduce the quality significantly without a notice while we are paying the same amount. I'm willing to pay $40 for the original GPT-4, though.

GPT-4 was remarkable 2 months ago. It could handle complex coding tasks and the reasoning was great. Now, it feels like GPT-3. It's sad. I had many things in mind that could have been done with the original GPT-4.

I hope we see real competitors to GPT-4 in terms of coding abilities and reasoning. Absence of real competitors made it a reasonable option for "Open"AI to lobotomize GPT-4 without a notice.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#809

Post by deanCommie »

deadeye wrote: post #791
jeeeberz wrote: post #778

Earlier quoted context omitted.

[dead]

Wow. I guess it isn't just "one woman".

Are you for real? This is a list of women that "should've" won because of...some unspecified unnamed unverified trans athlete that came ahead of them?

We don't know who is being accused of taking their glory, we don't know if it's 1 person or 100. We don't know if the people that supposedly defeated them is even Trans, or a CIS victim of the Trans Panic like https://en.wikipedia.org/wiki/Caster_Semenya

We don't know if the women who beat these "she won" women are self-identified, have been on hormones for 2 weeks, or 20 years.

What a ludicrous transphobic panic.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#810

Post by jeeeberz »

deanCommie wrote: post #809
deadeye wrote: post #791

Earlier quoted context omitted.

Wow. I guess it isn't just "one woman".

Are you for real? This is a list of women that "should've" won because of...some unspecified unnamed unverified trans athlete that came ahead of them? We don't know who is being accused of taking their glory, we don't know if it's 1 person or 100. We don't know if the people that supposedly defeated them is even Trans, or a CIS victim of the Trans Panic like https://en.wikipedia.org/wiki/Caster_Semenya We don't know…

The purpose of that website is to showcase the achievements of women athletes, not the males who unfairly displaced them in competition. If you look up names and tournaments in your preferred search engine, you will be able to find the additional information you're interested in.

Also, Caster Semenya is male, with a male-only DSD. This is a fact that was confirmed in the IAAF arbitration proceedings. Semenya's higher levels of testosterone, when compared to female athletes, are due to the presence of functional internal testes. Semenya has since fathered children.

Post reply on HN