Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

491–500 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#491
post #436

Earlier quoted context omitted.

Internet? Try the anarchists cookbook!

If you actually try the anarchists cookbook, you will find many of the recipes don't work or work that well.

Quite a few of them work just fine. Dissolving styrofoam into gasoline isn't exactly rocket science. Besides that, for every book that tells you made up bullshit, there are a hundred other books that give you real advice for how to create mayhem and destruction.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#492
This is why you can’t rely on GPT entirely for coding, or at all. Imagine you were a company that abandoned all hiring of software engineers and instead used prompt engineers to develop code, then one day the AI just becomes incompetent and quality of output deteriorates. Your prompt engineers no longer can get the AI to fix things or develop new code. Your entire company is snuffed out, overnight. You might try to hire true software engineers rapidly, but they’ve become very expensive and hard to hire. Too much demand too little supply.

You’re screwed, the AI was your most important tool and it’s broken and there’s nothing you can do about it, it’s a black box, you don’t control it.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#493

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

No, OpenAI 100% pushed an update recently that is very noticeable where they basically "nerfed" the responses. I'm sure it was a business decision (either costing them too much to give away the kitchen sink for free or they want to turn around and charge more for the "really good, old" version of GPT-4 ) but you can visually see where the bot used to try to answer complex tasks, now it has an extra layer where it says "that is too complex to discuss here" (aka it is being trained to not engage by default, whereas that used to not be the case)

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#494

Where does phind.com say that they use GPT-4?

When you submit a query with Phind, it will tell you which model it's using right underneath the query. At least it used to occasionally say that it's using GPT-4 by annotating it with (IIRC) "best model", but now I can only get it to reply with what it calls "Phind model". I don't know what that means or how it relates to GPT-3/4. Presumably they haven't created their own competitive language model from scratch without telling anyone, but who knows.

Though even when it was easier to get it to use GPT-4 it wasn't consistent, and would only use it when it inferred that the query was complex/technical enough.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#495

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

No, OpenAI 100% pushed an update recently that is very noticeable where they basically "nerfed" the responses. I'm sure it was a business decision (either costing them too much to give away the kitchen sink for free or they want to turn around and charge more for the "really good, old" version of GPT-4 ) but you can visually see where the bot used to try to answer complex tasks, now it has an extra layer where it say…

It's likely that this is the result of training it to avoid bullshitting. It gave confident garbage, and they're trying to stem the flow. This likely leads to less certain responses in complex tasks.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#496

Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has become very "cagey", qualifying everything to death with "As an AI model...". A paper[1] that took snapshots monthly mentioned that as the initi…

> Evil and accurate or woke and dumb. Apparently, we can choose. That's because woke is dumb. It's a set of highly biased, inconsistent and reason-defying ideas, evolving under selection pressures that favor emotional appeals and intellectual dishonesty, because one of the core assumptions seems to be that it's not about finding what's right and good for everyone - it's heavily overshooting it in an attempt to cancel…

Tell me you don't have the slightest clue what "woke" means to woke people without saying you don't have the slightest clue what "woke" means to woke people

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#497

Earlier quoted context omitted.

How is racism different from stereotype? How is stereotype different from pattern recognition? These questions don't seem to go through the minds of people when developing "unbiased/impartial" technology. There is no such thing as objective. So, why pretend to be objective and unbiased, when we all know its a lie? Worst, if you pretend to be objective but aren't, then you are actually racist.

I’m tired of the “it’s not racist if aggregate statistics support my racism” thing. Racism, like other isms, means a belief that a person’s characteristics define their identity. It doesn’t matter if confounding factors mean that you can show that people of their race are associated with bad behaviors or low scores or whatever. I used GPT3.5 to generate 100 short descriptions of families for a project. Every single o…

GPT is not a reality simulator. It is just picking the most likely response to an ambiguous question. All you're saying is that the distribution produced by the randomness in GPT doesn't match the true distribution. It's never going to for every single question you could possibly pose.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#498

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

Exactly this. The magic has worn off, and simultaneously, we're coming to terms with the reality that no model can exist in isolation. So changes made, like getting it to not create false rumors about people, affect the rest of the system.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#499

Earlier quoted context omitted.

Who has the necessary resources to run, let alone train the model?

I think a lot of people are unaware that these models have an enormous human training component performed through companies such as Amazon Mechanical Truk and dataannotation.tech. Called Human Intelligence Tasks, a large number of people have been working in this area for close to a decade. Dataannotation Tech claims to have over 100k workers. From Cloud Research, "How Many Amazon Mechanical Turk Workers Are There in…

Here's an account of a person in Africa that helped train (wading thru gnarly explicit content in the process): https://www.bigtechnology.com/p/he-helped-train-chatgpt-it-t...

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#500
post #103

Earlier quoted context omitted.

It's blocked in the EU because they don't want to/can't comply with GDPR.

It is not GDPR, it is available in some countries outside the EU with GDPR-like privacy regimes.

This is naïve though. Regulation — especially such as this — has to be enforced and there is obviously room to over and under interpret the text of the law on a whim, or varying fines. OAI knows this and looking at the EU lately, what they’re doing is wise.
Post reply on HN