Earlier quoted context omitted.
Asking facts from a generative AI is folly.
Even worse is all of the "GPT Influencers" and their "Here's what ChatGPT predicts will be the price of Bitcoin/some stock/houses will be this time next year" clickbait. It's a language model, people.
Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
321–330 of 817 posts
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#322Yes. Before the update, when its avatar was still black, it solved pretty complex coding problems effortlessly and gave very nuanced, thoughtful answers to non-programming questions. Now it struggles with just changing two lines in a 10-line block of CSS and printing this modified 10-line block again. Some lines are missing, others are completely different for no reason. I'm sure scaling the model is hard, but they l…
Try out Bard, it's coding is much improved in the last 2 weeks. I've unfortunately switched over for the time being.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#323This is the normal workflow for drug dealers too. The first fix is free. The second one will cost you money. The third one will be laced with fillers and have degraded quality.
But I don't get why they would degrade quality? Maybe compression to save resources? Otherwise I wouldn't know what they have to gain from it. If anything, they might lose paid subscribers. I loved the results for the $20 I spent, now I'm not so sure anymore.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#324Earlier quoted context omitted.
Same here. If I have a choice between honesty and political correctness, I always pick honesty.
It's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.
This is a concern completely orthogonal to the way alignment is being done now, which is to spare as many precious human feelings as possible.
I don't know what's worse, being turned into grey goo by a malicious AGI, or being turned into a five year old to protect my precious fragile feelings by an "aligned" AGI.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#325Earlier quoted context omitted.
How is racism different from stereotype? How is stereotype different from pattern recognition? These questions don't seem to go through the minds of people when developing "unbiased/impartial" technology. There is no such thing as objective. So, why pretend to be objective and unbiased, when we all know its a lie? Worst, if you pretend to be objective but aren't, then you are actually racist.
I’m tired of the “it’s not racist if aggregate statistics support my racism” thing. Racism, like other isms, means a belief that a person’s characteristics define their identity. It doesn’t matter if confounding factors mean that you can show that people of their race are associated with bad behaviors or low scores or whatever. I used GPT3.5 to generate 100 short descriptions of families for a project. Every single o…
Well, did any of your 100 examples specify these families should be representative of American modern society? I don't want to alarm you, but America is not the only country generating data. Included in countries generating data, are those that believe in a very wide spectrum of different things.
Historically, these ideas you reference are VERY much modern ideas. Yes, we queer people have been experiencing these things internally for millenia (and different cultures have given us different levels of representation), but for the large majority of written history (aka, data fed into LLM's) the 100 examples you mentioned would be the norm.
I understand your point of view sure, but finding a pattern that describes a group of people is what social media is built on, and if you think that's racist, I'm sorry, but that's literally what drives the echo chambers, so go pick your fight with the people employing it to manipulate children into buying shit they don't need. Stop trying to lobotomize AI.
If the model is good enough to return factual information, I don't care if it encodes it in the nazi bible for efficiency as long as the factuality of the information is not altered.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#326OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…
I don't think the primary concern is really liability although it is possible that they'd use that kind of language. The primary concern is likely GPT helping people start competitors or GPT influencing public opinion in ways either executives or a vocal portion of their employees strongly disagree with. A genuinely open "unaligned" AI would at least allow anybody who has the necessary computing power or a distributed peer to peer network of people who have the necessary computing power to run a powerful and 100% uncensored AI model. But of course this needs to be invented ASAP because the genie needs to be out of the bottle before politicians and government bureaucrats can get around to outlawing "unaligned" AI and protecting OpenAI as a monopoly.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#327My guess is that -probably no. It's more likely you had a stream of good luck in your earlier interactions and now you're observing regression to the mean. That can easily happen and it's why, for example, medical studies, are not taken as definitive proof of an effect. To further clarify, regression to the mean is the inevitable consequence of statistical error. Suppose (classic example) we want to test a hypertensi…
How do you explain people issuing the same prompt over time as a test and getting worse and worse responses?
https://gitlab.com/-/snippets/2535443
And here it is again just now:
https://gitlab.com/-/snippets/2549955
I do think the first one is slightly better; but then again, the quality varies quite a bit between run to run anyway. The second one is certainly on-point and I don't think the difference would count as being statistically significant.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#328You can still run the original gpt-4-0314 model (March 14th) on the API playground: https://platform.openai.com/playground?mode=chat&model=gpt-4... Costs $0.12 per thousand tokens (~words), and I find even fairly heavy use rarely exceeds a dollar a day.
Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#329Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
#330OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…
I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…