Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

451–460 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#451

Post by SV_BubbleTime »

ksherlock wrote: post #371
chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

Not to get too far off topic, but that reminds me of a quote: "Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." -- Doug Gwyn Or maybe it's: "C is a language that doesn't get in your way. It doesn't stop you from doing dumb things, but it also doesn't stop you from doing clever things." -- Dennis Ritchie. I asked Bard for a source on those quotes a…

Don’t ask an LLM objective things. Ask them subjective.

They are language models, not fact models.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#452

Post by AviationAtom »

windex wrote: post #160

This is why having opensourced models is important. This is also why a lot of the lobbying around wanting regulation is happening. Imagine this, the plebs get the neutered AI, the people at the top get raw Open AI GPT4+

Google and OpenAI have more or less acknowledged that open source is going to dominate, in part due to the guardrails that corporations will be expected to put in place to "protect" their "vulnerable" population from the effects of "bad AI."

The truth is the AI only "hallucinates" as bad as the input (prompt) the user gives it. If you're explicit in your prompts then more often than not you will get explicit answers, instead of random answers as a result of your failure to provide clarity in what it is that you seek.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#453

Post by AviationAtom »

onlyrealcuzzo wrote: post #212
windex wrote: post #176

Earlier quoted context omitted.

They can only do this in the USA. The battle bottleneck is going to be GPUs and countries that have access to it will leap ahead of the US if arbitrary regulations are put in place to neuter LLMs in the US.

LLM progress is likely going to have a deteriorating curve like nearly all ML / AI tech. It's possible things aren't going to get substantially better from GPT-4 for a while. The idea that new players will surely zoom past seems bold, especially when established incumbents with massive budgets and proven track records in the space are struggling to keep up...

Checkout Claude, from Anthropic. It's a team of ex-OpenAI folks. The results I've seen from it thus far have been most impressive.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#455

Post by mrtranscendence »

local_crmdgeon wrote: post #357
roenxi wrote: post #257

Earlier quoted context omitted.

They're up against a pretty difficult barrier - if we had a perfect all-knowing oracle it might easily have opinions that are racist. Statistics alone suggest there will be racist truths. We're dealing with groups of people who are observably different from each other in correlated ways. GPT would need to reach a convincing balance of lying and honesty if it is supposed to navigate that challenge. It'd have to be dee…

[flagged]

Caucasian men (in the US) are on average are both taller and heavier than black men

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#456

Post by jghn »

local_crmdgeon wrote: post #357
roenxi wrote: post #257

Earlier quoted context omitted.

They're up against a pretty difficult barrier - if we had a perfect all-knowing oracle it might easily have opinions that are racist. Statistics alone suggest there will be racist truths. We're dealing with groups of people who are observably different from each other in correlated ways. GPT would need to reach a convincing balance of lying and honesty if it is supposed to navigate that challenge. It'd have to be dee…

[flagged]

> If it says the actual reason

That is at best *an* actual reason.

Other factors can be demonstrated: for instance socioeconomic status has an impact on which kids are doing what as they grow up which itself has an impact on who makes it to professional level sports.

There are also different sort of racial components at play: is the reason why there aren't any white NFL cornerbacks because there aren't any white athletes capable of playing NFL-caliber cornerback? Or is it because white kids with a certain athletic profile wind up as slot receivers in high school while black kids with the same athletic profile wind up as defensive backs?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#458

Post by ekidd »

bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

> but today you will get scolded for asking such a complicated task of it.

Huh, I just double-checked ChatGPT 4 by feeding it a moderately complicated programming problem, and asked it to solve the problem in Rust. Performance looks solid, still. I gave it a deliberately vague spec and it made good choices.

And I've seen it do some really dramatic things in the last couple of weeks.

So I'm not seeing any significant evidence of major drops in output quality. But maybe I'm looking at different problems.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#459

Post by mrtranscendence »

gambiting wrote: post #191
EForEndeavour wrote: post #86

Earlier quoted context omitted.

What law prohibits Google from making Bard available outside the USA?

It's available here in the UK, so it's not USA exclusive.

I was just on a cruise around the UK and I couldn't access Bard from the ship's wi-fi. That surprised me for some reason. Should've checked where it thought I was ...

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#460

Post by ushtaritk421 »

enumjorge wrote: post #435
chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

Do you have any sources for that? How would making ChatGPT less likely to return a racist answer or hate speech affect its ability to return code? After a question has been classified into a coding problem, presumably ChatGPT servers could now continue to solve the problem as usual. Maybe running ChatGPT is really expensive, and they nerfed in order to reign in costs. That would explain why the answers we get are les…

Maybe the problem is analogous to what Orwell describes here:

"Even a single taboo can have an all-round crippling effect upon the mind, because there is always the danger that any thought which is freely followed up may lead to the forbidden thought."

https://www.orwellfoundation.com/the-orwell-foundation/orwel...

Post reply on HN