Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

541–550 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#541

Post by NLPaep »

dehrmann wrote: post #537
phillipcarter wrote: post #461

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

I suspect it's the same people complaining how Google's gotten worse. One time, I went to the hotel restaurant. I ordered something; it was amazing. Figured I'd do it again a few days later. It wasn't bad, but it wasn't as good. There wasn't any specific difference I could put my finger on other than that the novelty wore off.

No, GPT4 was originally a very kurtz asshole. It was hilarious and I shared it with everyone. Now it’s incredibly polite and explains things thoroughly. But it’s a little dumber.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#542

Post by OOPMan »

znagengast wrote: post #472
dasil003 wrote: post #430

Earlier quoted context omitted.

Who has the necessary resources to run, let alone train the model?

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

I once used a crowdsourcing system called CrowdFlower for a pretty basic task, the results were pretty bad.

Seems like with minimal oversight the human workers like to just say they did the requested task and make up an answer rather than actually do it (The task involved entering an address in Google maps, looking at the street view and confirming insofar as possible if a given business actually resided at the address in question, nothing complicated)

Edit: woops, mixed in the query with another reply that mentioned the human element XD

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#543

Post by geysersam »

vervez wrote: post #528
geysersam wrote: post #523

Earlier quoted context omitted.

> I like to imagine certain billion/trillion-dollar mega corps had a back-room say regarding things that they would really prefer OpenAI's models not be able to emit. What a weird conspiracy theory. Why would Microsoft have anything against your pdf-parser? More likely it just costs them insane amounts of money running their most capable models, and therefore they're "nerfing" them to reduce costs .

github copilot won't pay for itself.

So the theory is: Microsoft nerfs GPT4, a product they (basically) own that people pay to access, so that people will stop using that service and pay for another Microsoft product instead?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#544

Post by [deleted] »

loudmax wrote: post #374
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

I tend to be sympathetic to arguments in favor of openly accessible AI, but we shouldn't dismiss concerns about unaligned AI as frivolous. Widespread unfiltered accessibility to "unaligned" AI means that suicidal sociopaths will be able to get extremely well informed, intelligent directions on how to kill as many people as possible. It may be that the best defense against these terrorists is openly accessible AI givi…

[deleted]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#545

Post by IceHegel »

tomrod wrote: post #293
hanselot wrote: post #279

Earlier quoted context omitted.

How is racism different from stereotype? How is stereotype different from pattern recognition? These questions don't seem to go through the minds of people when developing "unbiased/impartial" technology. There is no such thing as objective. So, why pretend to be objective and unbiased, when we all know its a lie? Worst, if you pretend to be objective but aren't, then you are actually racist.

Actually, we folks who work with bias and fairness in mind recognize this. There are many kinds of bias. It is also a bit of a categorical error to say bias = pattern recognition. Bias is a systematic deviation of a parameter estimate based on sampling from its population distribution. The Fairlearn project has good docs on why there are different ways to approach bias, and why you can't have your cake and eat it too…

When did people start to use “folks” in this unnatural way.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#546

Post by enumjorge »

ushtaritk421 wrote: post #460
enumjorge wrote: post #435

Earlier quoted context omitted.

Do you have any sources for that? How would making ChatGPT less likely to return a racist answer or hate speech affect its ability to return code? After a question has been classified into a coding problem, presumably ChatGPT servers could now continue to solve the problem as usual. Maybe running ChatGPT is really expensive, and they nerfed in order to reign in costs. That would explain why the answers we get are les…

Maybe the problem is analogous to what Orwell describes here: "Even a single taboo can have an all-round crippling effect upon the mind, because there is always the danger that any thought which is freely followed up may lead to the forbidden thought." https://www.orwellfoundation.com/the-orwell-foundation/orwel...

This is what I'm talking about though. The fact that you're quoting Orwell suggests that you're having an emotional response to this topic, not a logical one. We're not talking about the human mind here. ChatGPT is not a simulation of human thought. At it's core it's statistics telling you what the answer to your question ought to look like. You're applying an observation about apples to oranges.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#547

Post by twobitshifter »

ForHackernews wrote: post #106
sirsinsalot wrote: post #83

Given the incoming compute capability from nvidia and the speed of advancement, we have to stop and think ... does it make sense to give access, paid or otherwise, to these models once they reach a certain sophistication? Or does it make even more sense to hoard the capability to out compete any competitor, of any kind, commercially or politically and hide the true extent of your capability to avoid scrutiny and legi…

I used to agree with what you're saying, but after reading this leaked Google memo I think maybe we're both wrong: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... Experts who are both in a position to know, and seeking to maximize the commercial potential of their work are saying the cat is already out of the bag. They make a persuasive case that public, open-source models are closing the gap with pri…

Interesting, but Google has the worlds largest search index to use to build models, and billions of android phones and gmail accounts. An open source model may share the same algorithm but it’s possible training set will be dwarfed by Google. It even might have the same number of connections. The article is arguing that a few billion is enough, but what about 5 years from now and even for fewer connections data quality wouldn’t matter? Sure you can run a model slowly on a raspberry pi, but custom silicon can’t do more?

There’s a linked “data doesn't do what you think” document in that post, which might counter this argument but the site is now down.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#549

Post by clint »

MuffinFlavored wrote: post #493
phillipcarter wrote: post #461

I don't think it's any worse at all. I think what most people are expressing here is reaching the limits of the technology and realizing that it's not magic.

No, OpenAI 100% pushed an update recently that is very noticeable where they basically "nerfed" the responses. I'm sure it was a business decision (either costing them too much to give away the kitchen sink for free or they want to turn around and charge more for the "really good, old" version of GPT-4 ) but you can visually see where the bot used to try to answer complex tasks, now it has an extra layer where it say…

I think you're wrong. I've noticed no change.
Post reply on HN