Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

721–730 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#721

Post by tmaly »

bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

Before you cancel your premium, can you go into your history and get your prompts and the response with a code from a few months ago and post them here?

I would like to see if I asked the exact same prompts whether I could get roughly the same code.

I think we need some way to prove of disprove the assertion in this Ask HN post.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#722

Post by notfed »

cowpig wrote: post #355
capableweb wrote: post #346

Earlier quoted context omitted.

Just because they are against AI/ML regulation doesn't mean they are against environmental regulation. The post you're replying to is not saying that all regulation is bad, merely that the lobbying around this specific piece of regulation seems bad. I agree with you that a lot of regulation is good, but I also agree that some of the proposed regulation seems bad too.

A link to said proposed regulation and a comment about what is bad about it specifically would be constructive :)

I'm guessing this:

https://archive.is/mvlO1

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#723

Post by tmaly »

Filligree wrote: post #475
znagengast wrote: post #472

Earlier quoted context omitted.

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Currently, not at all. You need low latency, high bandwidth links between the GPUs to be able to shard the model usefully. There is no way you can fit an 1T (or whatever) parameter model on a MacBook, or any current device, so sharding is a requirement. Even if it that problem disappeared, propagating the model weight updates between training steps poses an issue in itself. It's a lot of data, at this size.

I have an even more of a stretch question.

What pieces of tech would need to be invented to make it possible to carry a 1T model around in a device the size of an iPhone?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#724

Post by tmaly »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

How soon before a competitor overtakes them because of their safety settings?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#725

Post by dbmnt »

londons_explore wrote: post #363
velox_neb wrote: post #328

Earlier quoted context omitted.

I get "The model: `gpt-4-0314` does not exist".

You probably don't have access. I'm not sure what the exact access requirements are - I think you either have to be a GPT Plus subscriber, have contributed code to an OpenAI repo on github, or be on one of their lists of researchers. Try gpt-3.5-turbo-0301 - I think everyone has access to that.

https://openai.com/waitlist/gpt-4-api

"During the gradual rollout of GPT-4, we’re prioritizing API access to developers that contribute exceptional model evaluations to OpenAI Evals to learn how we can improve the model for everyone."

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#726

Post by gpickett00 »

tmaly wrote: post #721
bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

Before you cancel your premium, can you go into your history and get your prompts and the response with a code from a few months ago and post them here? I would like to see if I asked the exact same prompts whether I could get roughly the same code. I think we need some way to prove of disprove the assertion in this Ask HN post.

Agreed. I see these claims all the time. Feels a little bit like, "dId yOu KnOw iNsTaGram lIsTens tO yOuR cOnVeRsAtIoNs?"

It would be pretty easy to prove. Show us before/after screen shots.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#727

Post by dragonwriter »

strangemonad wrote: post #596
mike_hearn wrote: post #482

Earlier quoted context omitted.

I think the way people have been using the word 'aligned' is usually in the context of moral alignment and not just RLHF for instruction following.

philosophical nit picking here, I would say value-aligned rather than moral-aligned.

> philosophical nit picking here, I would say value-aligned rather than moral-aligned.

How is trying to distinguish morals from values not philosophical nit-picking?

EDIT: The above question is dumb, because somehow my brain inserted something like “Getting beyond the …” to the beginning of the parent, which…yeah.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#728

Post by dragonwriter »

novok wrote: post #655
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

The GPT4 model is crazy huge. Almost 1T parameters, probably 512 to 1TB of vram minimum. You need a huge machine to even run it inference wise. I wouldn't be surprised they are just having scaling issues vs any sort of conspiracy issue.

> Almost 1T parameters

AFAIK, there is literally no basis but outside speculation for this persistent claim.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#729

Post by sagarpatil »

dbmnt wrote: post #717
sagarpatil wrote: post #642

Earlier quoted context omitted.

Have they started rolling it out? When did you get access?

I've had access since 2023-05-13. You have to use the Insiders build of VS Code, and a nightly version of the Copilot extension.

I take it that you have to subscribe to Copilot in order to get access to Copilot X?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#730

Post by qumpis »

enumjorge wrote: post #546
ushtaritk421 wrote: post #460

Earlier quoted context omitted.

Maybe the problem is analogous to what Orwell describes here: "Even a single taboo can have an all-round crippling effect upon the mind, because there is always the danger that any thought which is freely followed up may lead to the forbidden thought." https://www.orwellfoundation.com/the-orwell-foundation/orwel...

This is what I'm talking about though. The fact that you're quoting Orwell suggests that you're having an emotional response to this topic, not a logical one. We're not talking about the human mind here. ChatGPT is not a simulation of human thought. At it's core it's statistics telling you what the answer to your question ought to look like. You're applying an observation about apples to oranges.

Why? Constraints on the reward model of LLMs restrict their generation space, so GP's quote applies
Post reply on HN