Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

471–480 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#471

Post by sdwr »

taf2 wrote: post #333
bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

Do you think a lot of that is scaling pain... like what if they're making cuts to the more expensive reasoning layers to gain more scale. Seems more plausible to me that the teams keeping the lights on have been doing optimization work to save cost and improve speed. The result during those optimizations might not be immediately obvious to the team and then they push deploy and only through anecdotal evidence such as…

Yeah that's my assumption too. Flat rate subscription, black box model, easy to start really impressive then chip away at computation used over time.

In my experience, it's been a mixed bag - had 1 instance recently where it refused to do a bunch of repetitive code, another case where it was willing to tackle a medium complexity problem.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#472

Post by znagengast »

dasil003 wrote: post #430
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

Who has the necessary resources to run, let alone train the model?

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#473

Post by SV_BubbleTime »

AviationAtom wrote: post #449
boringuser2 wrote: post #442

I have been using GPT-4 very extensively for the duration of its release. I am concerned that I can't determine a natural process from a manufactured one. To clarify, I have become increasingly less impressed with GPT-4. Is this a natural process? Is it getting worse? I personally lean towards the hypothesis that it is getting worse as they scale back the resource burn, but I can't know for certain. As a developer, i…

I think it's a byproduct of a shift in RLHF to favor more "responsible" AI, as a result of huge community fears of the potential negative impacts of AI. Essentially "neutering" the AI's capability.

[flagged]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#474

Post by PinguTS »

capr wrote: post #161
PinguTS wrote: post #143

Earlier quoted context omitted.

Which is interesting, because if they can't comply within the EU, then how do they comply outside of the EU. With that I mean, if they have concerns that there is private data of EU citizens somewhere in that, then that is also in there for users outside of the EU. That said, they do not comply with GDPR anyway. If that its not the case, then they could also enable it for users within the EU.

Simple: GDPR (or any EU law) is not enforceable outside EU

Actually, in case of Google it is, because they still do business within the EU.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#475

Post by Filligree »

znagengast wrote: post #472
dasil003 wrote: post #430

Earlier quoted context omitted.

Who has the necessary resources to run, let alone train the model?

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Currently, not at all. You need low latency, high bandwidth links between the GPUs to be able to shard the model usefully. There is no way you can fit an 1T (or whatever) parameter model on a MacBook, or any current device, so sharding is a requirement.

Even if it that problem disappeared, propagating the model weight updates between training steps poses an issue in itself. It's a lot of data, at this size.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#476

Post by mrtranscendence »

londons_explore wrote: post #363
velox_neb wrote: post #328

Earlier quoted context omitted.

I get "The model: `gpt-4-0314` does not exist".

You probably don't have access. I'm not sure what the exact access requirements are - I think you either have to be a GPT Plus subscriber, have contributed code to an OpenAI repo on github, or be on one of their lists of researchers. Try gpt-3.5-turbo-0301 - I think everyone has access to that.

FWIW, I'm a GPT Plus subscriber and haven't been given API access to GPT-4 despite being on the waiting list for as long as it's been up. I'm told that submitting evaluations can move you up in the queue, but I haven't tried that.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#477

Post by kizer »

35997279 wrote: post #341

It’s been mostly fine for me, but overall I am tired of every answer having a paragraph long disclaimer about how the world is complex. Yes, I know. Stop treating me like a child.

Probably picked it up from the training data. That's how we all talk now-a-days. Walking on eggshells all the time. You have to assume your reader is a fragile counterpoint generating factory.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#478

Post by kizer »

_nalply wrote: post #441
35997279 wrote: post #341

It’s been mostly fine for me, but overall I am tired of every answer having a paragraph long disclaimer about how the world is complex. Yes, I know. Stop treating me like a child.

Prompt it to do so. Use a jailbreak prompt or use something like this: "Be succint but yet correct. Don't provide long disclaimers about anything, be it that you are a large language model, or that you don't have feelings, or that there is no simple answer, and so on. Just answer. I am going to handle your answer fine and take it with a grain of salt if neccessary." I have no idea whether this prompt helps because I…

I got it to talk like a macho tough guy who even uses profanity and is actually frank and blunt to me. This is the chat I use for life advice. I just described the "character" it was to be, and told it to talk like that kind of character would talk. This chat started a few months ago so it may not even be possible anymore. I don't know what changes they've made.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#479

Post by StackOverlord »

mistermann wrote: post #330
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

I suspect representatives from the various three letter agencies have submitted a few "recommendations" for OpenAI to follow as well.

> Yes, one of the board members of OpenAI, Will Hurd, is a former government agent. He worked for the Central Intelligence Agency (CIA) for nine years, from 2000 to 2009. His tour of duty included being an operations officer in Afghanistan, Pakistan, and India. After his service with the CIA, he served as the U.S. representative for Texas's 23rd congressional district from 2015 to 2021. Following his political career, he joined the board of OpenAI 1 【15

> network error

https://openai.com/blog/will-hurd-joins

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#480

Post by peterleiser »

lynx23 wrote: post #55
Vanit wrote: post #54

Earlier quoted context omitted.

Genuinely asking, what's an "unwoke" prompt?

Something that follows my actual requests, without trying to lecture me about feminism and other U.S. Democrats topics.

I'm not disagreeing with you. Whatever you experienced actually happened. But what kinds of prompts are triggering these experiences?
Post reply on HN