Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

131–140 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#131
post #111
post #55

Earlier quoted context omitted.

Something that follows my actual requests, without trying to lecture me about feminism and other U.S. Democrats topics.

What is an example of a request that is causing these issues?

Explaining it to you will not fix my problems with OpenAI's instance. Let it suffice that I am a user of ChatGPT who really dislikes the lecturing style of woke-fine-tuning. If a human would lecture me in that style, I'd dumb them and avoid contact.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#134
post #40

Earlier quoted context omitted.

Try out Bard, it's coding is much improved in the last 2 weeks. I've unfortunately switched over for the time being.

“Bard isn’t currently supported in your country. Stay tuned!”

The Bard model (Bison) is available without region lock as part of Google Cloud Platform. In addition to being able to call it via an API, they have a similar developer UI to the OpenAI playground to interactively experiment with it.

https://console.cloud.google.com/vertex-ai/generative/langua...

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#136
post #54
post #53

Let me guess: the "woke fine-tuning" deteriorates quality of the model. Seriously, this lecturing style of GPT is totally useless. You ask it something vaguely unwoke, it lectures you about your style, you explain what it didn't understand about your request, it appologizes and does vaguely what you originally requested. This sort of deture is totally useless and a waste of my time. To stay with the terminology, if I…

Genuinely asking, what's an "unwoke" prompt?

To be fair to the commenter, I've gotten weird moralising on completely apolitical topics from ChatGPT before.

LLMs are (naturally) very sensitive to their prompts, so if the prompt includes something about being inclusive - at ChatGPT's obviously does - the LLM output will sometimes find ways to work that in in odd ways.

I don't mind as much as the commenter seems to - I have no beef with inclusivity - but I have rolled my eyes at times at what ChatGPT comes up with.

I've played with local LLMs too, including some 'uncensored' ones, but the gap in ability between them and something like GPT is still vast. I look forward to progress on this front though.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#137

There's no doubt that it's gotten a lot worse on coding, I've been using this benchmark on each new version of GPT-4 "Write a tiptap extension that toggles classes" and so far it's gotten it right every time, but not any more, now it hallucinates a simplified solution that don't even use the tiptap api any more. It's also 200% more verbose in explaining it's reasoning, even if that reasoning makes no sense whatsoever…

Do you have API access? If so, have you tried your tiptap question on the gpt-4-0314 model? That is supposedly the original version released to the public on March 14.

I did, but it got it almost the same as GPT-3.5 Turbo, the best version of it where there recently (~2-3 weeks ago), where it would make specific chunks of code-changes and explain the chunk in a concise and correct manner - even making suggestions on improvements. But that's entirely gone now..

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#138
post #14

Yes. Before the update, when its avatar was still black, it solved pretty complex coding problems effortlessly and gave very nuanced, thoughtful answers to non-programming questions. Now it struggles with just changing two lines in a 10-line block of CSS and printing this modified 10-line block again. Some lines are missing, others are completely different for no reason. I'm sure scaling the model is hard, but they l…

Same happened with Dalle-2. It went downhill after a couple of weeks.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#139

There's no doubt that it's gotten a lot worse on coding, I've been using this benchmark on each new version of GPT-4 "Write a tiptap extension that toggles classes" and so far it's gotten it right every time, but not any more, now it hallucinates a simplified solution that don't even use the tiptap api any more. It's also 200% more verbose in explaining it's reasoning, even if that reasoning makes no sense whatsoever…

Do you have by any chance tested the same question on the playground? I've noticed a quality decrease iny telegram bot as well that directly uses the API, and it drives me crazy because model versioning was supposedly implemented specifically to avoid response change without notice

Yes, using the general assistant role and the default content:

"You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture. Knowledge cutoff: 2021-09 Current date: 2023-05-31"

And custom roles with custom content via API.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#140
post #68
post #47

Earlier quoted context omitted.

Google's passion for region locking is insane to me

Its a legal thing, not something they want to do

Eh, more like limiting rollout because they can't/don't want to handle the scale.
Post reply on HN