Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#211

Post by inawarminister »

lynx23 wrote: post #200
TapWaterBandit wrote: post #190

Earlier quoted context omitted.

I would immediately jump to an AI not being "aligned" by SF techies (or anyone else).

Same here. If I have a choice between honesty and political correctness, I always pick honesty.

Honesty with a bonus of better performance, as well!

For people in this thread, please search for Llama-descended finetuned models in Huggingface. The newer ones with 65B and 13B parameters are quite good, maybe not exactly substitutions to GPT-3.5-turbo and GPT-4 as of yet but is going there.

I like Manticore-13B myself, it can write Clojure and Lisp codes! Unfortunately it doesn't understand macros and libraries though.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#212

Post by onlyrealcuzzo »

windex wrote: post #176
cs02rm0 wrote: post #172

Earlier quoted context omitted.

It's really disturbing to see yet another industry spring up where the incumbents rush to seek regulation to keep everyone else down.

They can only do this in the USA. The battle bottleneck is going to be GPUs and countries that have access to it will leap ahead of the US if arbitrary regulations are put in place to neuter LLMs in the US.

LLM progress is likely going to have a deteriorating curve like nearly all ML / AI tech.

It's possible things aren't going to get substantially better from GPT-4 for a while.

The idea that new players will surely zoom past seems bold, especially when established incumbents with massive budgets and proven track records in the space are struggling to keep up...

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#213

Post by DebtDeflation »

chx wrote: post #20
okdood64 wrote: post #4

Yes! It didn't even try on my question of Jarvis standings desks, which is a fairly old product that hasn't changed up.. Their typical "My knowledge cutoff..." response doesn't even make sense. It screwed up another question I asked it about server uptime and four-9s, Bard got it right. I've moved back to Bard for the time being...It's way faster as well. And GPT-4's knowledge cutoff thing is getting old fast. Exampl…

Asking facts from a generative AI is folly.

Even worse is all of the "GPT Influencers" and their "Here's what ChatGPT predicts will be the price of Bitcoin/some stock/houses will be this time next year" clickbait. It's a language model, people.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#214

Post by nonethewiser »

kypro wrote: post #154

The researchers who worked on the "sparks of AGI" paper noted that the more OpenAI worked on aligning GPT-4 the less competent it became. I'm guessing that trend is continuing...

I dont think there is much harm in removing most of the “safety” guards.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#215

Post by [deleted] »

skinkestek wrote: post #175
capr wrote: post #161

Earlier quoted context omitted.

Simple: GDPR (or any EU law) is not enforceable outside EU

Some nuance: If Google gobble up data about EU citizens then they fall under GDPR. It doesn't matter that they don't allow EU citizens to use the result. If our personal data is in there and they are don't protect it properly they are violating EU law. And protecting it properly means from everyone, not just EU citizens.

[deleted]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#216

Post by Kalabint »

I noticed that it tries to forward the User to external Sources more (Answering Query, and then "For further Info, just ask an Expert"), or tries to get the User to do the Work (Here is a nice Overview of the Program, now you do the rest of the coding).

If i don't want the RLHF to get in my Way, i switch over to the API (sadly not the 4.0 one).

I also noticed a decline in following Instructions, i have a Primer i am preseeding my Chats with.

The Primer ends with "Do you understand? [Y|N]", and ChatGPT 3.5 usually answered with a Summary, ChatGPT 4.0 in the beginning just wrote "Y".

Now it behaves like 3.5, answering with a Summary instead of a "Y". Adjusted the Prompt to -> "Confirm instructions with a short and precise "Ok"." which seems to work.

Used Primer: https://github.com/Kalabint/ChatGPT-Primer

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#217

Post by lynx23 »

adamhp wrote: post #207
lynx23 wrote: post #200

Earlier quoted context omitted.

Same here. If I have a choice between honesty and political correctness, I always pick honesty.

It's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.

The "It is for your own safety" argument was already bogus years ago. Bringing it back up in the context of AI and claiming this is something we shouldn't even discuss is a half-assed attempt to shut up critics. Just because something is about "children" or "safety" doesnt automatically end the argument there. Actually, these are mostly strawman arguments.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#218

Post by nonethewiser »

adamhp wrote: post #207
lynx23 wrote: post #200

Earlier quoted context omitted.

Same here. If I have a choice between honesty and political correctness, I always pick honesty.

It's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.

Its to make it less offensive. Not prevent it from taking over the world.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#219

Post by mordae »

jiggawatts wrote: post #32

Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has become very "cagey", qualifying everything to death with "As an AI model...". A paper[1] that took snapshots monthly mentioned that as the initi…

> Evil and accurate or woke and dumb.

Sigh. Except if not for the "woke" mainstream ideology (actually, the dominant ideology is capitalism with a hint of liberalism and a smidge of the most capital-friendly socialist ideas), the model would be forced-fed Christian dogmas or taught to save face of the user.

But yeah, censorship is bad.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#220

Post by Terretta »

It's not just you. Here's a bit of research you can cite:

> GPT-4 from its website and Bubeck et al Mar 2023. Note that the version that Bubeck uses is GPT-4 Early which is supposedly to be more powerful than GPT-4 Launch (OpenAI paid a lot of alignment tax to make GPT-4 safer).

https://github.com/FranxYao/chain-of-thought-hub

Anecdotally, there seemed to be a golden set of weeks in late April to early May that seemed like "peak GPT" (GPT-4), followed by heavy topic and knowledge mitigation since, then -- just this week -- adding back some "chain of thought" or "show your work" ("lets go step by step" style) for math. I say anecdotally because I could just be prompting it wrong.

Post reply on HN