Experiencing decreased performance with ChatGPT-4
community.openai.com
Experiencing decreased performance with ChatGPT-4
1–10 of 200 posts
Re: Experiencing decreased performance with ChatGPT-4
#2I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.
Re: Experiencing decreased performance with ChatGPT-4
#3I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.
Nice try, OpenAI
Re: Experiencing decreased performance with ChatGPT-4
#4Without concrete examples, I do wonder if much of this is perceptual. I love using ChatGPT, but once the amazement that it works as well as it does has worn off, one ends up spotting the flaws more than before.
I feel that advocates and critics of ChatGPT are both right, to a degree, but looking at the models responses from slightly different angles: it wouldn't be surprising if users' angles shift over time.
Re: Experiencing decreased performance with ChatGPT-4
#5Wonder if this is the same as the discussion from 35 days ago on "OpenAI Employee: GPT-4 has been static since March"
Re: Experiencing decreased performance with ChatGPT-4
#6Note that this post is from May 28, before the release of gpt-4-0613. By "the last two updates", I believe the poster is referring to some UI changes, that possibly also included some underlying model changes(?)
Re: Experiencing decreased performance with ChatGPT-4
#7Not exactly related, but it makes me wonder what kind of metrics you can devise for tracking "regressions" on LLMs.
I feel like OpenAI definitely has thought about this at length, but I'm curious about what matters most for them / raises OpsGenie/whatever alerts. Internal model metrics? Customer usage patterns? Random test conversation diffs?
Re: Experiencing decreased performance with ChatGPT-4
#8I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.
Observing large groups of humans acting like large groups of humans is a very interesting pastime.
Some people even make it their life's work.
Re: Experiencing decreased performance with ChatGPT-4
#9I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.
They've definitely changed something about the models, and it is in their interests to do so, both to create a low-latency experience, but most importantly, to save money.
While GPT-4 is still workable, GPT-3.5 flatly refuses requests these days, claiming that as an "AI language model" it couldn't help me write code.
Re: Experiencing decreased performance with ChatGPT-4
#10I've seen posts similar to this one maybe every week or two in various GPT forums. I suspect it's just an illusion, where you remember all the amazing hits that GPT4 had when you first started using it, and remember fewer of the times in the past that GPT4 gave a sub-par answer. In fact, it reminds me of the illusion that "Hacker News is turning into Reddit" and I think that it happens for a similar reason.