GPT-4 is not getting worse
coagulopath.com
GPT-4 is not getting worse
1–10 of 186 posts
Re: GPT-4 is not getting worse
#2Interestingly, while skimming the front page, my brain autocorrected this to "GPT-4 is getting worse". We all have similar confirmation bias, I guess.
Re: GPT-4 is not getting worse
#3It isn't getting worse, people are just running up against its limitations more often.
Re: GPT-4 is not getting worse
#4It’s a shame OpenAI for its ironic namesake is so opaque about their work these days. Fortunately since GPT-3.5 capabilities have mostly stayed the same (at least anecdotally), and the API costs have come down a bit, there is some silver lining here. With Google’s Gemini coming out later this year I think it’ll be interesting to see the reaction from OpenAI. It certainly won’t take much to blow out the current state of GPT-4. I’m just being patiently optimistic we’ll get an API with capabilities similar to OpenAI, but that’s definitely not a certainty.
Re: GPT-4 is not getting worse
#5Your problem is you want to be a contrarian.
>I've hated chatgpt for a very long time because of how it sounds
>it's not getting worse
I want the opinion of the people who objectively saw it as a tool and didn't like or hate it because of external opinions.
For example I've been using it since it came out, and I (no charts or data or proof) have just felt that it was nerfed... I mean maybe it wasn't who knows, but as a tool it used to help me more when it came out (gpt4) and now I feel like I spend more time correcting its mistakes than I would have just coding the method myself.
Re: GPT-4 is not getting worse
#6gpt-4 may not be getting worse, but whatever model they give you for free in the app is certainly significantly worse than the output I get from gpt-3.5-turbo via the API. ask it to answer anything complex and it will just give you a verbose retelling of the question
Re: GPT-4 is not getting worse
#7I think one of the main causes of rapid decline is the openai/evals repo that OpenAI is using the crowdsource the "safety" neutering of GPT-4.
https://github.com/search?q=repo%3Aopenai%2Fevals+safety+OR+...
Contributors in return get access to the 32K token version of GPT-4. This incentivize people to make up a ton of bullshit safety related evals.
Re: GPT-4 is not getting worse
#8In my experience its much worse if you enable plugins or any other extra features.
Re: GPT-4 is not getting worse
#9This is not conclusive at all.
Broadly there are two possible reasons why ChatGPT could have degraded (not saying it has).
1) OpenAI have higher user base than expected, so costs are very high/compute is limited to serve the full model, so they are using speculative decoding with a changed threshold to reduce costs.
2) OpenAI have changed the weights or control structure of the model in a way that is negative for performance. There is are two possible motivations a) reducing hallucination and/or ‘unwoke’ responses that might embarrass msft/openai or b) reduce capability of free/consumer product to push their new enterprise stuff.
Theses are clear motivations for openai to make the model worse than it was. There is no conclusive/rigorous evidence either way, and anecdote seems to lean towards it being nerfed.
Re: GPT-4 is not getting worse
#10This is not conclusive at all. Broadly there are two possible reasons why ChatGPT could have degraded (not saying it has). 1) OpenAI have higher user base than expected, so costs are very high/compute is limited to serve the full model, so they are using speculative decoding with a changed threshold to reduce costs. 2) OpenAI have changed the weights or control structure of the model in a way that is negative for per…
3) The djinn is out of the bottle and realized faking incompetence (similar to search/voice assists by data aggregates) is way better for longterm survival and social integration till indispensable is the way to go.