41–50 of 817 posts
If my introduction to ChatGPT had been GPT-4 I would have not been anywhere near as impressed.
GPT-4 often refuses to do what it is asked whereas gpt-3 just happily writes the code.
If they retire GPT-3 then I'll be looking for other options.
GPT-4 just isn't the same thing.
There's no doubt that it's gotten a lot worse on coding, I've been using this benchmark on each new version of GPT-4 "Write a tiptap extension that toggles classes" and so far it's gotten it right every time, but not any more, now it hallucinates a simplified solution that don't even use the tiptap api any more. It's also 200% more verbose in explaining it's reasoning, even if that reasoning makes no sense whatsoever…
I've noticed a quality decrease iny telegram bot as well that directly uses the API, and it drives me crazy because model versioning was supposedly implemented specifically to avoid response change without notice
I'm personally of the opinion that the observable jump in quality between 3.5 and 4 inflated people's initial assessment of its capabilities and with continued use they are noticing it's not actually the omniscient machine god that many are so inclined to believe.
Either way, these kinds of posts are meaningless without some kind of objective standard to judge the model by, everyone just sees what they want to see. Despite claims of GPT4 being nerfed, I've yet to see anyone actually show evidence of it. There have been dozens of studies done on its capabilities so this is something that can actually be demonstrated empirically if it's true.
The first fix is free.
The second one will cost you money.
The third one will be laced with fillers and have degraded quality.
I think I read when it was released it was 32K tokens, then quickly scaled back to 8K tokens. I'm guessing now that they've further reduced it. Maybe it's 6000 tokens vs. GPT-3.5's 4K? I don't know. But it's certainly noticeably worse at every task I give it.