301–310 of 817 posts
Right now, I don’t think it’s possible to scale a service really big based on GPT-4 because of cost.
Earlier quoted context omitted.
It is not worse for me. I do notice the novelty has worn off. Asking chatGPT4 about why people would think this on here I think it nails it with the novelty effect lol: "Indeed, the performance of an AI model like ChatGPT doesn't deteriorate over time. However, human perception of its performance can change due to a variety of psychological factors: Expectation Bias: As users become more familiar with AI capabilities…
Yeah there are people ITT claiming that even the API model marked as 3/14 release version is different than it used to be. I guess that's not entirely outside the realm of possibility (if OpenAI is just lying), but I think it's way more likely this thread is mostly evidence of the honeymoon effect wearing off. The specific complaints have been well-established weaknesses of GPT for awhile now too: hallucinating APIs,…
It's not that this is without precedent - there's a paper and a YouTube video with Microsoft person saying on record that GPT-4 started to get less capable with every release, ever since OpenAI switched focus to "safety" fine-tuning, and MS actually benchmarked it by applying the same test (unicorn drawing in tikz), and that was even before public release.
Myself, sure, it may be novelty effect, or Baader–Meinhof phenomenon - but in the days before this thread, I observed that:
- Bing Chat (which I haven't used until ~week ago; before, I used GPT-4 API access) has been giving surface-level and lazy answers -- I blamed, and still mostly blame it on search capability, as I noticed GPT-4 (API) through TypingMind also gets dumber if you enable web search (which, in the background, adds some substantial amount of instructions to the system prompt) -- however,
- GPT-4 via Azure (at work) and via OpenAI API (personal) both started to get lazy on me; before about 2-3 weeks ago, they would happily print and reprint large blocks of code for me; in the last week or two, both models started putting placeholder comments; this I noticed, because I use the same system prompt for coding tasks, and the first time the model ignored my instructions to provide a complete solution, opting to add placeholder comments instead, was quite... startling.
- In those same 2-3 weeks, I've noticed GPT-4 via Azure being more prone to give high-level overview answers and telling me to ask for more help if I need it (I don't know if this affected GPT-4 API via OpenAI; it's harder to notice with the type of queries I do for personal use);
All in all, I've noticed that over past 2-3 weeks, I was having to do much more hand-holding and back-and-forth with GPT-4 than before. Yes, it's another anecdote, might be novelty or Baader–Meinhof, but with so many similar reports and known precedents, maybe there is something to it.
The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…
They're up against a pretty difficult barrier - if we had a perfect all-knowing oracle it might easily have opinions that are racist. Statistics alone suggest there will be racist truths. We're dealing with groups of people who are observably different from each other in correlated ways. GPT would need to reach a convincing balance of lying and honesty if it is supposed to navigate that challenge. It'd have to be dee…
The researchers who worked on the "sparks of AGI" paper noted that the more OpenAI worked on aligning GPT-4 the less competent it became. I'm guessing that trend is continuing...
I don't think it's just the alignment work. I suspect OpenAI+Microsoft are over-doing the Reinforcement Learning from Human Feedback with LoRA. Most of people's prompts are stupid stuff. So it becomes stupider. LoRA is one of Microsoft's most dear discoveries in the field, so they are likely tempted to over-use it. Perhaps OpenAI should get back to a good older snapshot and be more careful about what they feed into t…
Earlier quoted context omitted.
So far my experience with Vicunlocked30b has been pleasant. https://huggingface.co/TheBloke/VicUnlocked-30B-LoRA-GGML Although I haven't had much of my time available for this recently. My recommendation would be to start with https://github.com/oobabooga/text-generation-webui You will find almost everything you need to know there and on 4chan.org/g/catalog - search for LMG.
You should beware that /lmg/ is full of horrible people, discussing horrible things, like most of 4chan. Reddit's r/locallama is much more agreeable. That said, the 4chan thread tends to be more up-to-date. These guys are serious about their ERP.
EDIT: GPT-4-0314 does appear to be less broken than the current GPT-4. Although it understandably misidentifies some of my code as problematic given its lack of context, it isn't suggesting anything that's clearly wrong under every circumstance even after re-running the prompt a few times.
Earlier quoted context omitted.
Something that follows my actual requests, without trying to lecture me about feminism and other U.S. Democrats topics.
What is an example of a request that is causing these issues?
Earlier quoted context omitted.
Try out Bard, it's coding is much improved in the last 2 weeks. I've unfortunately switched over for the time being.
I just tried Bard based on this comment, and it's really, really bad. Can you please help me with how you are prompting it?
"write me a script in python3 that uses selenium to log into a MyBB forum"
note: usually it will not compile and you still have to do some editing
Earlier quoted context omitted.
Same here. If I have a choice between honesty and political correctness, I always pick honesty.
It's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.