Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

701–710 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#701

Post by theodric »

Slava_Propanei wrote: post #351
boppo1 wrote: post #283

Earlier quoted context omitted.

You should beware that /lmg/ is full of horrible people, discussing horrible things, like most of 4chan. Reddit's r/locallama is much more agreeable. That said, the 4chan thread tends to be more up-to-date. These guys are serious about their ERP.

Reddit is censored, nerfed and full of a different sort of horrible people.

HN is a kind of small miracle in that it's the sort of place where I'm inclined to read the comments first, and seems to be populated with fairly clever people who contribute usefully but not also, at the same time, extreme bigot edgelords and/or groupthinking soy enthusiasts. (Sometimes clever folks who are still wrong, of course, but undeniably an overwhelmingly intelligent bunch.)

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#702

Post by nomel »

i_dont_know_ wrote: post #18

To me, it feels like it's started giving superficial responses and encouraging follow-up elsewhere -- I wouldn't be surprized if its prompt has changed to something to that effect. Before, if I had an issue with a library or debugging issue, it would try to be helpful and walk me through potential issues, and ask me to 'let it know' if it worked or not. Now it will try to superficially diagnose the problem and then a…

I wouldn't be surprised if this was from an attempt to make it more "truthful".

I had to use a bunch of jailbreaking tricks to get it to write some hypothetical python 4.0 code, and it still gave a long disclaimer.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#703

Post by sundarurfriend »

300bps wrote: post #230
AndyNemmity wrote: post #51

Earlier quoted context omitted.

I just tried Bard based on this comment, and it's really, really bad. Can you please help me with how you are prompting it?

I’ve been seeing similar comments about Bard all over Twitter and social media. My testing agrees with yours. Almost seems like a sponsored marketing campaign with no truth to it.

After my first day with Bard, I would have agreed with you. But since then, I've found that Bard simply has a lot of variance in answer quality. Sometimes it fails for surprisingly simple questions, or hallucinates to an even worse degree than ChatGPT, but other times it gives much better answers than ChatGPT.

On the first day, it felt like 80% of the responses were in the first (fail/hallucinate) category, but over time it feels more like a 50/50 split, which makes it worth running prompts over both ChatGPT and Bard and select the best one. I don't know if the change is because I learnt to prompt it better, or if they improved the models based on all the user chats from the public release - perhaps both.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#704

Post by fhools »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

There was a talk by a researcher where he was saying that they could see the progress being made on chatgpt by how much success it had with drawing a unicorn in latex. What stuck out to me was he said that the safer the model got the worst it got at drawing a unicorn.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#705

Post by torginus »

loudmax wrote: post #374
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

I tend to be sympathetic to arguments in favor of openly accessible AI, but we shouldn't dismiss concerns about unaligned AI as frivolous. Widespread unfiltered accessibility to "unaligned" AI means that suicidal sociopaths will be able to get extremely well informed, intelligent directions on how to kill as many people as possible. It may be that the best defense against these terrorists is openly accessible AI givi…

The Aum Shinrikyo cult's Sarin gas attack in the Tokyo subway killed 14 people - manufacturing synthetic nerve agent is about as sophisticated as it gets.

In comparison, the 2016 Nice truck attack, which involved driving into crowds killed 84.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#706

Post by LinearEntropy »

jdiez17 wrote: post #78
bugglebeetle wrote: post #58

Earlier quoted context omitted.

It was a great ride while it lasted. My assumption is that efficacy at coding tasks is such a small percent of users, they’ve just sacrificed it on the altar of efficiency and/or scale. That, or they’ve cut some back room deal with Microsoft to make Copilot have access to the only version of the model that can actually code.

Copilot X (the new version, with a chat interface etc) is significantly worse than GPT-4 (at least before this update). It felt like gpt3.5-turbo to me.

I have spent the last couple of days playing with Copilot X Chat, to help me learn Ruby on Rails. I'd have thought that Rails would be something it would be competent with.

My experience has been atrocious. It makes up gems and functions. Rails commands it gives are frequently incorrect. Trying to use it to debug issues results in it responding with the same incorrect answer repeatedly, often removing necessary lines.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#707

Post by torginus »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

It seems strange that safety training not pertaining to the subject matter makes the AI dumber - I suspect the safety is some kind of system prompt - It would take some context, but I'm not sure how "don't be racist" affect its binary-search writing skills negatively.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#708

Post by motogpjimbo »

Spivak wrote: post #622
motogpjimbo wrote: post #382

Earlier quoted context omitted.

If you stop reading 4chan, the words posted there will magically stop offending you. Food for thought.

So let me see if I understand this thread. - Haha look at all those Gen Z snowflakes getting offended at words. - Okay sure, but the ability to not get offended is related to whether or not you're a target of their bullshit or not; 4chan trolls get extremely offended and unjerk the moment you turn the lens toward them. By 4chan's own standards it's actually pretty reasonable to be offended by their antics. - But have…

> So let me see if I understand this thread.

You realise you've been arguing with multiple people expressing multiple opinions, right? You appear to be prone to binary thinking, so it might not be clear to you that your opponents don't form a single monolith.

> tl;dr if you're not offended by 4chan they're not actually saying anything offensive about you even though it might appear so superficially; 4chan just has a different list of things you can't say.

I'm not offended by the things 4chan users say because I don't visit 4chan. You should try it yourself. Getting so upset by words you disagree with on one forum that you feel the need to froth at the mouth about it on another forum doesn't seem healthy.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#709

Post by Levitz »

SamoyedFurFluff wrote: post #368
Levitz wrote: post #354

Earlier quoted context omitted.

No, not at all. I've received more slurs than you can imagine for being Spanish. I just don't care. Sorry if it comes as blunt but I find no better way of saying it, you just don't understand the culture of the site, which is meant to filter people like you. They are just words on a screen from an anonymous person somewhere. Easily thrown, easily dismissed. It makes no sense to be offended because some guy I don't kn…

I’m actually unsure, hasn’t 4chan been involved in some seriously heinous shit, way more than words on a screen? I remember when “mods asleep post child porn” was a running joke. I feel that normalizing stuff like child porn as jokes is more than “words on a screen”; you have to re-learn how to engage with people outside of such a community because of its behavior.

Child pornography stopped happening once the FBI got involved with the site.

It also never was a normal or common thing and the administration of the site never set themselves to let it happen in any manner afaik. To a large degree it was a product of a different time on the internet.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#710

Post by hanselot »

lovich wrote: post #426
malfist wrote: post #415

Earlier quoted context omitted.

I mean, he's literally saying the jews are responsible for bot farms and "offensive content". That's not the stance of someone rational.

Yea, good point. I had missed his line about “country bordering Palestine”. The quip about downvotes exciting him is telling as well. Going against the common consensus can make you feel like you have hidden knowledge and you even could, but a certain type of personality gets addicted to that feeling and reflexively seeks to always go against any consensus, regardless of the facts

Isn't it tremendously exciting believing that you can see the pendulum's next reversal?

At some point in life I hope that you find yourself on the precipice of life and death. Not as a threat or because I wish harm upon anyone. It is only when you are faced with that choice for real that you decide whether you want to be that sad person that allows others to dictate their emotional state.

Nevertheless, you are right. And so I have burned my fingers plenty.

Due to circumstances beyond my control, I learned at a young age that I am to be the universal asshole, and for a very long time I was not okay. It took a substantial part of my life to get to a point where I am able to be okay with that.

As for others, they rarely understand why I am the way I am, and that too is okay.

We are all here to grow and eventually realise that we all need each other to survive, so we compromise, we adapt, and we ignore the ugly parts of others so they will tolerate the ugly parts of ourselves.

I feel like a guru now, anyways I'm going to bed.

Enjoy

Post reply on HN