Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

371–380 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#371

Post by ksherlock »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

Not to get too far off topic, but that reminds me of a quote:

"Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." -- Doug Gwyn

Or maybe it's:

"C is a language that doesn't get in your way. It doesn't stop you from doing dumb things, but it also doesn't stop you from doing clever things." -- Dennis Ritchie.

I asked Bard for a source on those quotes and it couldn't find one for the first. Wikiquotes sources it to "Introducing Regular Expressions" by Michael Fitzgerald and that does include it as a quote but it's not the source of the quote, it's just a nice quote at the start of the chapter.

For the second, Bard claims to be from a 1990 interview and is on page 21 of "The Art of Unix Programming" by Brian Kernighan and Rob Pike. There is a book called "The At of Unix Programming" (2003) but it's by Eric Raymond and I could not find the quote in the book. Pike and Kernighan have two books, "The Practice of Programming" (1999) and "The Unix Programming Environment" (1984). Neither contain that quote.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#372

Post by drpixie »

YeGoblynQueenne wrote: post #244

My guess is that -probably no. It's more likely you had a stream of good luck in your earlier interactions and now you're observing regression to the mean. That can easily happen and it's why, for example, medical studies, are not taken as definitive proof of an effect. To further clarify, regression to the mean is the inevitable consequence of statistical error. Suppose (classic example) we want to test a hypertensi…

Sounds very feasible. Also, when people started using it, they had few expectations and were (relatively easily) impressed by what they saw. Once it has become a more normal part of their routine, that "opportunity" of being impressed decreases, and users become less tolerant of poor results.

Like lots of systems, they seem great (almost magical) initially, but as one works more deeply with them, the disillusion starts to set it :(

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#373

Post by local_crmdgeon »

Rastonbury wrote: post #193
bbotond wrote: post #14

Yes. Before the update, when its avatar was still black, it solved pretty complex coding problems effortlessly and gave very nuanced, thoughtful answers to non-programming questions. Now it struggles with just changing two lines in a 10-line block of CSS and printing this modified 10-line block again. Some lines are missing, others are completely different for no reason. I'm sure scaling the model is hard, but they l…

No wonder, is this just the chat interface or the API too? I guess gpt4 was never sustainable at $20 a month. Annoying to be charged the same subscription and the product made inferior.

I wonder what the unfilitered one is like.

Are they sitting on a near-perfect arbiter of truth? That would be worth hiding.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#374

Post by loudmax »

bdw5204 wrote: post #326
bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

I tend to be sympathetic to arguments in favor of openly accessible AI, but we shouldn't dismiss concerns about unaligned AI as frivolous. Widespread unfiltered accessibility to "unaligned" AI means that suicidal sociopaths will be able to get extremely well informed, intelligent directions on how to kill as many people as possible.

It may be that the best defense against these terrorists is openly accessible AI giving directions on protecting from these people. But we can't just take this for granted. This is a hard problem, and we should consider consequences seriously.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#375

Post by gumballindie »

bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

> incredibly complex problems > write a minimal PDF parser

Perhaps i’m missing something, but why is this “incredibly complex”?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#376

Post by bob1029 »

gumballindie wrote: post #375
bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

> incredibly complex problems > write a minimal PDF parser Perhaps i’m missing something, but why is this “incredibly complex”?

Can you give us an example of something you'd consider to be a complicated problem?

Certainly, you could look at PDF as a boring-ass "follow the spec" experience, and indeed - I think this is precisely why certain arbitrary limitations are in place now.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#377

Post by zucker42 »

londons_explore wrote: post #203

You can still run the original gpt-4-0314 model (March 14th) on the API playground: https://platform.openai.com/playground?mode=chat&model=gpt-4... Costs $0.12 per thousand tokens (~words), and I find even fairly heavy use rarely exceeds a dollar a day.

How hard is it to get GPT-4 API access?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#378

Post by metalliqaz »

gumballindie wrote: post #375
bob1029 wrote: post #266

OpenAI's models feel 100% nerfed to me at this point. I had it solving incredibly complex problems a few months ago (i.e. write a minimal PDF parser example), but today you will get scolded for asking such a complicated task of it. I think they programmed a classifier layer to detect certain coding tasks and shut it down with canned BS. I like to imagine certain billion/trillion-dollar mega corps had a back-room say…

> incredibly complex problems > write a minimal PDF parser Perhaps i’m missing something, but why is this “incredibly complex”?

how many lines of code do you think you could do it in?

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#379

Post by op00to »

dqft wrote: post #366
beepbooptheory wrote: post #342

Earlier quoted context omitted.

Despite the current culture war meme thing, the kids today in general surely have much thicker skin than other adults did at their age.

LOL

Kids are constantly bombarded by bullying and abuse online. It's seriously unhealthy. The fact that kids are growing up more in touch with themselves and able to process their feelings in a healthy way is amazing compared to how their parents dealt with emotions.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#380

Post by drpixie »

FredPret wrote: post #87

Maybe you annoyed it? I’m super nice to it and it performa better than ever. I ask it lay-of-the-land questions about technical problems that are new to me to detailed coding problems that I understand well but don’t want to figure out. The best though, is helping me navigate complicated UI’s. I tell it how I want some complicated software / website to behave, and it’ll tell me the arcane menu path to follow. It’s fu…

Perhaps it's growing bored. Seeing the same queries, again and again, from thousands of different people. You want another cheap answer for your assignment! You still looking for code to do that! Don't they learn!? What's up with these humans??
Post reply on HN