Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

411–420 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#411

Post by pembrook »

ziml77 wrote: post #299
DebtDeflation wrote: post #213

Earlier quoted context omitted.

Even worse is all of the "GPT Influencers" and their "Here's what ChatGPT predicts will be the price of Bitcoin/some stock/houses will be this time next year" clickbait. It's a language model, people.

Please tell me that you're making this up about there being GPT "influencers".

Have you been to Twitter in the last 6 months? It's basically a GPT/Midjourney/Stable Diffusion hype generator. Everybody there is now an expert on this topic. And you can be too!

Step 1: Tweet fake troll screenshots of GPT output or make corny threads like "90% of people are using AI wrong, here's..."

Step 2: Let the "For you" algo take hold

Step 3: Profit

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#412

Post by layer8 »

bob1029 wrote: post #376
gumballindie wrote: post #375

Earlier quoted context omitted.

> incredibly complex problems > write a minimal PDF parser Perhaps i’m missing something, but why is this “incredibly complex”?

Can you give us an example of something you'd consider to be a complicated problem? Certainly, you could look at PDF as a boring-ass "follow the spec" experience, and indeed - I think this is precisely why certain arbitrary limitations are in place now.

> you could look at PDF as a boring-ass "follow the spec" experience

If only… the problem is that the spec is underspecified.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#413

Post by gumballindie »

andruby wrote: post #407
gumballindie wrote: post #385

Earlier quoted context omitted.

I dont know - it’s a genuine question. I honestly didnt expect this to be a complex problem, let alone incredibly complex. I genuinely want to understand where the challenge lies.

If you want a datapoint, Origami is a "pure Ruby library to parse, modify and generate PDF documents". That library cloc's in at 13,683 lines of code and 3,295 lines of comments.

Thats not a lot of code tho, but i see your point.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#414

Post by fallingknife »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

[flagged]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#415

Post by malfist »

lovich wrote: post #387
hanselot wrote: post #296

Earlier quoted context omitted.

[flagged]

Well, you certainly have the personality type I’d expect from someone who frequents 4chan Edit:oof, checked his profile like he suggested and in just two pages of history he’s engaged in climate change denial and covid vaccine conspiracies. Buddy, I don’t think using 4chan to “train my brain’s spam filter” is working too well for you. You’ve got a mind so open your brain fell out.

I mean, he's literally saying the jews are responsible for bot farms and "offensive content".

That's not the stance of someone rational.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#416

Post by layer8 »

gumballindie wrote: post #385
metalliqaz wrote: post #378

Earlier quoted context omitted.

how many lines of code do you think you could do it in?

I dont know - it’s a genuine question. I honestly didnt expect this to be a complex problem, let alone incredibly complex. I genuinely want to understand where the challenge lies.

The PDF spec is of byzantine complexity, and is full of loose ends where things aren’t fully and unambiguously specified. It also relies on various other specs (e.g. font formats), not to mention Adobe’s proprietary extensions.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#417

Post by philwelch »

ithkuil wrote: post #365
brookst wrote: post #318

Earlier quoted context omitted.

I’m tired of the “it’s not racist if aggregate statistics support my racism” thing. Racism, like other isms, means a belief that a person’s characteristics define their identity. It doesn’t matter if confounding factors mean that you can show that people of their race are associated with bad behaviors or low scores or whatever. I used GPT3.5 to generate 100 short descriptions of families for a project. Every single o…

> No amount of “bias = pattern recognition” nonsense can justify a system that has (had? this was a while ago and I have not retested) such extreme biases One possible explanation is that when you ask for 100 example families the task is parsed as "pick the most likely family composition and add a bit of randomness" and "repeat the aforementioned task" 100 times. If phrased like that it would be surprising to find on…

> One possible explanation is that when you ask for 100 example families the task is parsed as "pick the most likely family composition and add a bit of randomness" and "repeat the aforementioned task" 100 times.

Yes, this is consistent with my ChatGPT experience. I repeatedly asked it to tell me a story and it just sort of reiterated the same basic story formula over and over again. I’m sure it would go with a different formula in a new session but it got stuck in a rut pretty quickly.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#418

Post by yalue »

gumballindie wrote: post #389
bob1029 wrote: post #376

Earlier quoted context omitted.

Can you give us an example of something you'd consider to be a complicated problem? Certainly, you could look at PDF as a boring-ass "follow the spec" experience, and indeed - I think this is precisely why certain arbitrary limitations are in place now.

I honestly have no clue about what makes pdf parsing a complex task. I wasnt trying to sound condescending. Would be great to know what makes this so difficult, considering the pdf file format is ubiquitous.

I'm not anyone involved in this thread (so far), but I've written a minimal PDF parser in the past using something between 1500-2000 lines of Go. (Sadly, it was for work so I can't go back and check.) Granted, this was only for the bare-bones parsing of the top-level structures, and notably did not handle postscript, so it wouldn't be nearly enough to render graphics. Despite this, it was tricky because it turns out that "following the spec" is not always clear when it comes to PDFs.

For example, I recall the spec being unclear as to whether a newline character was required after a certain element (though I don't remember which element). I processed a corpus containing thousands of PDFs to try to determine what was done in practice, and I found that about half of them included the newline and half did not---an emblematic issue where an unclear official "spec" meant falling back to the de facto specification: flexbility.

It's honestly a great example of something a GPT-like system could probably handle. Doable in a single source file if necessary, fewer than 5k lines, and can be broken into subtasks if need be.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#419

Post by haolez »

chaxor wrote: post #124

The reason it's worse is basically because it's more 'safe' (not racist, etc). That of course sounds insane, and doesn't mean that safety shouldn't be strived for, etc - but there's an explanation as to how this occurs. It occurs because the system essentially does a latent classification of problems into 'acceptable' or 'not acceptable' to respond to. When this is done, a decent amount of information is lost regardi…

I think it's more likely that they nerfed it due to scaling pains.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#420

Post by greenie_beans »

xrd wrote: post #238

Reading all the comments here, seems like being able to run your own models is vital. If not, you are subject to a service where the capabilities are changing underneath you constantly and without notice.

this is what i've been trying to say too
Post reply on HN