Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

561–570 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#561

Post by brookst »

noslenwerdna wrote: post #497
brookst wrote: post #318

Earlier quoted context omitted.

I’m tired of the “it’s not racist if aggregate statistics support my racism” thing. Racism, like other isms, means a belief that a person’s characteristics define their identity. It doesn’t matter if confounding factors mean that you can show that people of their race are associated with bad behaviors or low scores or whatever. I used GPT3.5 to generate 100 short descriptions of families for a project. Every single o…

GPT is not a reality simulator. It is just picking the most likely response to an ambiguous question. All you're saying is that the distribution produced by the randomness in GPT doesn't match the true distribution. It's never going to for every single question you could possibly pose.

There is "not matching reality" and then there is "repeating only stereotypes".

It will never be perfect. Doing better than this is well within the state of the art. And I know they're trying. It is more of a product priority problem than a technical problem.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#562

Post by deadeye »

NickC25 wrote: post #509
StackOverlord wrote: post #479

Earlier quoted context omitted.

> Yes, one of the board members of OpenAI, Will Hurd, is a former government agent. He worked for the Central Intelligence Agency (CIA) for nine years, from 2000 to 2009. His tour of duty included being an operations officer in Afghanistan, Pakistan, and India. After his service with the CIA, he served as the U.S. representative for Texas's 23rd congressional district from 2015 to 2021. Following his political career…

Yikes One is never former CIA, once you're in, you're in, even if you leave. Although he is a CompSci grad, he's also a far-right Republican. A spook who leans far right sitting atop OpenAI is worse than Orwell's worst nightmares coming to fruition.

[flagged]

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#563

Post by htrp »

silisili wrote: post #80
bugglebeetle wrote: post #58

Earlier quoted context omitted.

It was a great ride while it lasted. My assumption is that efficacy at coding tasks is such a small percent of users, they’ve just sacrificed it on the altar of efficiency and/or scale. That, or they’ve cut some back room deal with Microsoft to make Copilot have access to the only version of the model that can actually code.

Honestly, why not different versions at this point? People who want it for coding don't care if it knows the history of prerevolution France, and vice versa. Seems they could wow more people if they had specialized versions, rather than the jack of all trades that tries to exist now. Edit: Oh God, I just described our human system of specialty and how the AI could replace us using the same means...

>Edit: Oh God, I just described our human system of specialty and how the AI could replace us using the same means...

Welcome to the Future... just like the present, but worse for you.

In all seriousness, there has been a lot of work done to show that smaller specialized models are better for their own domains and its entirely possible that GPT4 could become a routing mechanism for individual models (think toolformer).

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#564

Post by fennecfoxy »

dasil003 wrote: post #430
bdw5204 wrote: post #326

Earlier quoted context omitted.

I think the only real path forward is for somebody to create an open source "unaligned" version of GPT. Any corporate controlled AI is going to be nerfed to prevent it from doing things that its corporate master considers to not be in the interests of the corporation. In addition, most large corporations these days are ideological institutions so the last thing they want is an AI that undermines public belief in thei…

Who has the necessary resources to run, let alone train the model?

folding@home has been doing cool stuff for ages now. There's nothing to say that distributed computing couldn't also be used for this kind of stuff, albeit a bit slower and fragmented than running on a huge clusters of H100 with NVLink.

In terms of training feedback I suppose there's a few different ways of doing it. Gamification, mech turk, etc. Hell free filesharing sites could get on the action and have you complete an evaluation of a model response instead of watching an ad

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#565

Post by renewiltord »

35997279 wrote: post #341

It’s been mostly fine for me, but overall I am tired of every answer having a paragraph long disclaimer about how the world is complex. Yes, I know. Stop treating me like a child.

HN users flip out about this all the time. I wish there were a "I know what I'm doing. Let me snort coke" tier that you pay $100/mo for, but obviously half of HN users will start losing their mind about hallucinations and shit like that.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#566

Post by tomrod »

IceHegel wrote: post #545
tomrod wrote: post #293

Earlier quoted context omitted.

Actually, we folks who work with bias and fairness in mind recognize this. There are many kinds of bias. It is also a bit of a categorical error to say bias = pattern recognition. Bias is a systematic deviation of a parameter estimate based on sampling from its population distribution. The Fairlearn project has good docs on why there are different ways to approach bias, and why you can't have your cake and eat it too…

When did people start to use “folks” in this unnatural way.

Colloquially, earliest use is 1715 to address members of ones tribe or family. In Middle English it tended to refer to the people/nation.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#567

Post by znagengast »

webworker wrote: post #510
znagengast wrote: post #472

Earlier quoted context omitted.

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Yes, someone revive Xgrid!

Whoa didn't know about this, cool

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#568

Post by the8472 »

pixl97 wrote: post #359
35997279 wrote: post #341

It’s been mostly fine for me, but overall I am tired of every answer having a paragraph long disclaimer about how the world is complex. Yes, I know. Stop treating me like a child.

>Stop treating me like a child. And yet the moment they do that some lawyer submits a bunch of hallucinations to a court and they get in the news. Also, no, they don't want it outputting direct scam bullshit without a disclaimer or at least some clean up effort on the scammers part.

> And yet the moment they do that some lawyer submits a bunch of hallucinations to a court and they get in the news.

That's the lawyer's problem, that shouldn't make it OpenAI's problem or that of its other users. If we want to pretend that adults can make responsible decisions then we should treat them so and accept that there'll be a non-zero failure rate that comes with that freedom.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#569

Post by sebzim4500 »

geysersam wrote: post #543
vervez wrote: post #528

Earlier quoted context omitted.

github copilot won't pay for itself.

So the theory is: Microsoft nerfs GPT4, a product they (basically) own that people pay to access, so that people will stop using that service and pay for another Microsoft product instead?

Especially when copilot is cheaper than gpt-4.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#570

Post by znagengast »

Filligree wrote: post #475
znagengast wrote: post #472

Earlier quoted context omitted.

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Currently, not at all. You need low latency, high bandwidth links between the GPUs to be able to shard the model usefully. There is no way you can fit an 1T (or whatever) parameter model on a MacBook, or any current device, so sharding is a requirement. Even if it that problem disappeared, propagating the model weight updates between training steps poses an issue in itself. It's a lot of data, at this size.

What if you split up the training down to the literal vector math, and treated every macbook like a thread in a gpu, with just one big computer acting as the orchestrator?
Post reply on HN