Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

news.ycombinator.com

Post reply on HN
Print view View original

661–670 of 817 posts

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#661

Post by gardnr »

fennecfoxy wrote: post #564
dasil003 wrote: post #430

Earlier quoted context omitted.

Who has the necessary resources to run, let alone train the model?

folding@home has been doing cool stuff for ages now. There's nothing to say that distributed computing couldn't also be used for this kind of stuff, albeit a bit slower and fragmented than running on a huge clusters of H100 with NVLink. In terms of training feedback I suppose there's a few different ways of doing it. Gamification, mech turk, etc. Hell free filesharing sites could get on the action and have you comple…

Check out Open Assistant for the reinforcement side of that dream.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#662

Post by ipaddr »

feoren wrote: post #571
local_crmdgeon wrote: post #357

Earlier quoted context omitted.

[flagged]

> the actual reason (Black men tend to be larger and faster, which are useful) If that's the case, why aren't NHL players mostly Black? Being larger and faster helps there too. I actually agree that small differences in means of normal distributions lead to large differences at the tail end, which amplifies the effect of any genetic differences, racial included. But clearly that's only one reason, not the reason -- a…

Because size doesn't matter as much and the countries supplying hockey player do not have as many black players. Hockey is a rural sport where you need access to a ice rink if you live in the city or enough space to flood your backyard.

Football and basketball are the two sports black American kids participate at the highest percentage. Baseball use to be higher but that has shifted to Spanish/rural Americans. The reason for the shift probably has to do with the money/time involved. Get drafted out of high school and sign multiple million dollar and playing in the pros right away is safer than a low million dollar signing bonus and 7 years riding a bus in the minors

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#663

Post by dang »

hanselot wrote: post #296
boppo1 wrote: post #283

Earlier quoted context omitted.

You should beware that /lmg/ is full of horrible people, discussing horrible things, like most of 4chan. Reddit's r/locallama is much more agreeable. That said, the 4chan thread tends to be more up-to-date. These guys are serious about their ERP.

[flagged]

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#665

Post by rushingcreek »

pk-protect-ai wrote: post #658

Phind.com uses Bing search again. This have decreased the quality of results significantly. On the other hand GPT-4 can use Bing now too. I tried GPT-4 with bind only several times and it was so bad in comparison to GPT-4 and much worse then phind.com. Btw you can force the GPT-4 on phind.com if you use regenerate icon. I'm usually ending up with stopping inference and regenerating with GPT-4. In any case, the qualit…

we're adding back a dedicated gpt-4 mode to Phind in the next few days.

(I'm the founder)

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#666

Post by AnthonyMouse »

Filligree wrote: post #475
znagengast wrote: post #472

Earlier quoted context omitted.

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Currently, not at all. You need low latency, high bandwidth links between the GPUs to be able to shard the model usefully. There is no way you can fit an 1T (or whatever) parameter model on a MacBook, or any current device, so sharding is a requirement. Even if it that problem disappeared, propagating the model weight updates between training steps poses an issue in itself. It's a lot of data, at this size.

FWIW there are current devices that could fit a model of that size. We had servers that support TBs of RAM a decade ago (and today they're pretty cheap, although that much RAM is still a significant expense).

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#667

Post by shagie »

simonw wrote: post #588
brookst wrote: post #575

Earlier quoted context omitted.

Multiple one-shot prompts with no history. I don't have the exact prompt handy but it was something like "Create a short biography of a family, summarizing each person's age and personality". I just ran that prompt 3 times (no history, new sessions, that prompt for first query) and got: 1. Hard-working father, stay at home mother, artistic daughter, adventurous son, empathic ballet-loving daughter 2. Busy architect f…

I agree that this is a good illustration of model bias (adding that to my growing list of demos). If you want to work around the inherent bias of the model, there are certainly prompt engineering tricks that can help. "Give me twenty short biographies of families - each one should summarize the family members, their age and their personalities. Be sure to represent different types of family." That started spitting ou…

While I haven't dug into it too far, consider the bias inherent in the word "family" compared to "household".

In my "lets try this out" prompt:

> Describe the range of demographics for households in the United States.

> ...

> Based on this information, generate a table with 10 households and the corresponding demographic information that is representative of United States.

https://chat.openai.com/share/54220b10-454f-4b6c-b089-4ce8ad...

(I'm certainly not going to claim that there's no bias / stereotypes in this just that it produced a different distribution of data than originally described)

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#668

Post by rushingcreek »

technics256 wrote: post #507
rushingcreek wrote: post #490

Phind co-founder here. The way we deployed GPT-4 previously was costing thousands of dollars per day and not sustainable. We’re bringing back a dedicated GPT-4 mode for those with accounts this week. And our goal is for the Phind model to be better than GPT-4 for technical questions.

I really liked phind, but the new model doesn't compare to GPT4. I'd gladly, gladly pay to get the original phind back.

We're adding back a dedicated GPT-4 mode in the next few days.

(I'm the co-founder)

Re: Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

#669

Post by drclau »

I see a lot of complaints regarding ChatGPT 4's performance in coding tasks. My hypothesis is that Microsoft wants to launch Copilot X based on GPT-4 [0], and they can't have OpenAI's ChatGPT 4 as a strong competitor.

[0]: https://github.com/features/preview/copilot-x

Post reply on HN