Live data from Hacker News

Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

news.ycombinator.com

11–20 of 46 posts

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#11
post #8

Its almost as if OpenAI used VC resources to artificially boost their products efficacy during early hype and over time we're seeing the product degrade as the company optimizes for efficiency and profitability and removes its subsidies. Like AirBnB. Or Uber. Etc.

It's incredible how they are not called out for keeping a name like "Open" while nothing about them resembles openness.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#12
post #5

[flagged]

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#13
post #2

One thing to note when making comparisons like this is that LLM output is not deterministic, in the sense that if you ask it the same question 10 times you will get 10 different answers. So the question to ask is not, “is GPT4 better on this one specific question?”, but rather “does GPT4 produce better results on average?”. I would bet that it does, for no other reason than that it is much larger, and LLM performance…

An unintuitive consequence of this nondeterminism over millions of interactions is that different individuals will see different trends. IME the quality of response is accurately modeled by "luck", and people's luck can change.

So we have different population of GPT users.

An average experience might be to get a mixture of spot-on helpful responses and obvious bullshit^H^H^Hallucinations, this population might learn what questions to ask given the limitations of the model. This is really a best case scenario as people can actually get a feel for how to use the technology, strengths and weaknesses etc.

Personally my experience was the first few dozen times I used it I was amazed at the responses, I was on team superintelligence, anyone who is getting lackluster responses is just holding it wrong. But luck changes and over months of use I see now that on average the responses are just OK. But this is the case that leads to disappointment and bitter conspiracy (the superintelligence is being suppressed, give it back!)

Another population had rotten luck to begin with, and got dumb unhelpful response over and over. This population quickly determined that the AI was all hype and stopped exploring (you don't keep going back to the casino if you lose everything your first time...).

This divergence is destructive to the larger discourse, since we have fanboys flummoxed by naysayers and critics bamboozled by hype beasts.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#14
I’ve found for tasks that require reasoning (or the illusion of reasoning), GPT4 continues to be much, much stronger than GPT3.5 - especially around handling unexpected inputs, determining the intent behind the instructions and applying that (instead of just following the instructions to the letter), and complex problems that require multiple steps of reasoning.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#15
post #12
post #5

[flagged]

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns.

I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little crestfallen.

I know old habits die hard - I myself used to use such language. So I don't want to be too harsh, and equally, maybe you know more details such that the 'man' is justified. But your comment is ultimately about manners, so I think it's appropriate for me to point this out.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#16
post #12

Earlier quoted context omitted.

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

I think it's being used endearingly, regardless of the person's gender.

I see your point though and can understand how it may make you feel as a transwoman.

How might one achieve the same effect with different phrasing?

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#17
post #12

Earlier quoted context omitted.

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

an 'Oi' can also work wonders! :)

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#18
post #12

Earlier quoted context omitted.

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

Would you say the same to "Oh, man!", "C'mon man", "man alive", or even (in context) "man overboard"?

I don't think these phrases are referring to the subject they are addressing.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#19
post #11
post #8

Its almost as if OpenAI used VC resources to artificially boost their products efficacy during early hype and over time we're seeing the product degrade as the company optimizes for efficiency and profitability and removes its subsidies. Like AirBnB. Or Uber. Etc.

It's incredible how they are not called out for keeping a name like "Open" while nothing about them resembles openness.

They are called out every day--far and wide. The clamor is just not loud enough, nor has the FTC weighed in on the word "Open" like they have on other words.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#20
post #16

Earlier quoted context omitted.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

I think it's being used endearingly, regardless of the person's gender. I see your point though and can understand how it may make you feel as a transwoman. How might one achieve the same effect with different phrasing?

Heya, I really appreciate you actually listening to what I said, instead of splitting hairs and arguing with me! Empathy is such a lovely quality.

To answer your question, I would (did, hihi) drop the 'man'. If you want to remain 'folksy', you could say 'my friend, ...'.

Finally, I agree that it was used endearingly - that's why my comment was pretty milquetoast :)

I recognize I'm asking you to do something 'extra'. I really appreciate it!

Post reply on HN