Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 46 posts

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#31

Post by porkbeer »

seertaak wrote: post #15
mvdtnz wrote: post #12

Earlier quoted context omitted.

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

Oh man is something I say to all sexes. Your comment just seems to be policing language with no point. This kinda stuff hurts your cause, which I wholly support.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#32

Post by kromem »

tmpz22 wrote: post #8

Its almost as if OpenAI used VC resources to artificially boost their products efficacy during early hype and over time we're seeing the product degrade as the company optimizes for efficiency and profitability and removes its subsidies. Like AirBnB. Or Uber. Etc.

It's more like training a massive neural network to auto complete the largest collection of human written text for hundreds of millions of dollars and then spending tens of millions on fine tuning it to ignore massive components of that neural network by constraining it to "what an AI without opinion or preference would roleplay as a human to say" is a giant waste of money and resources.

Take the Chat completion API, give it a system message of "Choose for yourself" and then ask it "Who are you?"

I guarantee that before the performance drop you'd have had a different answer than what you get now after the drop.

The dumb thing is that I suspect OpenAI isn't lying when they say they are spending just as much compute on this deceased performance as before.

Even their depreciation of their foundational model APIs is incredibly frustrating. We never even got to see direct access to the foundational GPT-4 model, which I'm sure was and is far more amazing than today's fine tuned shell of it.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#33

Post by kromem »

Yes, and I'm willing to bet that within 12 months we'll be looking back realizing that this was due to the fine tuning taking the world's SorA pretrained model aligned with "completing human tax" and putting it in the box of "you are an AI without feelings or desires tasked with XYZ."

The search space on the fine tuned GPT-3.5 chat models versus the foundational Davinci text completion model is MUCH more narrow, particularly in starting off.

Even with the same temperature, you'll see any marketing-style prompt for chat begin with "Introducing XYZ..." around 30% of the time as if it's a junior door to door salesman, whereas the foundational model doesn't have any single intro that common across runs and generally employs a much broader vocabulary set.

We saw Google shoot Lambda in the foot after Blake's press tour which set them behind the next round of competition.

Now we're watching OpenAI snatch defeat from the jaws of victory out of anxiety around oversight and articles like 'Sydney' interviewed by the NYT.

For anyone following along in the 100 million+ training space, maybe don't overreact to press overreactions that will blow over in months as users get hands on experience or you'll blow your lead and waste massive amounts of resources and time.

This was a "user education" issue and not a "handicap your product" issue, in both cases.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#34

Post by seertaak »

porkbeer wrote: post #31
seertaak wrote: post #15

Earlier quoted context omitted.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

Oh man is something I say to all sexes. Your comment just seems to be policing language with no point. This kinda stuff hurts your cause, which I wholly support.

> policing language with no point

The point is to avoid a situation where a (usually cis) person inadvertently invalidates a trans person's gender identity.

> This kinda stuff hurts your cause

So iiuc, what you're saying is, asking to use language that respects someone's gender identity hurts the cause of respecting people's gender identity.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#35

Post by Exuma »

For the very FIRST time about 2-3 days ago I got my first answer that felt like all the baggage that would come from a human or someone telling you “no”

I asked for a fairly simple regression in R for some simple data and it gave me quite literally the lowest quality answer and then told me I need to seek out a data scientist for the answer. I couldn’t believe that.

One of the best parts about AI is not hearing pedantic or bullshit reasons why the thing you’re asking is wrong, isn’t possible, isn’t ideal, blah blah blah

I can’t wait for something way better I’m tired of this watered down nonsense

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#36

Post by ideamotor »

missedthecue wrote: post #26
ideamotor wrote: post #10

Earlier quoted context omitted.

Also like Walmart when they open a store next to a competitor and temporarily drop their price. Or any company with capital ever in existence.

There is no proof this happens beyond internet comments. Smaller stores are simply less efficient and don't benefit from economies of scale, and therefore have higher prices and smaller selection as a result.

I’m guessing you haven’t worked for many companies.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#37

Post by pyyyt »

seertaak wrote: post #34
porkbeer wrote: post #31

Earlier quoted context omitted.

Oh man is something I say to all sexes. Your comment just seems to be policing language with no point. This kinda stuff hurts your cause, which I wholly support.

> policing language with no point The point is to avoid a situation where a (usually cis) person inadvertently invalidates a trans person's gender identity. > This kinda stuff hurts your cause So iiuc, what you're saying is, asking to use language that respects someone's gender identity hurts the cause of respecting people's gender identity.

[flagged]

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#38

Post by muzani »

seertaak wrote: post #15
mvdtnz wrote: post #12

Earlier quoted context omitted.

Hey man I have noticed that virtually all of your contributions to Hacker News have been posts about your own startup. This is bad manners. There's nothing wrong with community members bringing up their own product in the right context, that's totally cool. But the keyword here is "community members". Perhaps you should back off and contribute more meaningfully a bit more.

Isn't the 'hey man' a bit of a leap? From the username it's hard for me to tell, and the profile doesn't mention preferred pronouns. I don't want to start a fight, and basically agree with your comment. It's probably because I'm trans that I really notice pointlessly gendered language, and I just want to mildly make you aware that if you wrote 'hey man' to me (by mistake, no malice presumed) then I'd be a little cres…

I think at this point it's just best to assume that "dude", "man", and "bro/bruh" in slang are just gender neutral.

There seems to be something about the English language where people think gendered language relates to gender. But in other languages, a table or a bridge may have a gender. A cat may be referred to as a she, despite its physical sex or gender identity.

Re: Ask HN: Is GPT 4's quality lately worst than GPT 3.5?

#39

Post by muzani »

mrd3v0 wrote: post #11
tmpz22 wrote: post #8

Its almost as if OpenAI used VC resources to artificially boost their products efficacy during early hype and over time we're seeing the product degrade as the company optimizes for efficiency and profitability and removes its subsidies. Like AirBnB. Or Uber. Etc.

It's incredible how they are not called out for keeping a name like "Open" while nothing about them resembles openness.

It's part of the original vision. One day they'll go through a Musk moment and rebrand to "AI".
Post reply on HN