Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s CEO says the age of giant AI models is already over

wired.com

Post reply on HN
Print view View original

131–140 of 525 posts

Re: OpenAI’s CEO says the age of giant AI models is already over

#131

Post by manojlds »

qqtt wrote: post #123
boringuser2 wrote: post #78

Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.

First of all, if Altman continually makes misleading statements about AI he will quickly lose credibility, and that short term gain in whatever 'financial incentive' that birthed the lie would be eroded in short order by a lack of trust of the head of one of the most visible AI companies in the world. Secondly, all the competitors of OpenAI can plainly assess the truth or validity of Altman's statements. There are ma…

Elon Musk has been constantly doing this and thriving.

Re: OpenAI’s CEO says the age of giant AI models is already over

#133

Post by cowmix »

olalonde wrote: post #129
boringuser2 wrote: post #78

Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.

Does he even have any background in machine learning? I always found it bizarre that he was chosen to be OpenAI's CEO...

On the Lex Fridman podcast, he pretty much admitted he's not an AI (per se) and isn't the most excited about the tech (as he could be).

Re: OpenAI’s CEO says the age of giant AI models is already over

#134

Post by g_delgado14 »

boringuser2 wrote: post #78

Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.

IIRC Altman has no financial stake in the success or failure of OpenAI to prevent these sorts of conflicts of interests between OpenAI and society as a whole

Re: OpenAI’s CEO says the age of giant AI models is already over

#135

Post by summerlight »

The point is that now we're at the point of diminishing return for increasing model size, unless we find a better modeling architecture than Transformer.

I think this is likely true; while all the other companies underestimated the capability of transformer (including Google itself!), OpenAI made a fairly accurate bet on the transformer based on the scaling law, put all the efforts to squeeze it until the last drop and took all the rewards.

It's likely that GPT-4 is on the optimal spot between cost and performance and there won't be significant improvements on performance in a near future. I guess the next task would be more on efficiency, which has a significant implication on its productionization.

Re: OpenAI’s CEO says the age of giant AI models is already over

#136

Post by calderknight »

I didn't think this article was very good. Sam Altman actually implied that GPT-5 will be developed when he spoke at MIT. And if Sam said that scaling is over (I doubt he said this but I could be wrong) the interesting part would be the reasoning he provided for this statement - no mention of that in the article.

Re: OpenAI’s CEO says the age of giant AI models is already over

#137

Post by smeagull »

famouswaffles wrote: post #55
hcks wrote: post #46

Earlier quoted context omitted.

Yes, but it also tells us that if Altman is honest here, then he doesn’t believe GPT-like models can scale to near level human performances (because even if the cost of compute was 10x or even 100x it would still be economically sound).

No it doesn't. For one thing they're already at human performance. For another, i don't think you realize how expensive inference can get. Microsoft with no scant amount of available compute is struggling to run gpt-4 such that they're rationing it between subsidiaries while they try to jack up compute. So saying, it would be economically sound if it cost x10 or x100 what it costs now is a joke.

This tells me you haven't really stress tested the model. GPT is currently at the stage of "person who is at the meeting, but not really paying attention so you have to call them out". Once GPT is pushed, it scrambles and falls over for most applications. The failure modes range from contradicting itself, making up things for applications that shouldn't allow it, to ignoring prompts, to simply being unable to perform tasks at all.

Re: OpenAI’s CEO says the age of giant AI models is already over

#138

Post by dauertewigkeit »

spaceman_2020 wrote: post #90
thunderbird120 wrote: post #27

>"the company’s CEO, Sam Altman, says further progress will not come from making models bigger. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.” So to reiterate, he is not saying that the age of giant AI models is over. Current top-of-the-line AI models are giant and likely…

Is cost really that much of a burden? Intelligence is the single most expensive resource on the planet. Hundreds of individuals have to be born, nurtured, and educated before you might get an exceptional 135+ IQ individual. Every intelligent person is produced at a great societal cost. If you can reduce the cost of replicating a 135 IQ, or heck, even a 115 IQ person to a few thousand dollars, you're beating biology b…

Are intelligent people that valuable? There's lots of them at every university working for peanuts. They don't seem to be that valued by society, honestly.

Re: OpenAI’s CEO says the age of giant AI models is already over

#139

Post by marketerinland »

thewataccount wrote: post #101
sitkack wrote: post #88

Earlier quoted context omitted.

5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.

The problem is that it would take us the same amount of time to just add a form with django. Plus you have to handle failure cases, etc. And yeah I agree this would be a great use-case, and isn't that expensive. I'd like to do this in lots of places, and the problem is I have to convince my boss to pay for something that otherwise would have been free. The conversation would be "We have to add these fields to our mod…

There are excellent commercial AI resume parsers already - Affinda.com being one. Not expensive and takes minutes to implement.

Re: OpenAI’s CEO says the age of giant AI models is already over

#140

Post by dragonwriter »

smeagull wrote: post #137
famouswaffles wrote: post #55

Earlier quoted context omitted.

No it doesn't. For one thing they're already at human performance. For another, i don't think you realize how expensive inference can get. Microsoft with no scant amount of available compute is struggling to run gpt-4 such that they're rationing it between subsidiaries while they try to jack up compute. So saying, it would be economically sound if it cost x10 or x100 what it costs now is a joke.

This tells me you haven't really stress tested the model. GPT is currently at the stage of "person who is at the meeting, but not really paying attention so you have to call them out". Once GPT is pushed, it scrambles and falls over for most applications. The failure modes range from contradicting itself, making up things for applications that shouldn't allow it, to ignoring prompts, to simply being unable to perform…

Are we talking about bare GPT through the UI, or GPT with a framework giving it access to external systems and the ability to store and retrieve data?

Because, yeah, “brain in a jar” GPT isn’t enough for most tasks beyond parlor-trick chat, but being used as a brain in a jar isn’t the point.

Post reply on HN