Saying “hey don’t go down the path we are on, where we are making money and considered the best in the world.. it’s a dead end” rings pretty hollow.. like “don’t take our lunch please?” Might be a similar statement it feels..
[dead]
OpenAI’s CEO says the age of giant AI models is already over
21–30 of 525 posts
Re: OpenAI’s CEO says the age of giant AI models is already over
#22I'm no expert but doesn't the architecture of minigpt4 that's on the front page right now give some indication of what the future might look like?
eh, I haven't personally found a usecase for LLMs yet given the fact that you can't trust the output and it needs to be verified by a human (which might as well be just as time consuming/expensive as actually doing the task yourself)
I've enjoyed using it for very small automation tasks. For instance, it helped me write scripts to take all my audiobooks with poor recording quality, split them into 59-minute chunks, and upload them to Adobe's free audio enhancement site to vastly improve the listening experience.
Re: OpenAI’s CEO says the age of giant AI models is already over
#23Once you've trained on the internet and most published books (and more...) what else is there to do? You can't scale up massively anymore.
Video.
> YouTubers upload about 720,000 hours of fresh video content per day. Over 500 hours of video were uploaded to YouTube per minute in 2020, which equals 30,000 new video uploads per hour. Between 2014 and 2020, the number of video hours uploaded grew by about 40%.
Re: OpenAI’s CEO says the age of giant AI models is already over
#24An amusing thought I've had recently is whether LLMs are in the same league as the millions of monkeys at the keyboard, struggling to reproduce one of the complete works of William Shakespeare.
But I think not, since monkeys probably don't "improve" noticeably with time or input.
Re: OpenAI’s CEO says the age of giant AI models is already over
#25The most comforting AI news I have read this year.
If you are worried about AI, this shouldn't make you feel a ton better. GPT4 is just trained to predict the next word, a very simple but crude approach and look what it can do! Imagine when a dozen models are wired together and giving each other feedback with more clever training and algorithms on future faster hardware. It is still going to get wild
I worry that the hardware requirements are only going to accelerate the cloud-OS integration. Imagine a PC that's entirely unusable offline.
Re: OpenAI’s CEO says the age of giant AI models is already over
#26The most comforting AI news I have read this year.
[dead]
Re: OpenAI’s CEO says the age of giant AI models is already over
#27>"the company’s CEO, Sam Altman, says further progress will not come from making models bigger. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.”
So to reiterate, he is not saying that the age of giant AI models is over. Current top-of-the-line AI models are giant and likely will continue to be. However, there's not point in training models you can't actually run economically. Inference costs need to stay grounded which means practical model sizes have a limit. More effort is going to go into making models efficient to run even if it comes at the expense of making them less efficient to train.
Re: OpenAI’s CEO says the age of giant AI models is already over
#28[deleted]
Re: OpenAI’s CEO says the age of giant AI models is already over
#29The most comforting AI news I have read this year.
If you are worried about AI, this shouldn't make you feel a ton better. GPT4 is just trained to predict the next word, a very simple but crude approach and look what it can do! Imagine when a dozen models are wired together and giving each other feedback with more clever training and algorithms on future faster hardware. It is still going to get wild
> Imagine when a dozen models are wired together...
Wouldn't these models hallucinate more than normal, then?
Re: OpenAI’s CEO says the age of giant AI models is already over
#30Saying “hey don’t go down the path we are on, where we are making money and considered the best in the world.. it’s a dead end” rings pretty hollow.. like “don’t take our lunch please?” Might be a similar statement it feels..
It's a pretty sus argument for sure when they're scared to release even parameter size.
although the title is a bit misleading on what he was actually saying. still, there's a lot left to go in terms of scale. Even if it isn't parameter size(and there's still lots of room here too, it just won't be economical), contrary to popular belief, there's lots of data left to mine