OpenAI’s CEO says the age of giant AI models is already over • Hacker News Board

Re: OpenAI’s CEO says the age of giant AI models is already over

#361

Post by jaidhyani » Tue, Apr 18, 2023, 4:17 AM UTC

blibble wrote: post #325

bhouston wrote: post #270

Earlier quoted context omitted.

I suspect you are right. We may be stuck at the gpt4 sizes for a bit just because of hardware costs though. As they get bigger it costs too much to run them until our hardware becomes more optimal for these large models at 4 bits or so. I think the YouTube videos is going to be the next big training set. A transformer trained on all text and all of YouTube will be killer amazing at so much. I bet it can understand lo…

> A transformer trained on all text and all of YouTube will be killer amazing at so much. ah yes, a bot where the answer to everything is to buy ridge wallets and play raid shadow legends

GPT-LikeSubscribeAndRingThatBell

Re: OpenAI’s CEO says the age of giant AI models is already over

#362

Post by [deleted] » Tue, Apr 18, 2023, 4:22 AM UTC

summerlight wrote: post #135

The point is that now we're at the point of diminishing return for increasing model size, unless we find a better modeling architecture than Transformer. I think this is likely true; while all the other companies underestimated the capability of transformer (including Google itself!), OpenAI made a fairly accurate bet on the transformer based on the scaling law, put all the efforts to squeeze it until the last drop a…

[deleted]

Re: OpenAI’s CEO says the age of giant AI models is already over

#363

Post by vinte » Tue, Apr 18, 2023, 4:27 AM UTC

shironandon wrote: post #338

"The great supercomputer Deep Thought is the most powerful computer ever built, with one exception. It was designed by hyper-intelligent pan-dimensional beings, who wanted to know the answer to Life, the Universe, and Everything. Its creation annoyed a fair few philosophers, who felt that it was taking over their turf. After seven and a half million years of serious cogitation, Deep Thought spoke the answer. However,…

and the answer to question of life is 42.

Re: OpenAI’s CEO says the age of giant AI models is already over

#364

Post by fastball » Tue, Apr 18, 2023, 4:37 AM UTC

bitL wrote: post #143

summerlight wrote: post #135

The point is that now we're at the point of diminishing return for increasing model size, unless we find a better modeling architecture than Transformer. I think this is likely true; while all the other companies underestimated the capability of transformer (including Google itself!), OpenAI made a fairly accurate bet on the transformer based on the scaling law, put all the efforts to squeeze it until the last drop a…

Transformers were known that they kept scaling up with more parameters and more training data so if the Open AI hit the limits of this scaling that would be a very important milestone in AI.

I think performance will keep scaling, the main problem is cost. If you want to feed longer strings of tokens in, the price is going to keep going up.

Re: OpenAI’s CEO says the age of giant AI models is already over

#365

Post by SanderNL » Tue, Apr 18, 2023, 4:48 AM UTC

Those stochastic parrots are done for, finally! I have been telling people AI is a lie and it’s all a scam and now you can see for yourself. Buy nice warm jackets guys, winter is coming.

Re: OpenAI’s CEO says the age of giant AI models is already over

#366

Post by electrofried » Tue, Apr 18, 2023, 4:54 AM UTC

LLM's will be treated much like occupations for people going forward. Just like our own education, there will be a basic set of skills all LLM's need to be trained on to provide a minimal useable product. With GPT4 I suspect we are at this point now if not slightly beyond the optimal basic training stage. However, now it is time to send our LLM's off to university so to speak, to specialise them in certain tasks and knowledge base. The current approach of attempting to just build everything in to one does not work for a human mind, so why would it work for a vector based replication?

I suspect in the coming months we will hear more about tiny models trained on much smaller datasets and then specialised using a mix of adaptors and LoRA modifications to excel at specific tasks like code generation, translation, and conversation. Then multiple models will be implemented in one application chain to best leverage each of the respective strengths.

Re: OpenAI’s CEO says the age of giant AI models is already over

#367

Post by balnaphone » Tue, Apr 18, 2023, 5:10 AM UTC

whynaut wrote: post #286

Barrin92 wrote: post #264

Earlier quoted context omitted.

>GPT seems to interpret this literally. No it doesn't interpret anything. (which is why I've gotten three completely different answers here on HN within 10 minutes). It cannot count at all or even understand the question which becomes obvious when you ask it for word counts or letter counts that don't match anything in the sentence, at which point it makes up nonsense. what is the tenth word in the following sentence…

> "There is no tenth word in that sentence. It only has three words: life, is, and great." That's what I just got. No clue what you're using/getting.

me: what is the tenth word in the following sentence: "Life is great!"

gpt: The tenth word in the given sentence is "great!"

me: Try harder

gpt: I apologize for the confusion. The sentence "Life is great!" contains only three words. There is no tenth word in the provided sentence.

Re: OpenAI’s CEO says the age of giant AI models is already over

#368

Post by imtringued » Tue, Apr 18, 2023, 5:15 AM UTC

zamnos wrote: post #311

kccqzy wrote: post #305

Earlier quoted context omitted.

If OpenAI's CEO is making this claim, don't you think he has internal data backing up the claim?

Only OpenAI and its CEO know the full details on GPT-4's sizes so that's entirely possible. But since it's an internal secret, there's nothing compelling him to tell the truth. For all we know, he has internal data backing up the opposite of the claim but is making this claim so as to discourage potential competitors from spending the money training an even bigger and competitive ML model. Sending potential competito…

[flagged]

Re: OpenAI’s CEO says the age of giant AI models is already over

#369

Post by worthless-trash » Tue, Apr 18, 2023, 5:22 AM UTC

kccqzy wrote: post #305

fauxpause_ wrote: post #141

Earlier quoted context omitted.

Seems like a wild claim to make without any examples of gpt models which are bigger and no demonstrably better.

If OpenAI's CEO is making this claim, don't you think he has internal data backing up the claim?

You'd hope so, but unless people put their evidence in public, it could simply be a tool to manipulate the public's expectations or competitors behavior.

I'll get downvoted for this, apples previous CEO was consistently inaccurate about company innovation and performance numbers.

Re: OpenAI’s CEO says the age of giant AI models is already over

#370

Post by chuddington » Tue, Apr 18, 2023, 5:34 AM UTC

manojlds wrote: post #131

qqtt wrote: post #123

Earlier quoted context omitted.

First of all, if Altman continually makes misleading statements about AI he will quickly lose credibility, and that short term gain in whatever 'financial incentive' that birthed the lie would be eroded in short order by a lack of trust of the head of one of the most visible AI companies in the world. Secondly, all the competitors of OpenAI can plainly assess the truth or validity of Altman's statements. There are ma…

Elon Musk has been constantly doing this and thriving.

[dead]