Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

pcgamer.com

Post reply on HN
Print view View original

51–60 of 67 posts

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#51

Post by nojvek »

nirav72 wrote: post #39
A_D_E_P_T wrote: post #2

As Scott Alexander at ACX recently said: "AGE OF MIRACLES AND WONDERS: We seem to be in the beginning of a slow takeoff. We should expect things to get very strange for however many years we have left before the singularity. So far the takeoff really is glacially slow (everyone talking about the blindingly fast pace of AI advances is anchored to different alternatives than I am) which just means more time to gawk at…

When I read words like singularity or AGI..is when I know we’re at peak hype and correction is coming. I just hope its not another AI winter. Because I’d still like to see this area continue to grow.

The advances every week in papers in models, more TFLOPs per dollar per GPU, larger datasets, more VC billions being invested, more humans tinkering.

It’s definitely going to be wild next decade.

AI superiority is now a nation race like being first to moon, first nuclear bomb.

Whoever cracks it gets many trillions added to their GDP.

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#52

Post by ETH_start »

ninethirty wrote: post #45
epups wrote: post #14

There is not enough written text to scale a model like ChatGPT at even 10x its current parameters, so improvements will have to come from optimizations and/or longer training (at diminishing returns).

GPT-4 Will Have 100 Trillion Parameters — 500x the Size of GPT-3

That's likely a massive over-estimation. I would bet it's around 1 trillion.

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#53

Post by KeplerBoy »

zirgs wrote: post #47
zamadatix wrote: post #34

Earlier quoted context omitted.

The models are extremely large to ship around, sync, and load on consumer hardware over the internet. Maybe there are better ways to build and train a model for a distributed use case though.

The model that's used by Stable Diffusion is only around ~4 GB in size.

which would make traditional distributed training prohibitively slow.

You would have to sync the weights with residential internet speeds (10s of MB/s) instead of PCI-e Speeds (10s of GB/s).

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#54

Post by machiaweliczny »

shsbdncudx wrote: post #32
abudabi123 wrote: post #5

Could Ai learn from 3D mapping daytime soap operas and trace character development along the narrative arc? Then it has to function in the real world.

I imagine it could. Something for it to contend with though is that our published media, soap operas, reality tv, so and so, don’t really reflect what we humans are like in real life. They’re caricatures of us.

But instagram/tiktok has real data + you messages, people you know and talk to and listens to you 24/7. If they will be able to predict 1B of humans and social dynamics then I guess it’s enough of training data to basically simulate reality.

EDIT: it gives insane “soft” power so pretty sure NSA etc. are working on this

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#55

Post by imtringued »

brhsagain wrote: post #29

I have a probably really stupid question but I'm going to ask it because I really don't know the answer. What's stopping someone from starting a distributed effort to train an open source ChatGPT model, like Folding@home but for AI? Is it a technical limitation with how models are trained, difficulty of coordination, something else?

That is actually irrelevant. I couldn't try out flan-t5-xl but the model flan-t5-model is like an inferior GPT but it is opensource. I don't necessarily think that the size of the model is that significant. What makes ChatGPT so impressive is that you don't have to mess around with the settings vs Flan T5.

More effort has to be put into obtaining good training data than just throwing compute at the problem.

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#58

Post by tim333 »

nirav72 wrote: post #39
A_D_E_P_T wrote: post #2

As Scott Alexander at ACX recently said: "AGE OF MIRACLES AND WONDERS: We seem to be in the beginning of a slow takeoff. We should expect things to get very strange for however many years we have left before the singularity. So far the takeoff really is glacially slow (everyone talking about the blindingly fast pace of AI advances is anchored to different alternatives than I am) which just means more time to gawk at…

When I read words like singularity or AGI..is when I know we’re at peak hype and correction is coming. I just hope its not another AI winter. Because I’d still like to see this area continue to grow.

It's hype for decades until it happens and then not so much. (https://waitbutwhy.com/wp-content/uploads/2015/01/Intelligen...)

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#59

Post by yucky »

zamadatix wrote: post #34
brhsagain wrote: post #29

I have a probably really stupid question but I'm going to ask it because I really don't know the answer. What's stopping someone from starting a distributed effort to train an open source ChatGPT model, like Folding@home but for AI? Is it a technical limitation with how models are trained, difficulty of coordination, something else?

The models are extremely large to ship around, sync, and load on consumer hardware over the internet. Maybe there are better ways to build and train a model for a distributed use case though.

Sounds like we have our use-case for blockchain!

Re: Nvidia predicts AI models 1,000,000X more powerful than ChatGPT within 10 years

#60

Post by tim333 »

rhtgrg wrote: post #8
weinzierl wrote: post #7

Not saying it could no be true, but that's exactly what a company like Nvidia would say. Do we have any reasonably unbiased info on what the boundaries of the current approach are and when we will reach them?

"More powerful" here just means larger (another commenter says faster chips, but you don't even need that if you spend the next ten years training a model, although that could be a more accurate representation of what Nvidia is trying to say, since the alternative is a no-brainer). AI isn't really "smart," and it seems unlikely we're going to get there anytime soon (in other words, the "singularity" is still firmly o…

>In my opinion we haven't made a lick of qualitative progress in recent years in the "IQ" of AI

We've gone from not very good to beats all humans at chess and go, and passing exams in medicine and law, all of which are things that I can't do myself. When AI can do all things better than a human would that qualify as a lick of progress?

Post reply on HN