Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Meta trains large language models at scale

engineering.fb.com

Post reply on HN
Print view View original

21–30 of 213 posts

Re: How Meta trains large language models at scale

#21

Post by candiddevmike »

dinobones wrote: post #10
mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

Do you really think Google’s hardware expertise is better than Nvidia’s? If needed these other companies have the $$$ to buy the best chips money can buy from Nvidia. Better chips than Google could ever produce. If anything, this is why IMO Google will fail.

I thought NVIDIA's moat was mostly software/CUDA?

Re: How Meta trains large language models at scale

#22

Post by stygiansonic »

zdyn5 wrote: post #6
mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

H100s are far from consumer video cards

Yeah, ops comment makes it seem like they are building racks of RTX 4090s, when this isn’t remotely true. Tensor Core performance is far different on the data center class devices vs consumer ones.

Re: How Meta trains large language models at scale

#23

Post by HarHarVeryFunny »

mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

The chips are somewhat irrelevant. It's the overall system architecture, management, and fault recovery that matters.

Re: How Meta trains large language models at scale

#24

Post by mike_d »

dinobones wrote: post #10
mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

Do you really think Google’s hardware expertise is better than Nvidia’s? If needed these other companies have the $$$ to buy the best chips money can buy from Nvidia. Better chips than Google could ever produce. If anything, this is why IMO Google will fail.

Yes. Google was building custom HPC hardware 5-8 years before Nvidia decided to expand outside the consumer and "workstation" markets.

Re: How Meta trains large language models at scale

#26

Post by matt-p »

mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

Can't agree. This is like saying $popularApp will fail because they buy expensive hosting at AWS.

Rubbish they will fail because the product didn't fit the market, if they're successful they'll have money to buy servers and colo then drive down cost. If they succeed it will be in large part due to the fact they spent thier capital and more importantly time on code/engineers rather than servers.

Right now companies are searching for a use of AI that will add hundreds of billions to thier market cap. Once they find that they can make TPUs, right now only one thing matters; getting there first.

Re: How Meta trains large language models at scale

#27

Post by throwaway920102 »

mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

What can stop Google is building the wrong thing, or being so scared to launch anything that they smother their own fledgling products before they are born or before they can mature. Their product and finance and accounting teams should be tossed.

Re: How Meta trains large language models at scale

#28

Post by matt-p »

candiddevmike wrote: post #19
throwaway_ab wrote: post #15

Earlier quoted context omitted.

I think it's likely Nvidia's GPU's, many of which are $50,000+ for a single unit, far surpass Google's custom silicon otherwise why wouldn't Google be selling shovels like Nvidia? If Google had a better chip, or even a chip that was close, they would sell it to anyone and everyone. From a quick search I can see Google's custom chips are 15x to 30x slower to train AI compared to Nvidia's current latest gen AI specific…

They do sell shovels, you can get Google TPUs on Google Cloud.

Exactly and they are still about 1/18ths as good at training llms as a H100.

Maybe they are less than 1/18ths the cost, so google technically have a marginally better unit cost but i doubt it when you consider the R&D cost. They are less bad at inference, but still much worse than even an A100.

Re: How Meta trains large language models at scale

#29

Post by throwaway_ab »

candiddevmike wrote: post #19
throwaway_ab wrote: post #15

Earlier quoted context omitted.

I think it's likely Nvidia's GPU's, many of which are $50,000+ for a single unit, far surpass Google's custom silicon otherwise why wouldn't Google be selling shovels like Nvidia? If Google had a better chip, or even a chip that was close, they would sell it to anyone and everyone. From a quick search I can see Google's custom chips are 15x to 30x slower to train AI compared to Nvidia's current latest gen AI specific…

They do sell shovels, you can get Google TPUs on Google Cloud.

Wouldn't that be renting a shovel vs selling a shovel?

Re: How Meta trains large language models at scale

#30

Post by jauer »

tw04 wrote: post #17
mike_d wrote: post #4

Posts like this underscore why the smart money is betting on Google as the long term AI winner. Meta, Microsoft, OpenAI, etc. are trying to address problems with consumer video cards and spending billions to try and out bid each other to win Nvidia's favor - while Google is on their 6th generation of custom silicon. Literally the only thing that can stop Google now is the fact they keep bringing Microsoft and Oracle…

Except Microsoft is making their own chips as well? https://www.theverge.com/2023/11/15/23960345/microsoft-cpu-g...

and so is Meta: https://ai.meta.com/blog/next-generation-meta-training-infer...
Post reply on HN