Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

251–260 of 314 posts

Re: Building Meta's GenAI infrastructure

#251

Post by [deleted] »

echelon wrote: post #66
__loam wrote: post #55

Earlier quoted context omitted.

"My paycheck depends on this technology destroying every field producing cultural artifacts"

Said the butter churner, cotton ginner, and petrol pumper. I work in film. I've shot dozens of them the old fashioned way. I've always hated how labor, time, and cost intensive they are to make. Despite instructions from the luminaries to "just pick up a camera", the entire process is stone age. The field is extremely inequitable, full of nepotism and "who you know". Almost every starry-eyed film student winds up doi…

[deleted]

Re: Building Meta's GenAI infrastructure

#252

Post by danielhanchen »

dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

Another way to compete with the big tech incumbents is instead of hardware, try maths and software hacks to level the playing field! Training models is still black magic, so making it faster on the software side can solve the capital cost issue somewhat!

Re: Building Meta's GenAI infrastructure

#253

Post by hackerlight »

dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

Foundation models != application layer. The question is whether the application layer's lunch will be eaten by better foundation models.

Re: Building Meta's GenAI infrastructure

#254

Post by teaearlgraycold »

andy99 wrote: post #101
danielhanchen wrote: post #27

float8 got a mention! x2 more FLOPs! Also xformers has 2:4 sparsity support now so another x2? Is Llama3 gonna use like float8 + 2:4 sparsity for the MLP, so 4x H100 float16 FLOPs? Pytorch has fp8 experimental support, whilst attention is still complex to do in float8 due to precision issues, so maybe attention is in float16, and RoPE / layernorms in float16 / float32, whilst everything else is float8?

Is there float8 support in any common CPU intrinsics? It sounds interesting but curious what will be the impact if any on CPU inference.

I’m curious if there’s a meaningful quality difference between float8 and some uint8 alternative (fixed precision or a look up table).

Re: Building Meta's GenAI infrastructure

#255

Post by throwaway2037 »

verticalscaler wrote: post #113
oblio wrote: post #108

Earlier quoted context omitted.

The thing is, this could be considered basic research, right? Basic research IS setting money on fire until (and if) that basic research turns into TCP/IP, Ethernet and the Internet.

I wish. Funnily enough Arpanet and all that Xerox stuff were like Where as I think this more appropriately can be considered the meta PR budget. They simply can't not spend it, would look bad for Wall Street. Have to keep up with the herd.

    > Funnily enough Arpanet and all that Xerox stuff were like
That doesn't say much. The industry was in utter infancy. How much do you think it cost to move Ethernet from 100Mbit/sec to 1GBbit/sec to 10GB to 100GB to 400GB to 800GB? At least one or two orders of magnitude.

How about the cost to build a fab for the Intel 8088 versus a fab that produces 5nm chips running @ 5GHz. Again, at least one or two orders of magnitude.

Re: Building Meta's GenAI infrastructure

#256

Post by throwaway2037 »

verticalscaler wrote: post #204
lotsofpulp wrote: post #163

Earlier quoted context omitted.

> If you're a trillion dollar company the market will insist you set such sums on fire just to be in the race for $current-hype. If they do it drives their market cap higher still and if they don't they risk being considered un-innovative and therefore doomed to irrelevancy and the market cap will spiral downwards. You don’t think earning increasing amounts of tens of billions of dollars in net income per year at som…

$1T Market Cap lets it be known it will invest $10B a year into $current-hype that will change everything . P/E loosens speculatively on sudden new unbounded potential, Market Cap $1.1T. Hype funded. PR as innovator cemented.

If you look at the R&D expenditure of Apple, it is mindboggling.

https://www.macrotrends.net/stocks/charts/AAPL/apple/researc...

Roughly 30B USD per year. And what are we getting? Slightly slimmer phones and 3500USD AR/VR headsets?

Re: Building Meta's GenAI infrastructure

#257

Post by toxik »

danielhanchen wrote: post #252
dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

Another way to compete with the big tech incumbents is instead of hardware, try maths and software hacks to level the playing field! Training models is still black magic, so making it faster on the software side can solve the capital cost issue somewhat!

This kind of research is also incredibly capital intensive. You have to pay some of the smartest people around to work in it.

Re: Building Meta's GenAI infrastructure

#258

Post by bennyelv »

brookst wrote: post #244
latchkey wrote: post #227

Earlier quoted context omitted.

I pity the fools that think AI is just another internet hype cycle.

I’m old enough to remember the proud, defiant declarations that the internet was just a hype cycle.

Well it was wasn’t it? There was a massive boom where loads of companies over promised what they would achieve, followed by a crash when everyone realised lots of them couldn’t, followed by stability for the smaller number that could.

It was the very definition of a hype cycle as far as I can see. Hype cycle doesn’t mean “useless and will go away”, you have the second upward curve and then productivity.

https://en.m.wikipedia.org/wiki/Gartner_hype_cycle

Re: Building Meta's GenAI infrastructure

#259

Post by krschultz »

oblio wrote: post #109
rthnbgrredf wrote: post #62

Earlier quoted context omitted.

Meta could build their own cloud offering. But it would take years to match the current existing offerings of AWS, Azure and GCP in terms of scale and wide range of cloud solutions.

The real question is: why aren't they? They had the infrastructure needed to seed a cloud offering 10 years ago. Heck, if Oracle managed to be in 5th (6th? 7th?) place, Facebook for sure could have been a top 5 contender, at least.

Because they make more money using their servers for their own products than they would renting them to other people. Meta has an operating margin of 41% AFTER they burn a ton on Reality Labs, while AWS has a 21% margin with more disciplined spending. Social media is a more profitable business than infrastructure.

Re: Building Meta's GenAI infrastructure

#260

Post by throwaway2037 »

verticalscaler wrote: post #204
lotsofpulp wrote: post #163

Earlier quoted context omitted.

> If you're a trillion dollar company the market will insist you set such sums on fire just to be in the race for $current-hype. If they do it drives their market cap higher still and if they don't they risk being considered un-innovative and therefore doomed to irrelevancy and the market cap will spiral downwards. You don’t think earning increasing amounts of tens of billions of dollars in net income per year at som…

$1T Market Cap lets it be known it will invest $10B a year into $current-hype that will change everything . P/E loosens speculatively on sudden new unbounded potential, Market Cap $1.1T. Hype funded. PR as innovator cemented.

> Market Cap $1.1T. Hype funded.

I'm confused. How does your stock price, which determines market cat, affect your cashflow to fund R&D? It does not.

Post reply on HN