Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

61–70 of 314 posts

Re: Building Meta's GenAI infrastructure

#61

float8 got a mention! x2 more FLOPs! Also xformers has 2:4 sparsity support now so another x2? Is Llama3 gonna use like float8 + 2:4 sparsity for the MLP, so 4x H100 float16 FLOPs? Pytorch has fp8 experimental support, whilst attention is still complex to do in float8 due to precision issues, so maybe attention is in float16, and RoPE / layernorms in float16 / float32, whilst everything else is float8?

I was thinking why is this one guy on HN so deeply interested and discussing technical details from a minor remark. Then I clocked the name. Great work on Gemma bugs

Oh thanks :) I always like small details :)

Re: Building Meta's GenAI infrastructure

#62
post #16

I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads

Meta could build their own cloud offering. But it would take years to match the current existing offerings of AWS, Azure and GCP in terms of scale and wide range of cloud solutions.

Re: Building Meta's GenAI infrastructure

#63

Yann wants to be open and Mark seems happy to salt the earth.

I genuinely think one of the most plausible short-term dangers of AI is the creation of lifelike bots which will be absolutely indistinguishable from real humans in short-form online interaction.

Since people don't want to talk to algorithms, this would result in them shunning all social media, which is a huge danger to companies in the space.

Re: Building Meta's GenAI infrastructure

#64
post #34

Earlier quoted context omitted.

What does video not be in the future mean? In social media tiktok and reels are everywhere?

They are referring to Facebook/Meta’s 2015 “pivot to video”, speculating there may be a similar thing happening more recently with AI. https://en.wikipedia.org/wiki/Pivot_to_video

Interesting thanks!

Feels like in hind sight, maybe they were just to early to it.

Re: Building Meta's GenAI infrastructure

#65
post #56

Earlier quoted context omitted.

I still, for the life of me, can't understand why Google doesn't just start selling their TPUs to everyone. Nvidia wouldn't be anywhere near their size if they only made H100s available through their DGX cloud, which is what Google is doing only making TPUs available through Google Cloud. Good hardware, good software support, and market is starving for performant competitors to the H100s (and soon B100s). Would sell…

And undercut what they'd like to use as a huge motivator in people moving to GCP? Not likely. Even if they wanted to they can't keep up with their own internal demand. Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment.

>Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment.

Once you go out of your heavily curated hardware stack, the headaches multiply exponentially.

Re: Building Meta's GenAI infrastructure

#66
post #55
post #32

Earlier quoted context omitted.

As a practitioner in the field, I can assure you this is not a boondoggle. Those GPUs are going to subsume the entire music, film, and gaming industries. And that's just to start.

"My paycheck depends on this technology destroying every field producing cultural artifacts"

Said the butter churner, cotton ginner, and petrol pumper.

I work in film. I've shot dozens of them the old fashioned way. I've always hated how labor, time, and cost intensive they are to make.

Despite instructions from the luminaries to "just pick up a camera", the entire process is stone age. The field is extremely inequitable, full of nepotism and "who you know". Almost every starry-eyed film student winds up doing drudge work for the rest of their lives. Most will never make a feature to match their ambition.

If the whole task was to simply convey my thoughts and dreams to others, why am I scrambling around to sign location rights, capture photons on expensive glass, and then smear and splice things together for months on end? This is ceremonial and soon to be anachronistic. I'm glad that whole mess is going to be replaced. It's a farce.

To phrase it another way - would you like to be hand-writing assembly on punch cards? To only gain entrance into the field with your mathematics PhD?

To speak of the liberty and the economics, why should I have to sell the rights to my idea to a studio so I can get it off the ground? Why should I have to obey the studio's rules and mind their interference?

This whole Gen AI thing is going to be the biggest liberating moment for filmmaking creatives. I know, because I am one.

And if you think any Jack or Jill can just come in and text prompt a whole movie, you're crazy. It's still hard work and a metric ton of good taste.

Art will never die. It's the human soul. It'll take more than some tech bros with GPUs to kill it.

AI is just another tool for the artist. A "bicycle for the mind" to quote Jobs, and a rocket ship for the imagination to convey my own direct experience.

Re: Building Meta's GenAI infrastructure

#69
post #26

> At Meta, we handle hundreds of trillions of AI model executions per day Such a large number, makes sense?

How many ads does Meta serve a day, and how many AI model executions are done for each one? Repeat the same for stories, post and comment recommendations on Facebook and Instagram, and you have very big numbers. To that, Add VR, internal modeling and other backoffice/ offline analyses over billions of users and you'll easily get into the trillions.
Post reply on HN