Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

131–140 of 314 posts

Re: Building Meta's GenAI infrastructure

#131
post #123

The link mentions "our internal job scheduler" and how they had to optimize it for this work -- does anyone know what this job scheduler is called, or how it works?

it might be twine: https://www.usenix.org/system/files/osdi20-tang.pdf

but I suspect its not that, because Twine is optimised for services rather than batch processing, and doesn't really have the concept of priorities.

Re: Building Meta's GenAI infrastructure

#132
post #87

Earlier quoted context omitted.

> And if you think any Jack or Jill can just come in and text prompt a whole movie, you're crazy. It's still hard work and a metric ton of good taste. If you want anything good , yes. If you just want something … I reckon it'd take a week to assemble an incomprehensible-nonsense-film pipeline, after which it's just a matter of feeding the computer electricity. Short-term, this is going to funnel resources away from t…

> If you want anything good, yes. If you just want something ... You don't even need AI for that. https://en.wikipedia.org/wiki/YouTube_poop https://en.wikipedia.org/wiki/Skibidi_Toilet The idea that AI isn't going to be used as a creative tool too and that it won't lead to more and better art is a defeatist, Luddite attitude. Similarly shaped people thought that digital cameras would ruin cinema and photography. > S…

Many YouTube Poops are artistic expression (e.g. https://redirect.invidious.io/watch?v=dO4eIEvHjSw). Skibidi Toilet is definitely artistic expression: it's a full-on epic. (Reactions from one ≈50-year-old: “baffling” “how did they do that?” “why would anyone make this?”)

If you think the Luddites were defeatist, you don't know much about the Luddites.

> On the contrary - every budding film student will soon [1] be able to execute on their entire visions straight out of the gates. […] Creatives have more ways of building brands and followings for themselves than ever before.

Yet, we have no shortage of starving artists. Will AI provide them food and shelter?

This is unequivocally a win for creative expression for hobbyists, but it stands to harm professionals – at least in the short term, perhaps longer-term. It's not happening in a vacuum: the greedy are revoking livelihoods because they think AI can do it faster and cheaper (laundering appropriated hobbyist and increasingly-cheap professional labour).

> The studio system. Not the industry.

Huh, the word 'industry' has a specialised meaning in economics. Didn't know that.

Re: Building Meta's GenAI infrastructure

#133
post #16

I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors OpenAI takes money from MSFT and buys Azure services Anthropic takes Amazon money and buys AWS services (as do many robotics etc) I am fairly sure it’s not illegal but it…

NVidia also invests in their AI customers.

Re: Building Meta's GenAI infrastructure

#134

Total cluster they say will reach 350k H100, which at $30k street price is about $10b. In contrast, Microsoft is spending over $10b per quarter capex on cloud. That makes Zuck look conservative after his big loss on metaverse. https://www.datacenterdynamics.com/en/news/q3-2023-cloud-res...

the biggest cost at meta is infra.

> In contrast, Microsoft is spending over $10b per quarter capex on cloud.

to service other people's work load. Its a different business.

Re: Building Meta's GenAI infrastructure

#135
post #109

Earlier quoted context omitted.

The real question is: why aren't they? They had the infrastructure needed to seed a cloud offering 10 years ago. Heck, if Oracle managed to be in 5th (6th? 7th?) place, Facebook for sure could have been a top 5 contender, at least.

because meta sucks at software, documentation and making sure end user products work in a supported way. Offering reliable IaaS is super hard and capital intensive. Its also not profitable if you are perceived as shit.

>because meta sucks at software

Google started a cloud and their user-facing software is atrocious. Compared e.g. Angular to React, Tensorflow to Pytorch.

Re: Building Meta's GenAI infrastructure

#136
post #19

I'd be great if they could invest in an alternative to nvidia -- then, in one fell swoop, destroy the moats of everyone in the industry.

Except that "one fell swoop" would realistically be 20+ years of research and development from the top minds in the semiconductor industry.

It's not the hardware keeping NVidia ahead, it's the software. Hardware-wise AMD is competitive with NVidia, but their lack of a competitive CUDA alternative is hurting adoption.

Re: Building Meta's GenAI infrastructure

#138
post #16

I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads

I think Meta have avoided doing this because it would complicate their business priorities. They don’t really do B2B.

What do you mean by “they don’t do B2B”? They sell ads to companies, don’t they?

Re: Building Meta's GenAI infrastructure

#139

Earlier quoted context omitted.

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors OpenAI takes money from MSFT and buys Azure services Anthropic takes Amazon money and buys AWS services (as do many robotics etc) I am fairly sure it’s not illegal but it…

Facebook has more datacenter space and power than Amazon, Google, and Microsoft -- possibly more than Amazon and Microsoft combined...

Unless you've worked at Amazon, Microsoft, Google, and Facebook, or a whole bunch of datacenter providers, I'm not sure how you could make that claim. They don't really share that information freely, even in their stock reports.

Heck I worked at Amazon and even then I couldn't tell you the total datacenter space, they don't even share it internally.

Re: Building Meta's GenAI infrastructure

#140
post #16

I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors OpenAI takes money from MSFT and buys Azure services Anthropic takes Amazon money and buys AWS services (as do many robotics etc) I am fairly sure it’s not illegal but it…

Neither did AWS when they started. They were just building out data centers to run their little book website and decided to start selling the excess capacity. Meta could absolutely do the same, but in the short term, I think they find using that capacity more valuable than selling it.
Post reply on HN