Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

181–190 of 314 posts

Re: Building Meta's GenAI infrastructure

#181

Post by [deleted] »

miohtama wrote: post #169
lotsofpulp wrote: post #164

Earlier quoted context omitted.

How is it free equity? Spending money to invest it somewhere involves risks. You might recover some of it if the investment is valued by others, but there is no guarantee.

You do not need cash in hands to invest. Instead, you print your own money (AWS credit) and use that to drive up the valuation, because this money costs you nothing today. It might cost tomorrow though, when the company starts to use your services. However depending the deal structure they might not use all the credit, go belly up before credit is used or bought up by someone with real cash.

[deleted]

Re: Building Meta's GenAI infrastructure

#182

Post by virtuallynathan »

virtuallynathan wrote: post #126
lifeisstillgood wrote: post #60

Earlier quoted context omitted.

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors OpenAI takes money from MSFT and buys Azure services Anthropic takes Amazon money and buys AWS services (as do many robotics etc) I am fairly sure it’s not illegal but it…

Facebook has more datacenter space and power than Amazon, Google, and Microsoft -- possibly more than Amazon and Microsoft combined...

To date, facebook has built, or is building, 47,100,000 sq ft of space, totaling nearly $24bn in investment. Based on available/disclosed power numbers and extrapolating per sqft, I get something like 4770MW.

Last I updated my spreadsheet in 2019, Google had $17bn in investments across their datacenters, totaling 13,260,000 sq ft of datacenter space. Additional buildings have been built since then, but not to the scale of an additional 30mil sq ft.

Amazon operates ~80 datacenter buildings in Northern Virginia, each ~200,000 sq ft -- about 16,000,000sq ft total in that region, the other regions are much much smaller, perhaps another 4 mil sq ft. When I'm bored I'll go update all my maps and spreadsheets.

Re: Building Meta's GenAI infrastructure

#183

Post by negus »

logicchains wrote: post #135
KaiserPro wrote: post #117

Earlier quoted context omitted.

because meta sucks at software, documentation and making sure end user products work in a supported way. Offering reliable IaaS is super hard and capital intensive. Its also not profitable if you are perceived as shit.

>because meta sucks at software Google started a cloud and their user-facing software is atrocious. Compared e.g. Angular to React, Tensorflow to Pytorch.

Why would you prefer Pytorch to Tensorflow/Keras?

Re: Building Meta's GenAI infrastructure

#184

Post by CuriouslyC »

chasd00 wrote: post #45
CuriouslyC wrote: post #18

Earlier quoted context omitted.

In pretty much every interview, Yann has talked about how important that AI infrastructure is open and distributed for the good of humanity, and how he wouldn't work for a company that wasn't open. Since Mark doesn't have an AI product to cannibalize, it's in his interest to devalue the AI products of others ("salting the earth").

is "salting the earth", in the biblical sense of destroying your enemy and their land to the point where not even plants grow again, a SV term used for companies that promote open source?

It's a term used for making a certain type of business unviable. In this case, high quality open models will make closed source models less viable, since the closed source model providers won't be able to charge monopoly prices for their models, but will have to approach the price of cloud GPU time or lose customers to equally capable open models.

Re: Building Meta's GenAI infrastructure

#186

Post by dekhn »

aeyes wrote: post #170
dekhn wrote: post #150

Earlier quoted context omitted.

Do you mean, sell TPU hardware to other companies that would run it in their data centers? I can't imagine that would ever really work. The only reason TPUs work at Google is because they have huge teams across many different areas to keep them running (SRE, hardware repair, SWE, hardware infra) and it's coupled to the design of the data centers. To vend and externalize the software would require google to setup simi…

> Google used to sell a search appliance-in-a-box and eventually lost interest because hardware is so high-touch. We had a GSA for intranet search and other than the paint this was a standard Dell server. I remember not being impressed by what the GSA could do. We also had Google Urchin for web analytics, it wasn't a hardware appliance but the product wasn't very impressive either. They then killed that and tried to…

The server may have been dell, but it included a full stack of google3 software including chubby the lockserver.

We had one at my company and it was widely loved- far better intranet search and domain-specific search for biotech.

Re: Building Meta's GenAI infrastructure

#187

Post by claytonjy »

KaiserPro wrote: post #128
jvanderbot wrote: post #110

So, I'd love to work on optimizing pipelines like this. How does one "get into" it? It seems a ML scientist with some C/C++ and infra knowledge just dips down into the system when required? Or is it CUDA/SIMD experts who move "up" into ML?

A lot of the optimisation at this level is getting data into the right place at the right time, without killing the network. Its also a group effort to provide simple to use primitives that "normal" ML people can use, even if they've never used hyper scale clusters before. So you need a good scheduler, that understand dependencies (no, the k8s scheduler(s) are shit for this, plus it wont scale past 1k nodes without e…

can you say more about the network issues with thousands of k8s nodes? I'm regularly running 2-3000 nodes in a GKE cluster, majority have GPUs, is this something I need to be worrying about?

Re: Building Meta's GenAI infrastructure

#188

Post by jvalencia »

islewis wrote: post #8

I know we won't get it this from FB, but I'd be really interested to see how the relationship of compute power to engineering hours scales. They mention custom building as much as they can. If FB magically has the option to 10x the compute power, would they need to re-engineer the whole stack? What about 100x? Is each of these re-writes just a re-write, or is it a whole order of magnitude more complex? My technical u…

The cost of training quickly outpaces the cost of development as context length increases. So hardware is cheap until it isn't anymore, by orders of magnitude.

Re: Building Meta's GenAI infrastructure

#189

Post by samstave »

jvalencia wrote: post #188
islewis wrote: post #8

I know we won't get it this from FB, but I'd be really interested to see how the relationship of compute power to engineering hours scales. They mention custom building as much as they can. If FB magically has the option to 10x the compute power, would they need to re-engineer the whole stack? What about 100x? Is each of these re-writes just a re-write, or is it a whole order of magnitude more complex? My technical u…

The cost of training quickly outpaces the cost of development as context length increases. So hardware is cheap until it isn't anymore, by orders of magnitude.

But there is still significant cost in the physical buildouts of new pods/DCs, whatever and the human engineering hours to physically build, even though its a mix of resources across the vendors and FB? - it still would be interesting to know man hours into the physical build of the HW.

Re: Building Meta's GenAI infrastructure

#190

Post by VirusNewbie »

virtuallynathan wrote: post #182
virtuallynathan wrote: post #126

Earlier quoted context omitted.

Facebook has more datacenter space and power than Amazon, Google, and Microsoft -- possibly more than Amazon and Microsoft combined...

To date, facebook has built, or is building, 47,100,000 sq ft of space, totaling nearly $24bn in investment. Based on available/disclosed power numbers and extrapolating per sqft, I get something like 4770MW. Last I updated my spreadsheet in 2019, Google had $17bn in investments across their datacenters, totaling 13,260,000 sq ft of datacenter space. Additional buildings have been built since then, but not to the sca…

But Google built data centers aren't the only data centers google is running their machine fleet in...
Post reply on HN