Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

201–210 of 314 posts

Re: Building Meta's GenAI infrastructure

#201

Post by renegade-otter »

dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

Not everything has to be AI. You can run a small business infra for MUCH less than you did back then, especially if you adjust for inflation (!).

Training AI models costs a fortune, but so far it's been just front-loading costs in hopes of a windfall. We'll see what actually happens.

Re: Building Meta's GenAI infrastructure

#202

Post by gajjanag »

jvanderbot wrote: post #110

So, I'd love to work on optimizing pipelines like this. How does one "get into" it? It seems a ML scientist with some C/C++ and infra knowledge just dips down into the system when required? Or is it CUDA/SIMD experts who move "up" into ML?

Our group works on some of this stuff at Meta, and we have a pretty good diversity of backgrounds - high performance computing (the bulk), computer systems, compilers, ML engineers, etc. We are hiring.

Feel free to DM me to learn more.

Re: Building Meta's GenAI infrastructure

#203

Post by to11mtm »

mewpmewp2 wrote: post #199
andy99 wrote: post #195

Earlier quoted context omitted.

So far it's been pretty "democratic" - I feel in no way disadvantaged because I can't train a foundation model myself. Actually the ecosystem is a lot better than 25 years ago - there are open source (or source available) versions of basically everything you'd want to participate in modern AI/ML.

But none of those are remotely as good as GPT4 for example.

Mixtral?

Re: Building Meta's GenAI infrastructure

#204

Post by verticalscaler »

lotsofpulp wrote: post #163
verticalscaler wrote: post #105

Earlier quoted context omitted.

Wait till you find out how much they spent on VR. It is a real loophole in the economy. If you're a trillion dollar company the market will insist you set such sums on fire just to be in the race for $current-hype. If they do it drives their market cap higher still and if they don't they risk being considered un-innovative and therefore doomed to irrelevancy and the market cap will spiral downwards. Sort of reminds m…

> If you're a trillion dollar company the market will insist you set such sums on fire just to be in the race for $current-hype. If they do it drives their market cap higher still and if they don't they risk being considered un-innovative and therefore doomed to irrelevancy and the market cap will spiral downwards. You don’t think earning increasing amounts of tens of billions of dollars in net income per year at som…

$1T Market Cap lets it be known it will invest $10B a year into $current-hype that will change everything. P/E loosens speculatively on sudden new unbounded potential, Market Cap $1.1T. Hype funded. PR as innovator cemented.

Re: Building Meta's GenAI infrastructure

#205

Post by usiaiekamm »

dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

They are also so far profitless (unless you are nvidia) and useless. The last gasp of an industry on its last legs.

Re: Building Meta's GenAI infrastructure

#207

Post by Thaxll »

oblio wrote: post #109
rthnbgrredf wrote: post #62

Earlier quoted context omitted.

Meta could build their own cloud offering. But it would take years to match the current existing offerings of AWS, Azure and GCP in terms of scale and wide range of cloud solutions.

The real question is: why aren't they? They had the infrastructure needed to seed a cloud offering 10 years ago. Heck, if Oracle managed to be in 5th (6th? 7th?) place, Facebook for sure could have been a top 5 contender, at least.

Because it's not their business, they're not good at it and probably the ROI is not worth it.

Also how exactly they would do it, they don't have enough infra for renting, they would need to x10 what they have now.

Re: Building Meta's GenAI infrastructure

#208

Post by agar »

NBJack wrote: post #58
YetAnotherNick wrote: post #50

Earlier quoted context omitted.

> $3.5b Which is a fourth of what they spent in VR/AR in a year. And Gen AI is something they could easily get more revenue as it has now become proven technology, and Meta could possibly leapfrog others because of the data moat.

What moat exactly? Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. I'm not even sure they can use historical data. Meta certainly has an edge in engineer count, undoubtedly. But I'd say they really, really want the metaverse to succeed more to have their on walled garden (i.e. equivalent power of Apple and Google stores, etc.). T…

> There's a reason they gave a hard pass to a Google partnership.

AIUI, Google required Meta to basically cede control of a partnered OS to them:

"After years of not focusing on VR or doing anything to support our work in the space, Google has been pitching AndroidXR to partners and suggesting, incredibly, that WE are the ones threatening to fragment the ecosystem when they are the ones who plan to do exactly that.

"We would love to partner with them. They could bring their apps to Quest today! They could bring the Play store (with its current economics for 2d apps) and add value to all their developers immediately, which is exactly the kind of open app ecosystem we want to see. We would be thrilled to have them. It would be a win for their developers and all consumers and we’ll keep pushing for it.

"Instead, they want us to agree to restrictive terms that require us to give up our freedom to innovate and build better experiences for people and developers—we’ve seen this play out before and we think we can do better this time around."

-- From Mark Bosworth

Re: Building Meta's GenAI infrastructure

#210

Post by chatmasta »

virtuallynathan wrote: post #177
jedberg wrote: post #139

Earlier quoted context omitted.

Unless you've worked at Amazon, Microsoft, Google, and Facebook, or a whole bunch of datacenter providers, I'm not sure how you could make that claim. They don't really share that information freely, even in their stock reports. Heck I worked at Amazon and even then I couldn't tell you the total datacenter space, they don't even share it internally.

You can just map them all... I have. I also worked at AWS :)

This would be an interesting dataset to use for trading decisions (or sell to hedge funds).

But I wonder how much of their infrastructure is publicly mappable, compared to just the part of it that's exposed to the edge. (Can you map some internal instances in a VPC?)

That said, I'm sure there are a lot of side channels in the provisioning APIs, certificate logs, and other metadata that could paint a decently accurate picture of cloud sizes. It might not cover everything but it'd be good enough to track and measure a gradual expansion of capacity.

Post reply on HN