Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

51–60 of 314 posts

Re: Building Meta's GenAI infrastructure

#51

Post by conradev »

Legend2440 wrote: post #36
CuriouslyC wrote: post #18

Earlier quoted context omitted.

In pretty much every interview, Yann has talked about how important that AI infrastructure is open and distributed for the good of humanity, and how he wouldn't work for a company that wasn't open. Since Mark doesn't have an AI product to cannibalize, it's in his interest to devalue the AI products of others ("salting the earth").

I don't see how they're devaluing other people's AI products.

Mistral makes comparable models to Facebook. Mistral charges money, Facebook does not. This negatively affect’s Mistral’s pricing power because a customer can get 70% of the performance they need for 0% of the cost.

The “0% of the cost” part is unique to software businesses because you can copy software so cheaply

Re: Building Meta's GenAI infrastructure

#52

Post by scubbo »

michaelt wrote: post #47
tayo42 wrote: post #34

Earlier quoted context omitted.

What does video not be in the future mean? In social media tiktok and reels are everywhere?

There are reports [1] that a bunch of companies like "College Humor" were convinced to switch to producing native video for facebook (instead of directing users to their own sites) on the basis of bullshit metrics from facebook, and had an extremely bad time as a result, with some companies going bankrupt. Something like counting an autoplaying video that ran for 3 seconds as a 'view' IIRC [1] https://twitter.com/ada…

Thankfully, Dropout (a spin-off of College Humor) is alive and well, and producing some of the best D&D Actual Play series as well as other non-D&D comedy shows. One of the entertainment services that I happily pay for because I want to support what they're doing.

Re: Building Meta's GenAI infrastructure

#54

Post by foobarian »

renegade-otter wrote: post #25
hendersoon wrote: post #21

350k H100 cards, around ten billion dollars just for the GPUs. Less if Nvidia gives a volume discount, which I imagine they do not.

It will be ironic if Meta sinks all this money into the new trend and finds out later that it has been a huge boondoggle, just as publishers followed Facebook's "guidance" on video being the future, subsequently gutting the talent pool and investing into video production and staff - only to find out it was all a total waste.

There is still hope then for cheap gaming GPUs some day soon! I have pretty much the last 10 years of flagship releases to catch up on...

Re: Building Meta's GenAI infrastructure

#55

Post by __loam »

echelon wrote: post #32
renegade-otter wrote: post #25

Earlier quoted context omitted.

It will be ironic if Meta sinks all this money into the new trend and finds out later that it has been a huge boondoggle, just as publishers followed Facebook's "guidance" on video being the future, subsequently gutting the talent pool and investing into video production and staff - only to find out it was all a total waste.

As a practitioner in the field, I can assure you this is not a boondoggle. Those GPUs are going to subsume the entire music, film, and gaming industries. And that's just to start.

"My paycheck depends on this technology destroying every field producing cultural artifacts"

Re: Building Meta's GenAI infrastructure

#56

Post by ajcp »

crakenzak wrote: post #40
aeyes wrote: post #20

Earlier quoted context omitted.

Isn't Google trying to do this with their TPUs?

I still, for the life of me, can't understand why Google doesn't just start selling their TPUs to everyone. Nvidia wouldn't be anywhere near their size if they only made H100s available through their DGX cloud, which is what Google is doing only making TPUs available through Google Cloud. Good hardware, good software support, and market is starving for performant competitors to the H100s (and soon B100s). Would sell…

And undercut what they'd like to use as a huge motivator in people moving to GCP? Not likely. Even if they wanted to they can't keep up with their own internal demand.

Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment.

Re: Building Meta's GenAI infrastructure

#57

Post by hx8 »

mjburgess wrote: post #38
math_dandy wrote: post #12

Earlier quoted context omitted.

A company moving away from Nvidia/CUDA while the field is developing so rapidly would result in that company falling behind. When (if) the rate of progress in the AI space slows, then perhaps the big players will have the breathing room to consider rethinking foundational components of their infrastructure. But even at that point, their massive investment in Nvidia will likely render this impractical. Nvidia decisive…

I'm more concerned to avoid nvidia (et al.) market domination, than chasing the top-edge of the genAI benefits sigmoid. This will prevent much broad-based innovation.

This space is so compeitive, even if Nvidia is asleep at the wheel a competitor will come and push them before too long. AMD has a history of noticing when their competitors are going soft and rapidly being compeitive.

Re: Building Meta's GenAI infrastructure

#58

Post by NBJack »

YetAnotherNick wrote: post #50
fuddle wrote: post #35

How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b

> $3.5b Which is a fourth of what they spent in VR/AR in a year. And Gen AI is something they could easily get more revenue as it has now become proven technology, and Meta could possibly leapfrog others because of the data moat.

What moat exactly? Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. I'm not even sure they can use historical data.

Meta certainly has an edge in engineer count, undoubtedly. But I'd say they really, really want the metaverse to succeed more to have their on walled garden (i.e. equivalent power of Apple and Google stores, etc.). There's a reason they gave a hard pass to a Google partnership.

Re: Building Meta's GenAI infrastructure

#59

Post by zone411 »

Meta is still playing catch-up. Might be hard to believe but according to Reuters they've been trying to run AI workloads mostly on CPUs until 2022 and they had to pull the plug on the first iteration of their AI chip.

https://www.reuters.com/technology/inside-metas-scramble-cat...

Re: Building Meta's GenAI infrastructure

#60

Post by lifeisstillgood »

DEDLINE wrote: post #16

I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors

OpenAI takes money from MSFT and buys Azure services

Anthropic takes Amazon money and buys AWS services (as do many robotics etc)

I am fairly sure it’s not illegal but it’s definitely low quality revenue

Post reply on HN