Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

121–130 of 314 posts

Re: Building Meta's GenAI infrastructure

#121

Post by mrkramer »

BonoboIO wrote: post #99
mrkramer wrote: post #96

"Share this: Hacker News" Noice

I thought at first "what are you talking about", when i check my uBlock filters. Was blocking the whole "Share this" content section. Sharing on Hacker News ... they now their audience.

I also use uBlock but my filters are the default ones and I saw it without any problem but tbh this is the first time that I saw some post on the Web have HN as a share option or the first time that I was surprised seeing it. Maybe it has something to do with Google ranking "trusted human information and knowledge" higher than "non-human" information and knowledge[0] or simply some Meta software engineer loves and uses HN so s/he decided to include HN as well, idk.

[0] https://news.ycombinator.com/item?id=39423949

Re: Building Meta's GenAI infrastructure

#122

Post by echelon »

renegade-otter wrote: post #114
echelon wrote: post #87

Earlier quoted context omitted.

> If you want anything good, yes. If you just want something ... You don't even need AI for that. https://en.wikipedia.org/wiki/YouTube_poop https://en.wikipedia.org/wiki/Skibidi_Toilet The idea that AI isn't going to be used as a creative tool too and that it won't lead to more and better art is a defeatist, Luddite attitude. Similarly shaped people thought that digital cameras would ruin cinema and photography. > S…

> Similarly shaped people thought that digital cameras would ruin cinema and photography. Obviously, but you seem to be arguing that AI is just another evolution of productivity tools. You still need to have a photographer's eye while using this technology. If you couldn't make a good composition on film, a digicam will not save you, and it definitely did not replace photographers. Perhaps lowered the barrier of entr…

We're arguing the same point. :)

Re: Building Meta's GenAI infrastructure

#125

Post by YetAnotherNick »

dougb5 wrote: post #88
YetAnotherNick wrote: post #50

Earlier quoted context omitted.

> $3.5b Which is a fourth of what they spent in VR/AR in a year. And Gen AI is something they could easily get more revenue as it has now become proven technology, and Meta could possibly leapfrog others because of the data moat.

Proven technology, maybe, but proven product-market fit for the kinds of things Facebook is using it for? Their linked blog about AI features gives examples "AI stickers" and image editing... cool, but are these potential multi-billion dollar lifts to their existing business? I guess I'm skeptical it's worthwhile unless they're able to unseat ChatGPT with a market-leading general purpose assistant.

> unseat ChatGPT with a market-leading general purpose assistant.

It's not impossible. The prediction from many(not that I believe it) is that over long run modelling tricks would become common knowledge and only thing that matters is compute and data, both of which Meta has.

Also there could be a trend of LLMs for ads or feed recommendation in the future as they has large completely unstructured dataset per user across multiple sites.

Re: Building Meta's GenAI infrastructure

#126

Post by virtuallynathan »

lifeisstillgood wrote: post #60
DEDLINE wrote: post #16

I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors OpenAI takes money from MSFT and buys Azure services Anthropic takes Amazon money and buys AWS services (as do many robotics etc) I am fairly sure it’s not illegal but it…

Facebook has more datacenter space and power than Amazon, Google, and Microsoft -- possibly more than Amazon and Microsoft combined...

Re: Building Meta's GenAI infrastructure

#127

Post by crmd »

echelon wrote: post #87
wizzwizz4 wrote: post #81

Earlier quoted context omitted.

> And if you think any Jack or Jill can just come in and text prompt a whole movie, you're crazy. It's still hard work and a metric ton of good taste. If you want anything good , yes. If you just want something … I reckon it'd take a week to assemble an incomprehensible-nonsense-film pipeline, after which it's just a matter of feeding the computer electricity. Short-term, this is going to funnel resources away from t…

> If you want anything good, yes. If you just want something ... You don't even need AI for that. https://en.wikipedia.org/wiki/YouTube_poop https://en.wikipedia.org/wiki/Skibidi_Toilet The idea that AI isn't going to be used as a creative tool too and that it won't lead to more and better art is a defeatist, Luddite attitude. Similarly shaped people thought that digital cameras would ruin cinema and photography. > S…

>You'll be able to follow that steampunk vampire creator that's been missing from the world until now. Every long tail interest will be catered to. Even the most obscure and wild tastes, ideas, and designs. Stuff that would never get studio funding.

Your optimism reminds me of the optimism I had around the early internet. Power to the people, long tail, rise of the creative class, the fall of gatekeeping corporations, etc.

It was like that for a couple of years in the late 90s before power and control got vastly more centralized than before. Maybe this time it’ll be different.

Re: Building Meta's GenAI infrastructure

#128

Post by KaiserPro »

jvanderbot wrote: post #110

So, I'd love to work on optimizing pipelines like this. How does one "get into" it? It seems a ML scientist with some C/C++ and infra knowledge just dips down into the system when required? Or is it CUDA/SIMD experts who move "up" into ML?

A lot of the optimisation at this level is getting data into the right place at the right time, without killing the network.

Its also a group effort to provide simple to use primitives that "normal" ML people can use, even if they've never used hyper scale clusters before.

So you need a good scheduler, that understand dependencies (no, the k8s scheduler(s) are shit for this, plus it wont scale past 1k nodes without eating all of your network bandwidth), then you need a dataloader that can provide the dataset access, then you need the IPC that allows sharing/joining of GPUs together.

all of that needs to be wrapped up into a python interface that fairly simple to use.

Oh and it needs to be secure, pass an FCC audit (ie you need to prove that no user data is being used) have a high utilisation efficiency and uptime.

the model stuff is the cherry on the top

Re: Building Meta's GenAI infrastructure

#129

Post by vineyardmike »

fuddle wrote: post #35

How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b

It’s often forgotten now, but just a few years NVidia was cancelled production batches and writing down inventory when the GPU shortage cleared. No one needed more GPUs. It also happens to be when Meta first announced they were going to increase CapEx spending on compute.

I’m guessing that Meta got a sweetheart deal to help take a lot of inventory for NVidia and make commitments for future purchases.

Re: Building Meta's GenAI infrastructure

#130

Post by dsp »

virtuallynathan wrote: post #126
lifeisstillgood wrote: post #60

Earlier quoted context omitted.

FB does not have the flywheel of running data centres - all three of those mentioned run hyper scale datacentres that they can then juice by “investing” billions in AI companies who then turn around and put those billions as revenue in the investors OpenAI takes money from MSFT and buys Azure services Anthropic takes Amazon money and buys AWS services (as do many robotics etc) I am fairly sure it’s not illegal but it…

Facebook has more datacenter space and power than Amazon, Google, and Microsoft -- possibly more than Amazon and Microsoft combined...

[citation needed]
Post reply on HN