You've got to read "open" roughly 3x in a paragraph.
Building Meta's GenAI infrastructure
91–100 of 314 posts
Re: Building Meta's GenAI infrastructure
#92Earlier quoted context omitted.
What moat exactly? Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. I'm not even sure they can use historical data. Meta certainly has an edge in engineer count, undoubtedly. But I'd say they really, really want the metaverse to succeed more to have their on walled garden (i.e. equivalent power of Apple and Google stores, etc.). T…
I think the raw text inside Facebook groups is at least as valuable as Reddit data. Even if demographics data is restricted under European law, the raw text of people interacting is quite valuable.
facebooks downfall will be their lock in. every other social media platform lets you view a public profile, discussion groups etc. it's all locked inside facebook.
Re: Building Meta's GenAI infrastructure
#93it's really interesting just how similar these systems are to the designs adopted for HPC over the past few decades. I'm salty because it took a while for the ML community to converge on this (20+K GPUs connected by a real fabric with low latency and high bandwidth).
Re: Building Meta's GenAI infrastructure
#94I wonder if Meta would ever try to compete with AWS / MSFT / GOOG for AI workloads
I think Meta have avoided doing this because it would complicate their business priorities. They don’t really do B2B.
Re: Building Meta's GenAI infrastructure
#95How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b
That sounds like a reasonable budget for 3 years of hardware at a major AI company.
Re: Building Meta's GenAI infrastructure
#96"Share this: Hacker News" Noice
Re: Building Meta's GenAI infrastructure
#97I know we won't get it this from FB, but I'd be really interested to see how the relationship of compute power to engineering hours scales. They mention custom building as much as they can. If FB magically has the option to 10x the compute power, would they need to re-engineer the whole stack? What about 100x? Is each of these re-writes just a re-write, or is it a whole order of magnitude more complex? My technical u…
"just a re-write"
...the idea is that at some point it "just re-writes" itself.
Re: Building Meta's GenAI infrastructure
#98Earlier quoted context omitted.
> $3.5b Which is a fourth of what they spent in VR/AR in a year. And Gen AI is something they could easily get more revenue as it has now become proven technology, and Meta could possibly leapfrog others because of the data moat.
Proven technology, maybe, but proven product-market fit for the kinds of things Facebook is using it for? Their linked blog about AI features gives examples "AI stickers" and image editing... cool, but are these potential multi-billion dollar lifts to their existing business? I guess I'm skeptical it's worthwhile unless they're able to unseat ChatGPT with a market-leading general purpose assistant.
I have a few group chats just that devolve into hours of sending stickers or image generation back and forth, lately we've been "writing a book together" with @Meta AI as the ghost writer, and while it utterly sucks, its been a hilarious shared experience.
I don't think anyone else has gotten that group chat with AI thing so nailed.
Re: Building Meta's GenAI infrastructure
#99"Share this: Hacker News" Noice
I thought at first "what are you talking about", when i check my uBlock filters.
Was blocking the whole "Share this" content section.
Sharing on Hacker News ... they now their audience.
Re: Building Meta's GenAI infrastructure
#100How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b
Significantly more than that; MFN pricing for NVIDIA DGX H100 (which has been getting priority supply allocation, so many have been suckered into buying them in order to get fast delivery) is ~$309k, while a basically equivalent HGX H100 system is ~$250k, coming to a price per GPU at the full server level being ~$31.5k. With Meta’s custom OCP systems integrating the SXM baseboards from NVIDIA, my guess is that their…
350,000 NVIDIA H100 x $23k = $8b :0