Building Meta's GenAI infrastructure
engineering.fb.com
Building Meta's GenAI infrastructure
1–10 of 314 posts
Re: Building Meta's GenAI infrastructure
#2Honestly Meta is consistently one of the better companies at releasing tech stack info or just open sourcing, these kinds of articles are super fun
Re: Building Meta's GenAI infrastructure
#3Yann wants to be open and Mark seems happy to salt the earth.
Re: Building Meta's GenAI infrastructure
#4Honestly Meta is consistently one of the better companies at releasing tech stack info or just open sourcing, these kinds of articles are super fun
Do you find this informative?
Re: Building Meta's GenAI infrastructure
#5Just for comparison, Swiss CSCS new Alps system will get 5k GH200 nodes (each with a H100).
Re: Building Meta's GenAI infrastructure
#6Honestly Meta is consistently one of the better companies at releasing tech stack info or just open sourcing, these kinds of articles are super fun
I think some elements of this stack might flow into the open compute.
Re: Building Meta's GenAI infrastructure
#7I'd be great if they could invest in an alternative to nvidia -- then, in one fell swoop, destroy the moats of everyone in the industry.
Re: Building Meta's GenAI infrastructure
#8I know we won't get it this from FB, but I'd be really interested to see how the relationship of compute power to engineering hours scales.
They mention custom building as much as they can. If FB magically has the option to 10x the compute power, would they need to re-engineer the whole stack? What about 100x? Is each of these re-writes just a re-write, or is it a whole order of magnitude more complex?
My technical understanding of what's under the hood of these clusters is pretty surface level- super curious if anyone with relevant experience has thoughts?
Re: Building Meta's GenAI infrastructure
#9Yann wants to be open and Mark seems happy to salt the earth.
What do you mean?
Re: Building Meta's GenAI infrastructure
#10This reads more like a flex for the investment community.