Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

281–290 of 314 posts

Re: Building Meta's GenAI infrastructure

#281

Post by zooq_ai »

elbear wrote: post #261
zooq_ai wrote: post #165

Earlier quoted context omitted.

at the end of the day, you are still moving, storing and manipulating 1's and 0's, whether you are a front end engineer or a backend engineer or systems engieer or an ML engineer or an infra engineer

yeah, but how do you get the hiring managers to see things in the same way? :)

well at least I fit my resume to match the 'job description' because at the end of the day it's all hallucinations and 'real' software engineers that has core computer science skills can literally do anything

Re: Building Meta's GenAI infrastructure

#282

Post by ZiiS »

dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

Only hyper-scale companies like ATT could build the fibre; scrappy startups like Google and Amazon ate their lunch.

Re: Building Meta's GenAI infrastructure

#283

Post by virtuallynathan »

the-rc wrote: post #220
virtuallynathan wrote: post #182

Earlier quoted context omitted.

To date, facebook has built, or is building, 47,100,000 sq ft of space, totaling nearly $24bn in investment. Based on available/disclosed power numbers and extrapolating per sqft, I get something like 4770MW. Last I updated my spreadsheet in 2019, Google had $17bn in investments across their datacenters, totaling 13,260,000 sq ft of datacenter space. Additional buildings have been built since then, but not to the sca…

Does the square footage take into account multiple floors? What's the source? It can be misleading, because you don't know the compute density of what's inside. Using just public data, power is a more accurate proxy. Until at least 5-6 years ago, Google was procuring more electricity than Amazon. Before that, it had a further advantage from lower PUE, but I bet the big names are all comparable on that front by now. A…

Google certainly has more compute density than Amazon, the numbers I was able to find from the local power company was 250MW at Council Bluffs back in 2015 or so.

Amazon builds out 32MW shells, and the most utilized as of 5 or 6 years ago was 24MW or so, with most being much less than that.

Re: Building Meta's GenAI infrastructure

#285

Post by virtuallynathan »

chatmasta wrote: post #210
virtuallynathan wrote: post #177

Earlier quoted context omitted.

You can just map them all... I have. I also worked at AWS :)

This would be an interesting dataset to use for trading decisions (or sell to hedge funds). But I wonder how much of their infrastructure is publicly mappable, compared to just the part of it that's exposed to the edge. (Can you map some internal instances in a VPC?) That said, I'm sure there are a lot of side channels in the provisioning APIs, certificate logs, and other metadata that could paint a decently accurate…

I’m not sure mapping VPCs is super helpful - the physical infra is fairly distinct.

AWS has also disclosed 20 million Nitro adapters have been deployed, so you can do some backwards napkin math from that.

Re: Building Meta's GenAI infrastructure

#286

Post by anon373839 »

Jensson wrote: post #278
herval wrote: post #248

Earlier quoted context omitted.

I’m not sure we went through the same dot-com era, but in my experience, it was extremely expensive to spin up anything. You’d have to run your own servers, buy your own T1 lines, develop with rudimentary cgi… it was a very expensive mess - just like AI today Which gives me hope that - like the web - hardware will catch up and stuff will become more and more accessible with time

> I’m not sure we went through the same dot-com era, but in my experience, it was extremely expensive to spin up anything. You’d have to run your own servers, buy your own T1 lines, develop with rudimentary cgi… it was a very expensive mess - just like AI today To make your own competing LLM today you need hundreds of millions of dollars, the "very expensive" of this is on a whole different level. You could afford th…

I think the foundation models are a commodity, anyway. The bulk of the economic value, as usual, will be realized at the application layer. Building apps that use LLMs, including fine-tuning them for particular purposes, is well within reach even of indie/solo devs.

That’s why Sam Altman makes so much noise about “safety” - OpenAI would really like a government-backed monopoly position so they can charge higher rents and capture more of that value for themselves. Fortunately, I think that llama has already left the barn.

Re: Building Meta's GenAI infrastructure

#287

Post by infecto »

richardw wrote: post #263
dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

I find the market way more open and competitive than dot-com. Everyone is throwing up a chatbot or RAG solution. There are tradesmen and secretaries and infinite 19 year olds who are now able to wire together a no-code app or low-code bot and add value to real businesses. The hyper scalars are making some money but absolutely don't have this locked up. Any Groq or Mistral could wander in and eat their lunch, and we h…

Could not have said it better. Nobody has won the race yet and things are getting better. Building a foundation model is not cheap but not out of reach still for a startup.

Re: Building Meta's GenAI infrastructure

#288

Post by infecto »

verticalscaler wrote: post #113
oblio wrote: post #108

Earlier quoted context omitted.

The thing is, this could be considered basic research, right? Basic research IS setting money on fire until (and if) that basic research turns into TCP/IP, Ethernet and the Internet.

I wish. Funnily enough Arpanet and all that Xerox stuff were like Where as I think this more appropriately can be considered the meta PR budget. They simply can't not spend it, would look bad for Wall Street. Have to keep up with the herd.

Funny you pick a company that has very little to answer to the markets, out of all the large tech companies, META is the rare one that does not need to answer because Zuckerberg controls the company.

Re: Building Meta's GenAI infrastructure

#289

Post by infecto »

verticalscaler wrote: post #107
Dr_Birdbrain wrote: post #82

Earlier quoted context omitted.

I think the raw text inside Facebook groups is at least as valuable as Reddit data. Even if demographics data is restricted under European law, the raw text of people interacting is quite valuable.

Indeed, my deranged auntie posting on FB is approximately as valuable as my ADHD/PTSD quaranteeny nephew redditing.

Ahhh you had posted some other negative META criticism that was not even factual. Your made-up narratives really do not paint the correct picture.
Post reply on HN