Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Meta's GenAI infrastructure

engineering.fb.com

Post reply on HN
Print view View original

241–250 of 314 posts

Re: Building Meta's GenAI infrastructure

#241

Post by whatshisface »

boringg wrote: post #240
renegade-otter wrote: post #201

Earlier quoted context omitted.

Not everything has to be AI. You can run a small business infra for MUCH less than you did back then, especially if you adjust for inflation (!). Training AI models costs a fortune, but so far it's been just front-loading costs in hopes of a windfall. We'll see what actually happens.

Front loading costs to eventually extract rents on usage with one hell of a capital wall protecting the assets. Its easier to spin up a business for sure -- also easier to unwind it - there not as sticky as they used to be.

If the government can stay back far enough that more than one AI company can train their models, it will end up working like steel mills - barely enough profit to pay the massive cost of capital due to competition. If the government regulates the industry into a monopoly, all bets are off. Their investors are going to push hard for shutting the door behind them so watch out.

The only question is - what tactic? I don't really know, but one trick I am aware of is "specifying to the vendor." In other words, the introduction of regulatory requirements that are at every step in the process a description of the most favored vendor's product. As the favored players add more features, potentially safety features, those features are required in new regulations, using very specific descriptions that more or less mandate that you reproduce the existing technology, to use a software engineer's term, bug-for-bug. If your product is better in some ways but worse in others, you might have a chance in the market - but to no avail, if the regulations demand exactly the advantages of the established suppliers.

Re: Building Meta's GenAI infrastructure

#242

Post by brookst »

boringg wrote: post #240
renegade-otter wrote: post #201

Earlier quoted context omitted.

Not everything has to be AI. You can run a small business infra for MUCH less than you did back then, especially if you adjust for inflation (!). Training AI models costs a fortune, but so far it's been just front-loading costs in hopes of a windfall. We'll see what actually happens.

Front loading costs to eventually extract rents on usage with one hell of a capital wall protecting the assets. Its easier to spin up a business for sure -- also easier to unwind it - there not as sticky as they used to be.

This is typically called a high fixed cost business, like airlines, hotels/apartments, SpaceX, etc.

The dream may be barriers to entry that allow high margins (“rents” if you prefer the prejudicial), but all too often these huge capital costs bankrupt the company and lose money for investors (see: WeWork, Magic Leap). It is high risk, high return. Which seems fair.

Re: Building Meta's GenAI infrastructure

#243

Post by lmm »

zeroonetwothree wrote: post #229
spencerchubb wrote: post #223

All this compute and my Instagram Reels feed still isn't as good as my TikTok feed

What does that have to do with Gen AI

If Gen AI doesn't have anything to do with "Meta"'s actual business then WTF are they setting all this money on fire for?

Re: Building Meta's GenAI infrastructure

#244

Post by brookst »

latchkey wrote: post #227
valzam wrote: post #206

Earlier quoted context omitted.

Don't worry, this goal will change with the next hype cycle

I pity the fools that think AI is just another internet hype cycle.

I’m old enough to remember the proud, defiant declarations that the internet was just a hype cycle.

Re: Building Meta's GenAI infrastructure

#246

Post by latchkey »

brookst wrote: post #244
latchkey wrote: post #227

Earlier quoted context omitted.

I pity the fools that think AI is just another internet hype cycle.

I’m old enough to remember the proud, defiant declarations that the internet was just a hype cycle.

I got my first email in 1991 and started my first internet business in 1995 (a web dev shop). My entire life has been an endless hype cycle.

Re: Building Meta's GenAI infrastructure

#247

Post by lmm »

echelon wrote: post #66
__loam wrote: post #55

Earlier quoted context omitted.

"My paycheck depends on this technology destroying every field producing cultural artifacts"

Said the butter churner, cotton ginner, and petrol pumper. I work in film. I've shot dozens of them the old fashioned way. I've always hated how labor, time, and cost intensive they are to make. Despite instructions from the luminaries to "just pick up a camera", the entire process is stone age. The field is extremely inequitable, full of nepotism and "who you know". Almost every starry-eyed film student winds up doi…

> Said the butter churner, cotton ginner, and petrol pumper.

Said the bank teller, record producer, etc.. Plenty of cases where we've been told technology and automation would democratise the field and remove the middleman, and actually it's the opposite.

Yes, it would be nice if AI made it easy for anyone who wanted to make a great movie. That doesn't mean it's going to happen.

Re: Building Meta's GenAI infrastructure

#248

Post by herval »

dougdonohoe wrote: post #191

Having lived through the dot-com era, I find the AI-era slightly dispiriting because of the sheer capital cost of training models. At the start of the dot-com era, anyone could spin up an e-commerce site with relatively little infrastructure costs. Now, it seems, only the hyper-scale companies can build these AI models. Meta, Google, Microsoft, Open-AI, etc.

I’m not sure we went through the same dot-com era, but in my experience, it was extremely expensive to spin up anything. You’d have to run your own servers, buy your own T1 lines, develop with rudimentary cgi… it was a very expensive mess - just like AI today

Which gives me hope that - like the web - hardware will catch up and stuff will become more and more accessible with time

Re: Building Meta's GenAI infrastructure

#249

Post by bigcat12345678 »

trsohmers wrote: post #77
fuddle wrote: post #35

How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b

Significantly more than that; MFN pricing for NVIDIA DGX H100 (which has been getting priority supply allocation, so many have been suckered into buying them in order to get fast delivery) is ~$309k, while a basically equivalent HGX H100 system is ~$250k, coming to a price per GPU at the full server level being ~$31.5k. With Meta’s custom OCP systems integrating the SXM baseboards from NVIDIA, my guess is that their…

Would you kindly provide sources to the numbers? What is MFN?

Thanks! (Your number is consistent with what I hear of, but I never managed to get solid sources to back them up)

Re: Building Meta's GenAI infrastructure

#250

Post by fragmede »

verticalscaler wrote: post #107
Dr_Birdbrain wrote: post #82

Earlier quoted context omitted.

I think the raw text inside Facebook groups is at least as valuable as Reddit data. Even if demographics data is restricted under European law, the raw text of people interacting is quite valuable.

Indeed, my deranged auntie posting on FB is approximately as valuable as my ADHD/PTSD quaranteeny nephew redditing.

That ignores all the user groups that are on Facebook. From apartment communities aka Nextdoor to grief support counseling to the mindfulness therapy groups, there’s a wealth of user comments a tad bit higher than Uncle John’s racist rants.
Post reply on HN