Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta’s Hyperscale Infrastructure: Overview and Insights

cacm.acm.org

Post reply on HN
Print view View original

131–140 of 182 posts

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#131

Post by kccqzy »

lionkor wrote: post #54

> the image is not cached at CDN109 when the user requests it, CDN109 forwards the request to a nearby PoP. The PoP then forwards the request to the load balancer in a datacenter region, which retrieves the image from the storage system. Say I want a 1MB image, wouldn't it be faster to serve me the 1MB image over a slow connection with 100ms latency, than going through multiple hops of increasing latency, with multip…

The connection between CDN and PoP and data center is much faster than you think.

You don't need FTL tech. You just need good private fiber. Geographic distance is hardly ever the limiting factor on the public internet; yet it often is for private networks.

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#132

Post by UltraSane »

nk8620 wrote: post #76

> Moreover, for non-AI compute workloads, we offer only a single server type, equipped with one CPU and the same amount of DRAM (previously 64GB, now 256GB). I'm reading something like this for the first time. Is this common across industry or only typical to Meta? In contrast, we use multiple instances for sub-components of our ML training pipeline.

One reason for using a single CPU is that the number of cores per CPU has become very high and only using a single CPU avoids a lot of complexity around Non-Uniform Memory Architecture where the OS should allocate data to RAM that is "closest" to the CPU it is being used by for best performance.

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#133

Post by arbor_day »

ThinkBeat wrote: post #78

Quickly developed serverless functions combined with continuous deployment, and anyone can make edits in the entire codebase sounds like a dystopian nightmare. The amount of logging that is required to debug and find bugs is extreme. Using Erlang to write serverless functions seems like avoiding all the huge benefits BEAM can offer. >Additionally, product engineers predominantly write code in stateless, serverless fu…

I was suspicious before I joined, but it works way better than you'd think!

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#135

Post by mightyham »

ribadeo wrote: post #32

At least half their gak is due to them NOT moving quickly and NOT wanting to break things. IIRC, graphql is a means of papering over a bunch of legacy APIs. They removed foreign keys from mysql using it as a column store db, a vestige of the original LAMP stack still on PHP. I don't think Meta infrastructural choices are applicable to most folk. What does serverless land your average dev? A high AWS bill. Elastic man…

What is your actual point? You are criticizing Meta for building complex systems by iterating on boring tech like PHP and MySQL, and instead suggesting that they build their systems on top of different boring tech like YAML and Ansible. Why? The fact is that there are no "boring" off-the-shelf solutions for solving the problems that Meta is facing like whole region failures and performant cross-datacenter routing/sharding.

Your comment feels like it's not actually engaging with the contents of this article. It's not that Meta is creating bespoke technologies only out of fear of breaking past code. Their entire methodology of innovation is highly iterative and grounded in feedback through practical demonstration of results. You say that "Facebook creates tech to mitigate the pathologies their past continuously present", as if that is a bad approach, but considering Meta's success, I think it would be wise to seriously reconsider that position.

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#136

Post by stravant »

pavlov wrote: post #125
Simon_O_Rourke wrote: post #28

Earlier quoted context omitted.

They might have launched it quickly, but nobody (relatively speaking) gave a tinkers damn about the end product. All form and no function.

Threads has 320 million monthly active users and 100 million daily. It’s certainly not a flop. It’s almost as big as X globally. There’s a tremendous locality bias around Meta’s products. Social media use is very localized by geography and age. So if all your friends stopped using Facebook ten years ago, you assume that’s probably true everywhere, while in fact they’ve added several billion users on FB since then.

It seems like a flop not in user count but in that I'm not seeing it originate any valuable content.

I've seen several interesting posts from Bluesky referenced elsewhere already, but literally nothing from Threads despite it having had more users for longer.

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#137

Post by pavlov »

stravant wrote: post #136
pavlov wrote: post #125

Earlier quoted context omitted.

Threads has 320 million monthly active users and 100 million daily. It’s certainly not a flop. It’s almost as big as X globally. There’s a tremendous locality bias around Meta’s products. Social media use is very localized by geography and age. So if all your friends stopped using Facebook ten years ago, you assume that’s probably true everywhere, while in fact they’ve added several billion users on FB since then.

It seems like a flop not in user count but in that I'm not seeing it originate any valuable content. I've seen several interesting posts from Bluesky referenced elsewhere already, but literally nothing from Threads despite it having had more users for longer.

Elon Musk's trans daughter Vivian has been very active on Threads. Maybe she's on BlueSky now too, but I think she started on Threads.

(She's 20 years old, her father is the richest man in the world, and he regularly does interviews saying his child is literally dead to him. Enough drama for an Orson Welles script and a second-tier social media service.)

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#138

Post by eclark »

While I was at FB (it wasn't Meta then), I saw what a superpower the infrastructure is there. Product engineers build things of a scale in days. While I was there, I got to be tech lead for several different teams (2x distributed dbs, 1x Dev Efficiency, 1x Ads), some of which are called out by name here.

Shout out to the HBase and ZippyDB teams! This is the first public acknowledgment that ZippyDB was converged upon.

It's also super cool to see the Developer Efficiency pushes called out. 10,000 Services pushed daily, or every commit is so impressive.

When I left FB, I couldn't find anything close. So, I'm building the infra that I was missing as a startup. Batteries Included. https://www.batteriesincl.com/ https://github.com/batteries-included/batteries-included/

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#139

Post by umeshunni »

philg_jr wrote: post #107

Amazing. All of this truly wild, impressive technology and some of the absolute best engineers in the world, just to shove more ads in people's eyeballs. Sigh.

The tragedy of our times is that some of our brightest minds are working tirelessly, not to cure diseases or to explore the stars, but to improve the probability that someone will click on an ad by 0.1%.

Re: Meta’s Hyperscale Infrastructure: Overview and Insights

#140

Post by thrusong »

javier2 wrote: post #64
vasco wrote: post #4

> Moreover, once it [Threads] was developed, the infrastructure teams were given only two day’s notice to prepare for its production launch. Most large organizations would take longer than two days just to draft a project plan involving dozens of interdependent teams, let alone execute it. At Meta, however, we quickly established war rooms across distributed sites, bringing together both infrastructure and product te…

They already had these teams and they already had the infrastructure. 100M is a drop in the ocean to Meta

That's what I thought too when they touted this meteoric rise to 100M in the early days of Threads.

If it's tied to Instagram, and these users aren't using Instagram while they're on Threads, isn't it basically the same difference?

Post reply on HN