Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How meta built the infrastructure for Threads

engineering.fb.com

Post reply on HN
Print view View original

151–160 of 184 posts

Re: How meta built the infrastructure for Threads

#151

Post by alexbevi »

cj wrote: post #137
jjice wrote: post #130

Earlier quoted context omitted.

Pretty much any queue and a compute will do. SQS paired with lambda an event bridge (I forget if that's the service to trigger the lambda or not) would be solid. I lean towards compute that's a bit easier to run in different settings so just any old code checking the queue and executing as expected also works. SQS is cheap and it's pretty damn solid. I believe it was the first AWS service. But in reality, any managed…

Is using Lambda with an external data store (like MongoDB) a solved problem? I remember this being an issue a long time ago, where Lambda pretty much was a non starter for use cases where you're running 1000 independent tasks that each, on their own, is a small unit of work (just 1 or 2 database queries). Long way of asking: is there a way to share a database connection between lambda functions or does each run need…

For MongoDB specifically see the best practices docs for Lambda (https://www.mongodb.com/docs/atlas/manage-connections-aws-la...).

This outlines creating the handler outside the function context to allow the client to be reused between invocations.

Re: How meta built the infrastructure for Threads

#152

Post by treesknees »

It really irks me when a big company like Meta tries to describe a new project as a startup created by a "nimble team of engineers". In the blog post they link to [1]

>Threads was developed in an environment more akin to a startup.

OK sure, a small group of people broke the model of development and did something new from the ground up?

>Threads scaled successfully to 100 million users without any major downtime thanks to Meta’s underlying infrastructure and engineering foundations, which were critical to the successful launch.

Ah, no. All of the underlying hard parts were done already for them. That's not a startup.

[1] https://engineering.fb.com/2023/09/07/culture/threads-inside...

Re: How meta built the infrastructure for Threads

#153

Post by sangnoir »

treesknees wrote: post #152

It really irks me when a big company like Meta tries to describe a new project as a startup created by a "nimble team of engineers". In the blog post they link to [1] >Threads was developed in an environment more akin to a startup. OK sure, a small group of people broke the model of development and did something new from the ground up? >Threads scaled successfully to 100 million users without any major downtime thank…

They never claimed their stack was startup-like, just the process. They are likely contrasting it with their usual[0] engineering practices: no focus-grouping, applying for headcount from 3 directors and making a 4-year roadmap after a gaggle of project managers finally agree on what to build. Perhaps its just one decision-maker and bunch if empowered engineers and no red tape - which would be very much be closer to startups, without denying themselves access to humongous datacenters.

0. Exaggerating some of the bureaucracy I've seen in larger orgs

Re: How meta built the infrastructure for Threads

#154

Post by hu3 »

evanelias wrote: post #147
hnbad wrote: post #138

Earlier quoted context omitted.

From having talked to Facebook engineers (I suspect Instagram/Threads is a very different beast for historical reasons), Facebook is only built on "PHP and MySQL" in the sense that the statement is technically correct but not helpful. The way Facebook apparently uses MySQL is not "MySQL + k/v store" but "MySQL is the k/v store". Forget SQL, they treat it as a key-value lookup and it's good enough for that. This also…

This is not correct. Facebook has a large number of distinct MySQL tiers, which serve different purposes and have wildly different workloads. None of them are traditional "key-value lookup" workloads, not even the core user database (entities/associations); this k/v claim is often repeated here, but has never been an accurate representation of the architecture. The TAO graph access pattern absolutely requires range s…

Thank you for sharing your experience and expertise.

I'm a big fan of MySQL's scaling properties and some of the tooling (Vitess).

Do you see any emergent general purpose RDBMS achieving similar popularity of MySQL and/or PostgreSQL in the near future?

I heard good things about Clickhouse but it's a different tool (column based) from my understanding.

Re: How meta built the infrastructure for Threads

#155

Post by Ukv »

MikusR wrote: post #150
Ukv wrote: post #145

Earlier quoted context omitted.

> Then it has not launched in your region, probably. For me in EU it shows option to login with instagram or browse without a profile. Threads itself is launched in my region. Could you screenshot the front threads.net page you see (when logged out)? > No app needed Do you mean you're able to create an account without the app, or do you just mean that there's no app needed for browsing in your region (with no account…

https://imgur.com/I10hXZ8 https://imgur.com/ZhzfDap

Option to browse without a profile does seem to be specific to the EU[0], and apparently has some limited functionality (can't search for individual posts).

The second screenshot is a form I can get to, but immediately after logging in it'll prompt to download the app to set up your profile.

[0]: https://timesofindia.indiatimes.com/gadgets-news/threads-mak...

Re: How meta built the infrastructure for Threads

#156

Post by pphysch »

trompetenaccoun wrote: post #126
pphysch wrote: post #76

Earlier quoted context omitted.

> I'm swamped with US Neo-Nazi tweets, as well as pro-Russia and pro-Hamas content. > adjusted its content accordingly to convince me of a hundred times more radical positions Globally, these positions are not radical (sans Neo-Nazi). The average human is (strongly) opposed to American foreign policy, if not outright pro-Russian/pro-Palestinian. That said, these topics are also great engagement bait and X does push t…

They are common but not majority opinion. The average Indian is more supportive of Israel for example. That's the most populous country in the world and you definitely see them on X as well. The average Chinese doesn't care about either and doesn't use any foreign social media apps. Many of us have a very selective view of the world. When you have a platform that allows most speech from anyone on the planet, such an…

The Chinese are overwhelmingly opposed to US foreign policy. Outright pro-Israel and pro-NATO views are virtually non-existent on Chinese social media.

Some Hindu-supremacist Indians support Israel because of shared Islamophobia, more or less. There is not much rabid Islamophobia outside of the West + India.

>They are common but not majority opinion.

To suggest that the (powerless) majority of 8 billion humans are not opposed to US foreign policy is... wildly out of touch. I would even wager that the average American is generally opposed to current US foreign policy.

Re: How meta built the infrastructure for Threads

#157

Post by [deleted] »

sangnoir wrote: post #153
treesknees wrote: post #152

It really irks me when a big company like Meta tries to describe a new project as a startup created by a "nimble team of engineers". In the blog post they link to [1] >Threads was developed in an environment more akin to a startup. OK sure, a small group of people broke the model of development and did something new from the ground up? >Threads scaled successfully to 100 million users without any major downtime thank…

They never claimed their stack was startup-like, just the process. They are likely contrasting it with their usual[0] engineering practices: no focus-grouping, applying for headcount from 3 directors and making a 4-year roadmap after a gaggle of project managers finally agree on what to build. Perhaps its just one decision-maker and bunch if empowered engineers and no red tape - which would be very much be closer to…

[deleted]

Re: How meta built the infrastructure for Threads

#158

Post by adonese »

pityJuke wrote: post #37
jonathanstrange wrote: post #16

Earlier quoted context omitted.

Honestly, I can't believe any real person still uses Twitter. I haven't subscribed to anyone and have no subscribers. Nevertheless, whenever I open it, I'm swamped with US Neo-Nazi tweets, as well as pro-Russia and pro-Hamas content. I'm not saying it's the only content I see, but enough to make Twitter look like the front page of a terrorist organization. Apparently, the Twitter algorithm has learned that I'm using…

You’re not wrong. The FYP, at least for my unused account, is garbage tier. Hell, the latest phenomenon it has picked up on is meme pages used to advertise OnlyFans and gambling, and outright pornography and porn ads. Some of the worst human beings are on there spreading ideas about race theory. Responses to anything vaguely popular are filled with blue check marks either trying to make money with engagement bait, or…

I don't what is it with porn on twitter, the blue check was supposed to stop them but they have gotten way worse than before.

I was never a good user for twitter, I don't engage that much except for just reading news and just follow some techie people and communities. But even for my light usage I can clearly see the downgrade of the service

Re: How meta built the infrastructure for Threads

#159

Post by gopher2000 »

laweijfmvo wrote: post #89

(This is performance evaluation week at Meta) I can only dream of being able to cite an article like this on my impacc...

I've never seen a company where performance evaluation gets as much focus as it does at Meta. Some people's entire existence at the company is to game PSCs.

Re: How meta built the infrastructure for Threads

#160

Post by evanelias »

hu3 wrote: post #154
evanelias wrote: post #147

Earlier quoted context omitted.

This is not correct. Facebook has a large number of distinct MySQL tiers, which serve different purposes and have wildly different workloads. None of them are traditional "key-value lookup" workloads, not even the core user database (entities/associations); this k/v claim is often repeated here, but has never been an accurate representation of the architecture. The TAO graph access pattern absolutely requires range s…

Thank you for sharing your experience and expertise. I'm a big fan of MySQL's scaling properties and some of the tooling (Vitess). Do you see any emergent general purpose RDBMS achieving similar popularity of MySQL and/or PostgreSQL in the near future? I heard good things about Clickhouse but it's a different tool (column based) from my understanding.

I am not a good judge of popularity, given that I still post a lot about MySQL on HN in the year 2024 :)

In all seriousness though, as a bootstrapped founder I'm not usually able to devote time to tinkering with emergent databases, sorry!

From what I've heard, Clickhouse is excellent, but it is OLAP focused rather than OLTP.

Post reply on HN