Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How meta built the infrastructure for Threads

engineering.fb.com

Post reply on HN
Print view View original

131–140 of 184 posts

Re: How meta built the infrastructure for Threads

#131

Post by stickfigure »

aschobel wrote: post #104

What do folks use for something like Meta’s Async? RabbitMQ? > The workloads commonly executed on Async are those that do not require blocking an active user’s experience with a product and can be performed anywhere from a few seconds to several hours after a user’s action.

I'm an enormous fan of Google Cloud Tasks (previously known as the Google App Engine task service). The low-level mechanism is "call this URL later" but with a tiny bit of application glue the abstraction is "run this code later".

Re: How meta built the infrastructure for Threads

#132

Post by lamroger »

neon_electro wrote: post #90
erulabs wrote: post #88

Gotta love how far MySQL + k/v store can get ya. I have no experience with zippyDB, but MySQL and ScyllaDB certainly feels like having an M1 Abrams and a B2 Spirit in my back pocket. I’m constantly pushing the “relationships from MySQL, data from Cassandra” narrative. It sounds like Facebook has a similar approach - although this post doesn’t make it clear where message data actually resides. MySQL, ScyllaDB, and Rab…

Can you share your take on why MySQL and not Postgres?

I haven't worked in big tech but I imagine their tooling is really good for this particular stack. Internal expertise too for running into big scale edge cases.

Re: How meta built the infrastructure for Threads

#133

Post by butlike »

dissident_coder wrote: post #73
captn3m0 wrote: post #56

Earlier quoted context omitted.

This is true for all Meta/Google/Twitter properties. No phone number results in a high chance of suspension irrespective of your usage.

It's a bot prevention measure (allegedly). It also conveniently blocks out people that are more likely to not want to share all of their personal information with them. No skin off my back, the less toxic social media garbage I'm allowed to use, the better for my sanity, good riddance.

TO BE FAIR (and I'm still on the fence how I feel about this)... the less personal information you share, the more bot-like you appear to be.

Re: How meta built the infrastructure for Threads

#134

Post by Ukv »

MikusR wrote: post #57
ranting-moth wrote: post #51

Earlier quoted context omitted.

Worth mentioning it's also a data harvesting app-only offering.

And here I was hoping that with technological advances spambots would at least post semi-truthful spam. Threads are fully functional on web.

When I tried to join, there was nothing on the threads.net website except the word "Threads" in front of some spiral particles and a QR code to get the app[0].

More recently it seems they've added a login dialog to the site, but attempting to log in still just immediately prompts you to download the app[1].

Presumably once you've used the app to create and set up an account you can now use it to browse online, but that's not really "fully functional on web" and, relevant for the accusation of data-harvesting, still requires you to download the app.

[0]: https://i.imgur.com/PQF7bkm.png

[1]: https://i.imgur.com/zLfubq1.png

Re: How meta built the infrastructure for Threads

#135

Post by kenhwang »

As a user of Threads and X (formerly known as Twitter). Threads is so dog slow, even compared to Instagram. Slow to respond to requests for just about everything (several seconds), even slower at showing an up to date feed (10s of seconds). Really think it's what's holding back adoption because Twitter can show up to date personalized posts in under a second.

I love all the quality of life features Threads is adding, but it severely lags FB/Instagram/Twitter in showing up to date information quickly.

Re: How meta built the infrastructure for Threads

#136

Post by jauntywundrkind »

aschobel wrote: post #104

What do folks use for something like Meta’s Async? RabbitMQ? > The workloads commonly executed on Async are those that do not require blocking an active user’s experience with a product and can be performed anywhere from a few seconds to several hours after a user’s action.

Generally I'd suggest combining different messaging/streaming(ex: Kafka) & FaaS/serverless (ex: OpenFaaS, Fission, OpenWhisk) layers to achieve something like Async. Where-as Async seems to be a bit of both. I think over time we'll see streaming systems expand their footprint & keep eating away at the space pure FaaS occupies.

Apache Pulsar has been around for a while as a streaming system, and it does have a fairly significant Pulsar Functions layer which might make it a decent all in one candidate. (Pulsar in general feels like a much smarter lower-latency take on Kafka, allowing much more control/flexibility of where data gets backed/replicated/read from that makes it exceedingly more useful for "log/stream is the database" styles.) https://pulsar.apache.org/docs/3.1.x/functions-overview/

There's so many up and coming streaming things here. Apache Ballista. Netflix's Mantis. RisingWave. A million years ago there was SOA; a startup I was at used Apache Camel. Benthos. Others. Many of these will decidedly remain more streaming oriented, not encroach on FaaS space, which is fine, but I think we'll see overlap grow. Reductionist, but a function call is "run this code when a function name and parameters happen" versus maybe "run this code when an event type happens", which seems not so far apart to me.

Re: How meta built the infrastructure for Threads

#137

Post by cj »

jjice wrote: post #130
salil999 wrote: post #127

Earlier quoted context omitted.

Yeah but what about something that won't make my wallet cry?

Pretty much any queue and a compute will do. SQS paired with lambda an event bridge (I forget if that's the service to trigger the lambda or not) would be solid. I lean towards compute that's a bit easier to run in different settings so just any old code checking the queue and executing as expected also works. SQS is cheap and it's pretty damn solid. I believe it was the first AWS service. But in reality, any managed…

Is using Lambda with an external data store (like MongoDB) a solved problem?

I remember this being an issue a long time ago, where Lambda pretty much was a non starter for use cases where you're running 1000 independent tasks that each, on their own, is a small unit of work (just 1 or 2 database queries).

Long way of asking: is there a way to share a database connection between lambda functions or does each run need to re-establish / re-authenticate a new connection with the database?

Re: How meta built the infrastructure for Threads

#138

Post by hnbad »

erulabs wrote: post #88

Gotta love how far MySQL + k/v store can get ya. I have no experience with zippyDB, but MySQL and ScyllaDB certainly feels like having an M1 Abrams and a B2 Spirit in my back pocket. I’m constantly pushing the “relationships from MySQL, data from Cassandra” narrative. It sounds like Facebook has a similar approach - although this post doesn’t make it clear where message data actually resides. MySQL, ScyllaDB, and Rab…

From having talked to Facebook engineers (I suspect Instagram/Threads is a very different beast for historical reasons), Facebook is only built on "PHP and MySQL" in the sense that the statement is technically correct but not helpful.

The way Facebook apparently uses MySQL is not "MySQL + k/v store" but "MySQL is the k/v store". Forget SQL, they treat it as a key-value lookup and it's good enough for that. This also explains a lot of oddities coming out of FB engineering open source that at times look like they're trying to roll their own database on top of ... well, a database.

I wouldn't go to Facebook for architecture advice unless you have both the scale of Facebook and the technological baggage of Facebook. I'm not sure how transferrable this writeup about Threads is either given that Threads is so strongly tied to Instagram (although EU legislation seems to have motivated Meta at least to no longer require users to delete their Instagram account in order to delete their Threads profile).

Re: How meta built the infrastructure for Threads

#139

Post by ericwood »

modeless wrote: post #95
itslennysfault wrote: post #83

I'm kinda loving the obvious "X" shills in the comments coming just to declare Threads dead. It is literally the 2nd most downloaded app in the AppStore (was #1 for a bit, but Temu is seemingly unstoppable). Personally, it has become my preferred social media platform. It's overflowing with content and I check in on it several times a day, and have been really enjoying the community that is emerging. It DOES however…

My threads feed started OK at launch but now is a wasteland of pointless rage bait and r/antiwork style content. I'm sure it's popular but it's not what I want to read. On X I get mostly informative ML and robotics content. Of course it took some curation and liberal use of the "Mute Words" feature to get there. But I just don't think that content is there on Threads. My go-to example of why I don't like what's happe…

The recommendation algorithm is pretty awful and needs a lot of intervention to start surfacing great content. Life got a lot better once I started clicking "hide" on particularly egregious posts and avoiding interaction. It seems to disproportionately weight "click to view" interactions so avoiding morbid curiosity when seeing a bad take helps.

If there weren't pressure to use it for my small side business I'd have given up on it almost immediately.

Re: How meta built the infrastructure for Threads

#140

Post by MikusR »

Ukv wrote: post #134
MikusR wrote: post #57

Earlier quoted context omitted.

And here I was hoping that with technological advances spambots would at least post semi-truthful spam. Threads are fully functional on web.

When I tried to join, there was nothing on the threads.net website except the word "Threads" in front of some spiral particles and a QR code to get the app[0]. More recently it seems they've added a login dialog to the site, but attempting to log in still just immediately prompts you to download the app[1]. Presumably once you've used the app to create and set up an account you can now use it to browse online, but th…

Then it has not launched in your region, probably. For me in EU it shows option to login with instagram or browse without a profile. No app needed.
Post reply on HN