Live data from Hacker News

Production Twitter on one machine? 100Gbps NICs and NVMe are fast

thume.ca

151–160 of 500 posts

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#151
post #136

Earlier quoted context omitted.

Incidentally, a lot of people have argued that the massive datacenters used by e.g. AWS are effectively single large ("warehouse-scale") computers. In a way, it seems that the mainframe has been reinvented.

I wouldn't really agree with this since those machines don't share address spaces or directly attached busses. Better to say it's a warehouse-scale "service" provided by many machines which are aggregated in various ways.

I wonder though.. could you emulate a 20k-core VM with 100 terabytes of RAM on a DC?

Ethernet is fast, you might be able to get in range of DRAM access with an RDMA setup. cache coherency would require some kind of crazy locking, but maybe you could do it with FPGAs attached to the RDMA controllers that implement something like Raft?

it'd be kind of pointless and crash the second any machine in the cluster dies, but kind of a cool idea.

it'd be fun to see what Task Manager would make of it if you could get it to last long enough to boot Windows.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#152
post #62

He will be up for surprise. HTTP with connection: keep-open can serve 100k req/sec. But that's for one client being served repeatedly over 1 connection. And this is the inflated number that's published in webserver benchmark tests. For more practical down to earth test, you need to measure performance w/o keep-alive. Request per second will drop to 12k / sec then. And that's for HTTP without encryption or ssl handsha…

> Use HTTPS and watch it fall down to only 400 req / sec under load test [ without connection: keep-alive ].

I'm running about 2000 requests/s in one of my real-world production systems. All of the requests are without keep-alive and use TLS. They use about one core for TLS and HTTP processing.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#153
post #145

Earlier quoted context omitted.

Genuinely asking, why do you think Twitter needs 24 million vcpus to run? This is not apples to apples but Whatsapp is a product that entirely ran on 16 servers at the time of acquisition (1.5 billion users). It really begs the question why Twitter uses so much compute if there are companies that have operated significantly more efficiently. Twitter was unprofitable during acquisition and spent around half their reve…

Presumably there's an entire data engineering / event processing pipeline that's being used to track user interactions at a fine grained level. These events are going to be aggregated and munged by various teams for things like business analytics, product / experiment feature analysis, ad analysis, as well as machine learning model feature development (just to name a few massive ones off the top of my head). Each of…

And for the most part, this herculean effort is wasted. Most people just want to see latest tweets from people they follow. Everything else is fluff to manipulate engagement metrics, pad resumes and attempt to turn twitter into something it's users never wanted.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#154

Earlier quoted context omitted.

It is way more than 1.4TBs a second globally.

I wonder how much is api traffic and how much is assets & images.

I wonder how much of that is crypto spam bots replying to each other.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#155
post #10

Earlier quoted context omitted.

My bad - not English, but ASCII. The assumed max tweet size is in bytes rather than (UTF) characters.

That size in bytes is based on the max size in UTF-8 and UTF-16. Codepoints below U+1100 are counted as one "character" by twitter and will need at most 2 bytes. Codepoints above it are counted as two "characters" by twitter and will need at most 4 bytes. Therefore 560 bytes, and it supports all languages. Side note, this is more pessimistic than it needs to be, if you're willing to transcode. The larger codepoints f…

I was referring to the comment in the Rust code, not the analysis.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#156

If you really wanted to run Twitter on one machine at any cost, wouldn't an IBM mainframe be much more practical? You can even run Linux on them now. The specs he cites would actually be fairly small for a mainframe, which can reach up to 40TB of memory. I'm not saying this is a good idea, but it seems better than what the OP proposes.

My friend mentioned this just before I published and I think that probably is the fastest largest thing you can get which would in some sense count as one machine. I haven't looked into it, but I wouldn't be surprised if they could get around the trickiest constraint, which is how many hard drives you can plug in to a non-mainframe machine for historical image storage. Definitely more expensive than just networking a…

> which is how many hard drives you can plug in to a non-mainframe machine for historical image storage.

You would be surprised. First off, SSDs are denser than hard drives now if you're willing to spend $$$.

Second, "plug in" doesn't necessarily mean "in the chassis". You can expand storage with external disk arrays in all sorts of ways. Everything from external PCI-e cages to SAS disk arrays, fibre channel, NVMe-over-Ethernet, etc...

It's fairly easy to get several petabytes of fast storage directly managed by one box. The only limit is the total usable PCIe bandwidth of the CPUs, which for a current-gen EPYC 9004 series processors in a dual-socket configuration is something crazy like 512 GB/s. This vastly exceeds typical NIC speeds. You'd have to balance available bandwidth between multiple 400 Gbps NICs and disks to be able to saturate the system.

People really overestimate the data volume put out by a service like Twitter while simultaneously underestimating the bandwidth capability of a single server.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#157
post #62

He will be up for surprise. HTTP with connection: keep-open can serve 100k req/sec. But that's for one client being served repeatedly over 1 connection. And this is the inflated number that's published in webserver benchmark tests. For more practical down to earth test, you need to measure performance w/o keep-alive. Request per second will drop to 12k / sec then. And that's for HTTP without encryption or ssl handsha…

I agree most HTTP server benchmarks are highly misleading in that way, and mention in my post how disappointed I am at the lack of good benchmarks. I also agree that typical HTTP servers would fall over at much lower new connection loads. I'm talking about a hypothetical HTTPS server that used optimized kernel-bypass networking. Here's a kernel-bypass HTTP server benchmarked doing 50k new connections per core second…

TLS handling would dominate your performance, kernel bypassing would not help here unless you would also do TLS NIC offloading, you still need to process new TLS sessions from OP example and they would dominate your http processing time (excluding application business logic processing).

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#158
"Through intense digging I found a researcher who left a notebook public including tweet counts from many years of Twitter’s 10% sampled “Decahose” API and discovered the surprising fact that tweet rate today is around the same as or lower than 2013! Tweet rate peaked in 2014 and then declined before reaching new peaks in the pandemic. Elon recently tweeted the same 500M/day number which matches the Decahose notebook and 2013 blog post, so this seems to be true! Twitter’s active users grew the whole time so I think this reflects a shift from a “posting about your life to your friends” platform to an algorithmic content-consumption platform."

I know it's not the core premise of the article, but this is very interesting.

I believe that 90% of tweets per day are retweets, which supports the author's conclusion that Twitter is largely about reading and amplifying others.

That would leave 50 million "original" tweets per day, which you should probably separate as main tweets and reply tweets. Then there's bots and hardcore tweeters tweeting many times per day, and you'll end up with a very sobering number of actual unique tweeters writing original tweets.

I'd say that number would be somewhere in the single digit millions of people. Most of these tweets get zero engagement. It's easy to verify this yourself. Just open up a bunch of rando profiles in a thread and you'll notice a pattern. A symmetrical amount of followers and following typically in the range of 20-200. Individual tweets get no likes, no retweets, no replies, nothing. Literally tweeting into the void.

If you'd take away the zero engagement tweets, you'll arrive at what Twitter really is. A cultural network. Not a social network. Not a network of participation. A network of cultural influencers consisting of journalists, politicians, celebrities, companies and a few witty ones that got lucky. That's all it is: some tens of thousands of people tweeting and the rest leeching and responding to it.

You could argue that is true for every social network, but I just think it's nowhere this extreme. Twitter is also the only "social" network that failed to (exponentially) grow in a period that you might as well consider the golden age of social networks. A spectacular failure.

Musk bought garbage for top dollar. The interesting dynamic is that many Twitter top dogs have an inflated status that cannot be replicated elsewhere. They're kind of stuck. They achieved their status with hot take dunks on others, but that tactic doesn't really work on any other social network.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#159

Earlier quoted context omitted.

> However, it does show how much real compute bloat these companies actually have No, it doesn’t. It’s a fun exercise in approaching Twitter as an academic exercise. It ignores all of the real-world functionality that makes it a business rather than a toy. A lot of complicated businesses are easy to prototype out if you discard all requirements other than the core feature. In the real world, more engineering work oft…

Genuinely asking, why do you think Twitter needs 24 million vcpus to run? This is not apples to apples but Whatsapp is a product that entirely ran on 16 servers at the time of acquisition (1.5 billion users). It really begs the question why Twitter uses so much compute if there are companies that have operated significantly more efficiently. Twitter was unprofitable during acquisition and spent around half their reve…

Chat apps are mostly one on one interaction, it is much harder run an open platform where every user can potentially interact with every other user, not even talking about search and how complex it gets. If Twitter is bloated or not is a valid discussion, but comparison it to WhatsApp is not.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#160
post #41

Earlier quoted context omitted.

I specifically assumed a max tweet size based on the maximum number of UTF-8 bytes a tweet can contain (560), with a link to an analysis of that, and discussion of how you could optimize for the common case of tweets that contain way fewer UTF-8 bytes than that. Everything in my post assumes unicode.

Did you consider URLs? They don't seem to count against tue size and can be very large indeed (like 4k)

URLs are shortened and the shortened size counts against the tweet size. The URL shortener could be a totally separate service that the core service never interacts with at all. Though I think in real twitter URLs may be partially expanded before tweets are sent to clients, so if you wanted to maintain that then the core service would need to interact with the URL shortener.
Post reply on HN