71–80 of 500 posts
Earlier quoted context omitted.
Kubernetes is useful if you have many teams working on things in parallel and you want them to deploy in similar ways to not have to reinvent the same wheel in 5 different ways by 5 different teams. If you don't have multiple teams, you don't need it.
It's also useful if you want your app to update without being down, which even a single team might want to do.
I see a lot of comments here assuming that this proves something about Twitter being inefficient. Before you jump to conclusions, take a look at the author’s code: https://github.com/trishume/twitterperf
Notably absent are things like serving HTTP, not to even mention HTTPS. This was a fun exercise in algorithms, I/O, and benchmarking. It wasn’t actually imitating anything that resembles actual Twitter or even a usable website.
Earlier quoted context omitted.
On the other hand, Twitter does (or did) handle over 450 million monthly active users (based on stats websites), with a target for 315 monetizable daily active users (based on their earnings calls pre-privatization). Handling that amount of concurrency and beaming millions of tweets a day to home feeds and notifications is going to be logistically hard.
Is that 315 million monetizable DAUs? That sounds like a lot if the total is only 450 MAU. OTOH, 315k DAU seems like it wouldn't be enough to pay the bills.
They were targeting 315 mDAUs for Q4 2023, but in the final earnings it was only 238 mDAUs. Actual MAU stats weren't public iirc but some random stats sites seemed to say 450m global MAUs, which likely includes people with no ad preferences or who only view NSFW content (which can't be shown next to (most?) ads).
https://www.forbes.com/sites/johnkoetsier/2022/11/14/twitter...
Earlier quoted context omitted.
My friend mentioned this just before I published and I think that probably is the fastest largest thing you can get which would in some sense count as one machine. I haven't looked into it, but I wouldn't be surprised if they could get around the trickiest constraint, which is how many hard drives you can plug in to a non-mainframe machine for historical image storage. Definitely more expensive than just networking a…
> I wouldn't be surprised if they could get around the trickiest constraint, which is how many hard drives you can plug in to a non-mainframe machine for historical image storage. Some commodity machines use external SAS to connect to more disk boxes. IMHO, there's not a real reason to keep images and tweets on the same server if you're going to need an external disk box anyway. Rather than getting a 4u server with a…
Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…
Because OP example is very simplistic and left on the table very important details, you would base 250M on a single machine? What about backups, obervability, how do you update that stack without bringing down everything ... Also this is napkin maths, this could be off by 10 or 100x which would change everything. It's very simple to make a PoC on a very powerful machine, make it ready from production serving hunderd…
Several ways of doing this without relying on k8s
> observability
This doesn't require k8s neither and it's more on your app. Systemd can restart systems by itself
> how do you update that stack without bringing down everything
That's probably where redundancy helps the most. I wouldn't run a big service without it (but again it found be at server level)
If you really wanted to run Twitter on one machine at any cost, wouldn't an IBM mainframe be much more practical? You can even run Linux on them now. The specs he cites would actually be fairly small for a mainframe, which can reach up to 40TB of memory. I'm not saying this is a good idea, but it seems better than what the OP proposes.
My friend mentioned this just before I published and I think that probably is the fastest largest thing you can get which would in some sense count as one machine. I haven't looked into it, but I wouldn't be surprised if they could get around the trickiest constraint, which is how many hard drives you can plug in to a non-mainframe machine for historical image storage. Definitely more expensive than just networking a…
Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…
Because OP example is very simplistic and left on the table very important details, you would base 250M on a single machine? What about backups, obervability, how do you update that stack without bringing down everything ... Also this is napkin maths, this could be off by 10 or 100x which would change everything. It's very simple to make a PoC on a very powerful machine, make it ready from production serving hunderd…
Nobody should be looking at this and thinking that it’s realistic to actually serve a functional website at this scale on a single machine with actual real world requirements.
Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…
Because OP example is very simplistic and left on the table very important details, you would base 250M on a single machine? What about backups, obervability, how do you update that stack without bringing down everything ... Also this is napkin maths, this could be off by 10 or 100x which would change everything. It's very simple to make a PoC on a very powerful machine, make it ready from production serving hunderd…
Earlier quoted context omitted.
Because OP example is very simplistic and left on the table very important details, you would base 250M on a single machine? What about backups, obervability, how do you update that stack without bringing down everything ... Also this is napkin maths, this could be off by 10 or 100x which would change everything. It's very simple to make a PoC on a very powerful machine, make it ready from production serving hunderd…
In addition, you should worry about what happens to your app if a hardware error, network problem or natural disaster makes your machine unavailable.