Live data from Hacker News

Production Twitter on one machine? 100Gbps NICs and NVMe are fast

thume.ca

41–50 of 500 posts

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#41
post #10

Earlier quoted context omitted.

My bad - not English, but ASCII. The assumed max tweet size is in bytes rather than (UTF) characters.

I specifically assumed a max tweet size based on the maximum number of UTF-8 bytes a tweet can contain (560), with a link to an analysis of that, and discussion of how you could optimize for the common case of tweets that contain way fewer UTF-8 bytes than that. Everything in my post assumes unicode.

Did you consider URLs? They don't seem to count against tue size and can be very large indeed (like 4k)

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#42
How much bandwidth does Twitter use for images and videos? Less than 1.4Tb/s globally? If so, we could probably fit that onto a second machine. We can currently serve over 700Gb/s from a dual-socket Milan based server[1]. I'm still waiting for hardware, but assuming there are no new bottlenecks, that should directly scale up to 1.4Tb/s with Genoa and ConnectX-7, given the IO pathways are all at least twice the bandwidth of the previous generation.

There are storage size issues (like how big is their long tail; quite large I'd imagine), but its a fun thing to think about.

[1] https://people.freebsd.org/~gallatin/talks/euro2022.pdf

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#43

Earlier quoted context omitted.

Kubernetes and containers are a means to service architecture; It enabled scalability but does not require it. You should still be containerizing your applications to ensure a consistent environment, even if you only throw it in a docker-compose file on your production server.

I don't even use containers - I aim primarily for simplicity and so far I have found I am able to build entire sophisticated systems without a single container. Containers I find make things much more complex.

Docker can be super simple. Like, if I want to run a Python service, that's just a few lines in a Dockerfile and a docker-compose.yml stub. Then I can trivially deploy that anywhere.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#44

Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…

> The thing most appealing about single server configs is the simplicity. The more simple a system easy, likely the more reliable and easy to understand.

What if your unique machine crash?

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#45

Earlier quoted context omitted.

>… without understanding how much capacity there is in horizontal. I think you mean vertical, right?

Ha ha yes I do! (corrected)

Freudian slip :)

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#47

Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…

> The thing most appealing about single server configs is the simplicity. The more simple a system easy, likely the more reliable and easy to understand. What if your unique machine crash?

[dead]

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#49

Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…

> The thing most appealing about single server configs is the simplicity. The more simple a system easy, likely the more reliable and easy to understand. What if your unique machine crash?

Well you gotta have a backup strategy. I'm talking about the primary machine here, I assumed that would be obvious but maybe not. You build your failover strategy into your architecture - there's lots of ways to do it - I use Postgres so I would favor something based around log shipping.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#50

Most projects I encounter these days instantly reach for kubernetes, containers and microservices or cloud functions. I find it much more appealing to just make the whole thing run on one fast machine. When you suggest this tend to people say "but scaling!", without understanding how much capacity there is in vertical. The thing most appealing about single server configs is the simplicity. The more simple a system ea…

> The thing most appealing about single server configs is the simplicity. The more simple a system easy, likely the more reliable and easy to understand. What if your unique machine crash?

You spin up a backup as the new "unique machine."
Post reply on HN