Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
1–10 of 10 posts
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#2I thought they outsourced it all to Amazon
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#3Ants are almost brainless, yet the ant hills are massive and complex. Simple rules and endless repetition. End result giant fractals.
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#4Any insight why they need so many clusters?
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#5Any insight why they need so many clusters?
microservices
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#6Any insight why they need so many clusters?
they probably have different clusters for different use cases: att big companies, this will lead to a very long tail of small clusters.
cassandra struggles to scale horizontally after a while so i am guessing that they also do some sort of application level sharding across multiple clusters
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#7There was a time when this would have impressed me, a bit like learning that Facebook had one ops engineer for every ten thousand servers they had. But it just makes me sad now: we put the best minds of a generation to work and for what? Netflix is a shadow of itself. The UI is a slurry pond of whatever the algorithm decides to toss up. The content is dire. I just feel so sad about what we built.
Its not just Netflix of course. I just can't shake the feeling that we've built entirely the wrong future.
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#822k nodes for 12M req/s?!
Seriously, what the hell. You can have 1M req/s on a good mid-range VPS or bare-metal server. Fine, have 3-4 for redundancy.
Even with that, you'd be more than well served with 50 strong nodes.
Why so many?
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#922k nodes for 12M req/s?! Seriously, what the hell. You can have 1M req/s on a good mid-range VPS or bare-metal server. Fine, have 3-4 for redundancy. Even with that, you'd be more than well served with 50 strong nodes. Why so many?
Probably for planet scale replication. A single instance in Ireland can be very fast for the people in Ireland, both in terms of req/s and latency, but the latency will be abysmal in, say, Australia.
Re: Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s
#10There was a time when this would have impressed me, a bit like learning that Facebook had one ops engineer for every ten thousand servers they had. But it just makes me sad now: we put the best minds of a generation to work and for what? Netflix is a shadow of itself. The UI is a slurry pond of whatever the algorithm decides to toss up. The content is dire. I just feel so sad about what we built. Its not just Netflix…
I am Anti-Apple in the recent years, but I have to give them credit as they seems to be the only one in the industry who still give a shit about UX. ( Even though they are losing it somewhat )
> I just can't shake the feeling that we've built entirely the wrong future.
The others, Microsoft, Facebook, Google, Netflix. Pretty much the whole Tech industry are exceptionally poor at it. And there is something fundamentally wrong with Tech companies. And I am not entirely sure what it is.