Scala is awesome language to implement scalable things, like Twitter services
Why?
The Architecture Twitter Uses to Deal with 150M Active Users
151–160 of 166 posts
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#152The surprise for me is that the core component is Redis. My first guess would have been custom C code. Yeah, you have to do everything yourself. Yeah, it would be hard to write. But you'd control every little bit of it. Obviously, I must not fully understand the problem and what Redis buys. Sam Puralla (if you are reading) -- do you know why didn't Twitter go with a full custom system at its heart? Josh
I'm probably a better person to answer this than Sam - I'm a former lead on this project - so I'll take a swing: We chose Redis because it gave us the specific, incremental improvements over our existing memcached-based system that we required, without requiring us to write (yet another) component. There was enough to do, and this choice has turned out to be good enough, I think. As the project progressed though, we…
Sounds like good engineering choices. I think of Twitter as having unlimited resources but, of course, that can't be true.
Two follow-on questions:
1. Did your changes make it back into open source or were they only relevant to Twitter? When you say upstream to you mean on Redis or earlier in the Twitter pipeline?
2. How much is Redis on the critical path? Is it 90% of the processing work in the large fanout cases?
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#153Re: The Architecture Twitter Uses to Deal with 150M Active Users
#154I really question the current trend of creating big, complex, fragile architectures to "be able to scale". These numbers are a great example of why, the entire thing could run on a single server, in a very straight forward setup. When you are creating a cluster for scalability, and it has less CPU, RAM and IO than a single server, what are you gaining? They are only doing 6k writes a second for crying out loud.
Two thoughts: scaling down can be just as important as scaling up, and cluster nodes that need to talk to each other are probably less common than stateless cluster nodes that need to talk to the client.
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#155Earlier quoted context omitted.
What's better about Redis? Many caching strategies require the storage to auto expire, memcache does this automatically. Redis does not do this (I believe). Is it significantly faster?
A simple Google search defeats your belief http://redis.io/commands/set > Options > EX seconds -- Set the specified expire time, in seconds. http://redis.io/commands/expireat > EXPIREAT has the same effect and semantic as EXPIRE, but instead of specifying the number of seconds representing the TTL (time to live), it takes an absolute Unix timestamp (seconds since January 1, 1970)
I apologize for not being specific. Setting timed expiration is not what I meant. Memcache will fill the entire cache and auto select items to be destroyed, based on current resource availability and order of entry into the cache (FIFO). I want that behaviour. I do not want to set and manage the time things expire.
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#156Earlier quoted context omitted.
That is actually a very interesting question and it turns out that whether it's better to fanout on write or on read depends on a few different things. There's a very widely read paper on the subject you might enjoy: http://research.yahoo.net/node/3203
Title? Link doesn't work.
Huh, weird. Works fine for me. Anyways:
Feeding Frenzy: Selectively Materializing Users’ Event Feeds Authors: Silberstein, A.; Terrace, J.; Cooper, B.F.; Ramakrishnan, R.
A quick Google should turn it up for you somewhere.
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#157Earlier quoted context omitted.
I'm probably a better person to answer this than Sam - I'm a former lead on this project - so I'll take a swing: We chose Redis because it gave us the specific, incremental improvements over our existing memcached-based system that we required, without requiring us to write (yet another) component. There was enough to do, and this choice has turned out to be good enough, I think. As the project progressed though, we…
Sounds like good engineering choices. I think of Twitter as having unlimited resources but, of course, that can't be true. Two follow-on questions: 1. Did your changes make it back into open source or were they only relevant to Twitter? When you say upstream to you mean on Redis or earlier in the Twitter pipeline? 2. How much is Redis on the critical path? Is it 90% of the processing work in the large fanout cases?
1. Yes, most everything we changed is in the open source Redis code base. That's what I'm referring to as upstream, above.
2. Redis is in the critical path for a majority of API requests. I can't provide a specific percentage.
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#158> Twitter knows a lot about you from who you follow and what links you click on. No kidding. But we don't care as we live in the glass house of a celebrity culture. PS: downvote the quoted text if you must. My point is not obvious.
You realize it's public data, right? When I tweet something, I don't expect it to be private. HN knows a lot about me too.
Yup, I mentioned glass houses.
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#159Earlier quoted context omitted.
Redis is memcache only better.
What's better about Redis? Many caching strategies require the storage to auto expire, memcache does this automatically. Redis does not do this (I believe). Is it significantly faster?
It's all the functionality of memcache, plus persistence and data types like lists or sorted sets. You have much more control of your data.
It's the swiss army knife of in-memory data stores.
Re: The Architecture Twitter Uses to Deal with 150M Active Users
#160Earlier quoted context omitted.
I have not used nginx, and have never heard of it spoken about in such glowing terms. Can you maybe compare/contrast it with apache?
> I have not used nginx, and have never heard of it spoken about in such glowing terms. Seriously? There are lots of glowing blog posts, articles etc about Nginx all the time, including on HN. Just last week or so, it was reported that it powers the majority of the top-1000 biggest sites.
Yeah I guess what I mean was the post I was replying to called it the most useful tool powering the internet. I have not seen that before. I also like to ask people on HN who seem extremely passionate about something to explain why they are passionate because you often learn something new, that you often would not from articles etc.