Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
7 Years Of YouTube Scalability Lessons In 30 Minutes
31–40 of 77 posts
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#32Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#33What's this? Python? Apache? MySQL? But I thought you had to be running beta-release key value stores, esoteric web servers, and experimental programming languages if you wanted to scale! /s
The fastest, easiest, arguably most reliable way to scale is throwing money at it. And apparently it's easier to hire good people who know 20-year old tech rather than 3-year old.
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#34Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#35"Dummer code is easier to grep for and easier to maintain. The more magical the code is the harder is to figure out how it works." A nice formulation of the kind of advice I keep reading here in HN.
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#36Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
The other way to solve the problem though would be to handle that circumstance cleanly. There are ways to resolve a thundering herd without creating a scalability problem.
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#37Earlier quoted context omitted.
Is RPC an actual technology? I thought it was more of a protocol design pattern.
Remote Procedure Call is a design paradigm for synchronous call-and-response network communication. The Sun RPC protocol is an actual technology defined in RFC1057: http://www.ietf.org/rfc/rfc1057.txt It's not insane, though not terribly relevant in the modern world. The only common technology still using it is NFS.
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#38Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#39Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
Re: 7 Years Of YouTube Scalability Lessons In 30 Minutes
#40"Dummer code is easier to grep for and easier to maintain. The more magical the code is the harder is to figure out how it works." A nice formulation of the kind of advice I keep reading here in HN.