For me this link just opens the Medium app and fails to load the article. I had to force it to open in a browser.
Seems like a pretty bad Medium bug.
Keeping Netflix Reliable Using Prioritized Load Shedding
11–20 of 77 posts
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#12For me this link just opens the Medium app and fails to load the article. I had to force it to open in a browser. Seems like a pretty bad Medium bug.
Seems like pretty standard browser/app handover behaviour to me, although the app not working is a massive fail and should - hopefully - flag up automatically as a critical issue on Medium's side.
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#13A lot of websites will now fail requests early based on a timeout, forcing users to refresh the page. I have to wonder if ad-based sites enjoy this behavior because it could lead to more ad impressions. Talking about you reddit.
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#14When deciding what mechanism to employ to load shed, you should keep in mind the layer at which you are load shedding. Modern distributed systems are comprised of many layers. You can do it at the load balancer, at the OS level, or in the application logic. This becomes a trade-off. As you get closer to the core application logic, the more information you will have to make a decision. On the other hand, as you get closer, the more work you have already performed and the more cost there is to throwing away the request.
You may employ techniques more complex than a simple bucketing mechanism, such as acutely observing the degree at which clients are exceeding their baseline. However, these techniques aren’t free. The cost of simply throwing away the request can overwhelm your server - and the more steps you add before the shedding part the lower the maximum throughput you can tolerate before going to 0 availability. It’s important to understand at what point this happens when designing a system that takes advantage of this technique.
For example, If you do it at the OS level, it is a lot cheaper than leaving it to the server process. If you choose to do it in your application logic, think carefully about how much work is done for the request before it gets thrown away. Are you validating a token before you are making your decision?
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#15According to the diagram, Netflix is injecting chaos into the chaos control panel. Is that right? Looks like the arrow goes the wrong direction.
I’m wondering if it’s more of a “Chaos Injector” component/service that reads configuration data from the Chaos Control Plane on what to target, with parameters on how/when to do so. That would make the arrow make sense in my mind given it sounds like that’s a solid pattern for scaling these data/control plane flows: https://aws.amazon.com/builders-library/avoiding-overload-in...
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#16Totally anecdotal evidence, but I was in a rural NY house served by DSL for the past 6 months. The DSL has consistent packet loss between 4 and 6%. The only video service that could handle this level of packet loss well was Amazon Prime. Netflix couldn't even load its browse screen until the past two weeks, where something changed, and suddenly Netflix could handle the high packet loss as well as Amazon Prime. Thank…
Slightly unrelated but does the packet loss happen all the time or when close to maximum of the line.
Used to have similar problems with an ADSL line but found if I limited the line (Both up and down) I could find a magic number where the packet loss went away. (Well most of the time :))
Though it did need to be tuned for different times of they . ie high congestion times need it to be lower.
Though technically it shouldn't be your problem :(
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#17Totally anecdotal evidence, but I was in a rural NY house served by DSL for the past 6 months. The DSL has consistent packet loss between 4 and 6%. The only video service that could handle this level of packet loss well was Amazon Prime. Netflix couldn't even load its browse screen until the past two weeks, where something changed, and suddenly Netflix could handle the high packet loss as well as Amazon Prime. Thank…
Seperate annecdote - I worked on an inflight satelite wifi project and I was surprised at how well both Youtube and Netlifx worked over a medium-bandwidth/high-latency connection.
Granted, we had specific QoS/traffic shaping to improve reliability without gobbling up all the bandwidth (stream Netflix was an advertised feature of the wifi service), but it still seemed like magic.
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#18I love the phrase "Prioritized Load Shedding" as corporate-speak for "dropping the less-important traffic."
How is it corporate-speak? Sounds just like standard thoughtful naming. If I was working on a module that did this I would be happy to name it this even if it never got mentioned in any corporate context.
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#19So you can drop all that traffic and the users are unimpacted... So why not just always drop that traffic and don't even bother writing the code to implement those features that clearly nobody cares about?
That's like saying "if you can drive a car without an air-conditioner, why even bother installing an air-conditioner?"
Re: Keeping Netflix Reliable Using Prioritized Load Shedding
#20Totally anecdotal evidence, but I was in a rural NY house served by DSL for the past 6 months. The DSL has consistent packet loss between 4 and 6%. The only video service that could handle this level of packet loss well was Amazon Prime. Netflix couldn't even load its browse screen until the past two weeks, where something changed, and suddenly Netflix could handle the high packet loss as well as Amazon Prime. Thank…
> Netflix couldn't even load its browse screen until the past two weeks
I assume the browse screen is based entirely on TCP?
I'm struggling to understand why packet loss would prevent it from loading -- it should be slower but TCP should handle re-transmission, no?
Or is Netflix doing something tricky with UDP even in their browsing UX?