Cable TV (or even OTA antenna in the right service area) is simply a superior live product compared to anything streaming. The Masters app is the only thing that comes close imo. Cable TV + DVR + high speed internet for torrenting is still an unmatched entertainment setup. Streaming landscape is a mess. It's too bad the cable companies abused their position and lost any market goodwill. Copper connection direct to ev…
Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
761–770 of 986 posts
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#762Earlier quoted context omitted.
It's very hard to do a representative dry run when the most likely potential points of failure are highly load-dependent. You can try and predict everything that'll happen in production, but if you have nothing to extrapolate from, e.g. because this is your very first large live event, the chances of getting that right are almost zero. And you can't easily import that knowledge either, because your system might have…
They could have done a dry run. They could have spun up a million virtual machines somewhere, and tested their video delivery for 30 minutes. Even my small team spins up 10,000 EC2 instances on the regular. Netflix has the money to do much more. I'm sure there are a dozen ways they could have stress-tested this beforehand. It's not like someone sprang this on them last week and they had to scramble to put together a…
These would likely have completely different network connectivity and usage patterns, especially if they don't have historical data distributions to draw from because this was their first big live event.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#763Internet live streaming is harder than cable tv sattelite live streaming over "dumb" TV boxes cable. They should not have used internet for this honestly. A TV signal can go to millions live.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#764Earlier quoted context omitted.
hopefully their stock takes a big hit monday - these types only understand one thing
Are you serious? You think they don't care about this except if the stock 'takes a bit hit'??
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#765It's incomprehensible to me that Netflix, one of the most highly skilled engineering teams in the world - completely sh*t the bed last night and provided a nearly unwatchable experience that was not even in the same league as pre-internet live broadcast from 30 years ago.
My bet is that a technical manager told his executive (multiple times) that he needed more resources and engineering time to make live work properly, and they just told him to make do because they didn't want to spend the money. It could come down to something as stupid as: Executive: "we handled [on demand show ABCD] on day one, that was XX million" Engineering: "live is really different" Executive: (arguing about w…
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#766A see in the comments multiple people talking about how "cable" companies who have migrated to IPTV has solved this problem.
I'd disagree.
I'm on IPTV and any major sporting event (World Series, Super Bowl, etc) is horrible buffering when I try to watch on my 4K IPTV (streaming) channel. I always have to downgrade to the HD channel and I still occasionally experience buffering.
So Netflix isn't alone in this matter.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#767Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#768Every time a big company screws up, there are two highly informed sets of people who are guaranteed to be lurking, but rarely post, in a thread like this: 1) those directly involved with the incident, or employees of the same company. They have too much to lose by circumventing the PR machine. 2) people at similar companies who operate similar systems with similar scale and risks. Those people know how hard this is a…
For an event like this, there already exists an architecture that can handle boundless scale: torrents. If you code it to utilize high-bandwidth users upload, the service becomes more available as more users are watching -- not less available. It becomes less expensive with scale, more available, more stable. The be more specific, if you encode the video in blocks with each new block hash being broadcast across the n…
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#769Earlier quoted context omitted.
Why is live a lot harder? Aside from latency (which isn't much of a problem unless you are competing with TV or some other distribution system), it seems easier than on-demand, since you send the same data to everyone and don't need to handle having a potentially huge library in all datacenters (you have to distribute the data, but that's just like having an extra few users per server). My guess is that the problem w…
Latency is somewhat important for huge sporting events; you don't want every tense moment spoiled by the cheers of your neighbours whose feed is 20 seconds ahead. With on-demand you can push the episodes out through your entire CDN at your leisure. It doesn't matter if some bottleneck means it takes 2 hours to distribute a 1 hour show worldwide, if you're distributing it the day before. And if you want to test, or fi…
People using over the air antennas get it “live“. Getting it from cable or a streaming service meant anywhere between a few seconds and over a minute of delay.
It was absolutely common to have a friend text you about something that just happened when you haven’t even seen it yet.
You can’t even say that $some_service is fast, some of them vary over 60 seconds just between their own users.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#770This topic is really just fun for me to read based on where I work and my role. Live is a lot harder than on demand especially when you can't estimate demand (which I'm sure this was hard to do). People are definitely not understanding that. Then there is that Netflix is well regarded for their engineering not quite to the point of snobbery. What is actually interesting to me is that they went for an event like this…
Why is live a lot harder? Aside from latency (which isn't much of a problem unless you are competing with TV or some other distribution system), it seems easier than on-demand, since you send the same data to everyone and don't need to handle having a potentially huge library in all datacenters (you have to distribute the data, but that's just like having an extra few users per server). My guess is that the problem w…
* Encoding - low latency encoders are quite different than storage encoders. There is a tradeoff to be made in terms of the frequency of key frames vs. overall encoding efficiency. More key frames means that anyone can tune in or recover from a loss more quickly, but it is much less efficient, reducing quality. The encoder and infrastructure should emit transport streams, which are also less efficient but more reliable than container formats like mp4.
* Adaptation - Netflix normally encodes their content as a ladder of various codecs and bitrates. This ensures that people get roughly the maximum quality that their bandwidth will allow without buffering. For a live event, you need the same ladder, and the clients need to switch between rungs invisibly.
* Buffering - for static content, you can easily buffer 30 seconds to a minute of video. This means that small latency or packet loss spikes are handled invisibly at the transport/buffering layer. You can't do this for a live event, since that level of delay would usually be unacceptable for a sporting event. You may only be able to buffer 5-10 seconds. If the stream starts to falter, the client has only a few seconds to detect and shift to a lower rung.
* Transport - Prerecorded media can use a reliable transport like TCP (usually HLS). In contrast, live video would ideally use an unreliable transport like UDP, but with FEC (forward error correction). TCP's reaction to packet loss halves the receive window, which halves bandwidth, which would have to trash the connection to shift to a lower bandwidth rung.
* Serving - pre-recorded media can be synchronized to global DCs. Live events have to be streamed reliably and redundantly to a tree of servers. Those servers need to be load balanced, and the clients must implement exponential backoff or you can have cascading failures.
* Timing - Unlike pre-recorded media, any client that has a slightly fast clock will run out of frames and either need to repeat frames and stretch audio, or suffer glitches. If you resolve this on the server side by stretching the media, you will add complication and your stream will slowly get behind the live event.
* DVR - If you allow the users to pause, rewind, catch up, etc., you now have a parallel pre-recorded infrastructure and the client needs to transition between the two.
* DRM - I have no idea how/if this works on a live stream. It would not be ideal that all clients use the same decryption keys and have the same streams with the same metadata. That would make tracing the source of a pirate stream very difficult. Differentiation/watermarking adds substantial complexity, however.