Every time a big company screws up, there are two highly informed sets of people who are guaranteed to be lurking, but rarely post, in a thread like this: 1) those directly involved with the incident, or employees of the same company. They have too much to lose by circumventing the PR machine. 2) people at similar companies who operate similar systems with similar scale and risks. Those people know how hard this is a…
The only time I worked on a project that had a live television launch, it absolutely tipped over within like 2 minutes, and people on HN and Reddit were making fun of it. And I know how hard everyone worked, and how competent they were, so I sympathize with the people in these cases. While the internet was teeing off with easy jokes, engineers were swarming on a problem that was just not resolving, PMs were pacing up…
Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
751–760 of 986 posts
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#752The way to deal with this is to constantly do live events, and actually build organizational muscle. Not these massive one off events in an area the tech team has no experience in.
I have this argument a lot in tech. We should always be doing (the thing we want to do) Somme examples that always get me in trouble (or at least big heated conversations) 1. Always be building: It does not matter if code was not changed, or there has been no PRs or whatever, build it. Something in your org or infra has likely changed. My argument is "I would rather have a build failure on software that is already re…
"Test what you fly, and fly what you test" (Supposedly from aviation)
"There should be one joint, and it should be greased regularly" (Referring to cryptosystems I think, but it's the same principle. Things like TLS will ossify if they aren't exercised. QUIC has provisions to prevent this.)
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#753Every time a big company screws up, there are two highly informed sets of people who are guaranteed to be lurking, but rarely post, in a thread like this: 1) those directly involved with the incident, or employees of the same company. They have too much to lose by circumventing the PR machine. 2) people at similar companies who operate similar systems with similar scale and risks. Those people know how hard this is a…
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#754Earlier quoted context omitted.
It's not guaranteed, but much fewer points of failure.
> It's not guaranteed, but much fewer points of failure. Can you explain where this is relevant to buffering issues? Also, you are very wrong regarding failure modes. The larger the service, the more failure modes it has. Moreover, in monoliths if a failure mode can take down/degrade the whole service, all other features are taken down/degraded. Is having a single failure mode that brings down the whole service what…
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#755It's incomprehensible to me that Netflix, one of the most highly skilled engineering teams in the world - completely sh*t the bed last night and provided a nearly unwatchable experience that was not even in the same league as pre-internet live broadcast from 30 years ago.
I had buffering issues but then backed off and let a bit of it buffer up (maybe 1 or 2 mintues?) and then it was fine for the entire Tyson Paul match. There was no reason I needed it to be live vs. a 1 or 2 minute delay.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#756Every time a big company screws up, there are two highly informed sets of people who are guaranteed to be lurking, but rarely post, in a thread like this: 1) those directly involved with the incident, or employees of the same company. They have too much to lose by circumventing the PR machine. 2) people at similar companies who operate similar systems with similar scale and risks. Those people know how hard this is a…
Especially in a time where the gates have come crashing down to pronouncements of, "now anybody can learn to code by just using LLMs," there is a shocking tendency to overly simplify and then pontificate upon what are actually bewilderingly complicated systems wrapped up in interfaces, packages, and layers of abstraction that hide away that underlying complexity.
It reminds me of those quantum woo people, or movies like What the Bleep Do We Know!? where a bunch of quacks with no actual background in quantum physics or science reason forth from drastically oversimplified, mathematics-free models of those theories and into utterly absurd conclusions.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#757Earlier quoted context omitted.
For an event like this, there already exists an architecture that can handle boundless scale: torrents. If you code it to utilize high-bandwidth users upload, the service becomes more available as more users are watching -- not less available. It becomes less expensive with scale, more available, more stable. The be more specific, if you encode the video in blocks with each new block hash being broadcast across the n…
The protocol for a torrent is that random parts of a file get seeded to random people requesting a file, and that the clients which act as seeds are able to store arbitrary amounts of data to then forward to other clients in the swarm. Do the properties about scaling still hold when it's a bunch of people all requesting real time data which has to be in-order? Do the distributed Rokus, Apple TVs, Fire TVs and other s…
The problems with using it as part of a distributed service have more to do with asymmetric connections: using all of the limited upload bandwidth causes downloads to slow. Along with firewalls.
But the biggest issue: privacy. If I'm part of the swarm, maybe that means I'm watching it?
[1]: Chainsaw: P2P streaming without trees, https://link.springer.com/chapter/10.1007/11558989_12
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#758Earlier quoted context omitted.
It is weird because this was a solved problem. Every major network can broadcast the Super Bowl without issue. And while Netflix claims it streamed to 280 million, that’s if every single subscriber viewed it. Actual numbers put it in the 120 million range. Which is in line with the Super Bowl. Maybe Netflix needs to ask CBS or ABC how to broadcast
Do you live stream the superbowl? Me and everyone I know watch it over antenna broadcast tv. I think it is easier to have millions of tvs catch airwaves vs millions of point to point https video streams.
Exactly! It was a solved problem.
The Superbowl isn't even the biggest. World Cup finals bring in billions of viewers.
Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson
#759Earlier quoted context omitted.
For an event like this, there already exists an architecture that can handle boundless scale: torrents. If you code it to utilize high-bandwidth users upload, the service becomes more available as more users are watching -- not less available. It becomes less expensive with scale, more available, more stable. The be more specific, if you encode the video in blocks with each new block hash being broadcast across the n…
Torrents are awful for live events. 1. Everyone only cares about the most recent "block". By the time a "user" has fully downloaded a block from Netflix's seedbox, the block is stale, so why would any other user choose to download from a peer rather from netflix directly? 2. If all the users would prefer to download from netflix directly rather than a p2p user, then you already have a somewhat centralized solution, a…
But it does seem the capacity of a hybrid system of Netflix servers plus P2P would be strictly greater than either alone? It's not an XOR.
And note that in this case of "live" streaming, it still has a few seconds of buffer, which gives a bandwidth-delay product of a few MB. That's plenty to have non-stale blocks and do torrent-style sharing.