Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

sportingnews.com

Post reply on HN
Print view View original

571–580 of 986 posts

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#571

Post by ignoramous »

softwaredoug wrote: post #495

The way to deal with this is to constantly do live events, and actually build organizational muscle. Not these massive one off events in an area the tech team has no experience in.

> ...the tech team has no experience in

Unless Netflix eng decides to release a public postmorterm, we can only speculate. In my time organizing small-time live streams, we always had up to 3 parallel "backup" streams (Vimeo, Cloudflare, Livestream). At Netflix's scale, I doubt they could simply summon any of these providers in, but I guess Akamai / Cloudflare would have been up for it.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#572

Post by mplewis »

andai wrote: post #570
mbrumlow wrote: post #559

Earlier quoted context omitted.

I have this argument a lot in tech. We should always be doing (the thing we want to do) Somme examples that always get me in trouble (or at least big heated conversations) 1. Always be building: It does not matter if code was not changed, or there has been no PRs or whatever, build it. Something in your org or infra has likely changed. My argument is "I would rather have a build failure on software that is already re…

This is great, but what possible counterargument is there? We should prolong indefinitely a spooky ambiguity about whether the system works or not?

The common and flawed counterargument is “when we deploy, outages happen.” You’ll hear this constantly at companies with bad habits.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#573

Post by margaretdouglas »

deanCommie wrote: post #534
y-c-o-m-b wrote: post #526

Earlier quoted context omitted.

Amazon had their fair share of livestream failures and for notably less viewers. I don't think they deserve a spot on that list. I briefly worked in streaming media for sports and while it's not a novel problem, there are so many moving parts and points of failure that it can easily all go badly.

There is no one "Amazon" here, there are at least 3: * Twitch: Essentially invented live streaming. Fantastic. * Amazon Interactive Video Service [0]: Essentially "Twitch As A Service", built by Twitch engineers. Fantastic. * Prime Video. Same exact situation as Netflix: original expertise is all in static content. Lots of growing pains with live video and poor reports. But they've figured it out: now there are regul…

Doesn't twitch almost fall over (other non-massive streams impacted) when anyone gets close to 4-5m concurrent viewers? I remember last time it happened everything started falling over, even for smaller streams. Even if Netflix struggled with the event, streaming other content worked just fine for me.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#574

Post by ukuina »

andai wrote: post #570
mbrumlow wrote: post #559

Earlier quoted context omitted.

I have this argument a lot in tech. We should always be doing (the thing we want to do) Somme examples that always get me in trouble (or at least big heated conversations) 1. Always be building: It does not matter if code was not changed, or there has been no PRs or whatever, build it. Something in your org or infra has likely changed. My argument is "I would rather have a build failure on software that is already re…

This is great, but what possible counterargument is there? We should prolong indefinitely a spooky ambiguity about whether the system works or not?

Finite compute, people, and opportunity cost.

It is just a reframing of build vs maintain.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#575

Post by bostik »

diggan wrote: post #444
dylan604 wrote: post #426

People just do not appreciate how many gotchas can pop up doing anything live. Sure, Netflix might have a great CDN that works great for their canned content and I could see how they might have assumed that's the hardest part. Live has changed over the years from large satellite dishes beaming to a geosat and back down to the broadcast center($$$$$), to microwave to a more local broadcast center($$$$), to running ded…

> People just do not appreciate how many gotchas can pop up doing anything live. Sure thing, but also, how much resources do you think Netflix threw on this event? If organizations like FOSSDEM and CCC can do live events (although with way smaller viewership) across the globe without major hiccups on (relatively) tiny budgets and smaller infrastructure overall, how could Netflix not?

The CCC video crew has its fair share of geeks from broadcasting corporations and studio houses. Their combined institutional knowledge about live events and streaming distribution is probably in the same ballpark as that of giant global TV networks.

They also have the benefit of having practiced their craft at the CCC events for more than a decade. Twice a year. (Their summer event is smaller but still fairly well known. Links to talks show up on HN every now and then.)

Funky anecdote: the video crew at Assembly have more broadcasting and live AV gear for their annual event than most medium-sized studios.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#576

Post by margaretdouglas »

prasadjoglekar wrote: post #553
mfiguiere wrote: post #544

Earlier quoted context omitted.

The examples given here are not on the same scale. The numbers known so far: - 120m viewers [1] - Entire Netflix CDN Traffic grew 4x when the live stream started [2] [1] https://www.rollingstone.com/culture/culture-news/jake-paul-... [2] https://x.com/DougMadory/status/1857634875257294866

Disney HotStar managed to stream ~60M livestreams for the Cricket world cup a year ago. The problem has been solved. Livestreaming sports just have a different QoS expectations than on demand. https://www.geeksforgeeks.org/how-did-hotstar-managed-5-9-cr...

I wouldn't say it's a solved problem, how many other companies are pulling off those numbers? Isn't that the current record for concurrent streams? And wasn't it mostly to mobile devices?

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#577

Post by ukuina »

tempest_ wrote: post #567
bena wrote: post #446

Earlier quoted context omitted.

It is weird because this was a solved problem. Every major network can broadcast the Super Bowl without issue. And while Netflix claims it streamed to 280 million, that’s if every single subscriber viewed it. Actual numbers put it in the 120 million range. Which is in line with the Super Bowl. Maybe Netflix needs to ask CBS or ABC how to broadcast

When Netflix started it was the first in the space and breaking ground which is how they became a "tech" company that happens to stream media however it has been 15 years and since than the cloud providers have basically build "netflix as a service". I suspect most of the big streamers are using that instead of building their own in house thing and going through all the growing pains netflix is.

You know they were commoditized when "Build Netflix" became a system-design interview question.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#578

Post by qingcharles »

wordofx wrote: post #103
walrushunter wrote: post #56

I'm an engineering manager at a Fortune 500 company. The dumbest engineer on our team left for Netflix. He got a pay raise too. Our engineers are fucking morons. And this guy was the dumbest of the bunch. If you think Netflix hires top tier talent, you don't know Netflix.

None of the Fortune 500 companies hire top talent. They have a few good people but 98% of the engineers are average at best. Over paid.

This is every dev house I've worked at. For most people (mostly not the ones on HN), coding is a 9-5 job. No ambition. Just lines of code. Go home. I don't know there is anything particularly wrong with that.

You just have to accept most staff at any corporation are simply just average. There has to be an average for there to be better and worse.

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#579

Post by AlwaysRock »

bena wrote: post #446
dylan604 wrote: post #426

People just do not appreciate how many gotchas can pop up doing anything live. Sure, Netflix might have a great CDN that works great for their canned content and I could see how they might have assumed that's the hardest part. Live has changed over the years from large satellite dishes beaming to a geosat and back down to the broadcast center($$$$$), to microwave to a more local broadcast center($$$$), to running ded…

It is weird because this was a solved problem. Every major network can broadcast the Super Bowl without issue. And while Netflix claims it streamed to 280 million, that’s if every single subscriber viewed it. Actual numbers put it in the 120 million range. Which is in line with the Super Bowl. Maybe Netflix needs to ask CBS or ABC how to broadcast

Solves differently though, right? Cable broadcasts are not the same as a streaming video over the internet, right?

Re: Netflix buffering issues: Boxing fans complain about Jake Paul vs. Mike Tyson

#580

Post by toast0 »

dylan604 wrote: post #481
ironhaven wrote: post #457

Earlier quoted context omitted.

Do you live stream the superbowl? Me and everyone I know watch it over antenna broadcast tv. I think it is easier to have millions of tvs catch airwaves vs millions of point to point https video streams.

If you watch it over cable, you're live streaming it. Let's face it, that's where the vast majority of viewers see it. Few people view OTA even if the quality is better. Live sports do not broadcast the event directly to a streamer. They push it to their broadcast centers. It then gets distributed from there to whatever avenues it needs to go. Trying to push a live IP stream directly from the remote live venue rarely…

Is cable video over IP now? Last time I looked (which was forever ago), even switched video was atsc with a bit of messaging for the cable box to ask what channel to tune to, and to keep the stream alive. TV over teleco systems seems to be highly multicast, so kind of similar, headend only has to send the content once, in a single bitrate.

Not really the same as an IP service live stream where the distribution point is sending out one copy per viewer and participating in bitrate adaptation.

AFAIK, Netflix hasn't publicly described how they do live events, but I think it's safe to assume they have some amount of onsite production that outputs the master feed for archiving and live transcoding for the different bitrate targets (that part may be onsite, or at a broadcast center or something cloudy), and then goes to a distribution network. I'd imagine their broadcast center/or onsite processing feeds to a limited number of highly connected nodes that feed to most of their CDN nodes; maybe more layers. And then clients stream from the CDN nodes. Nobody would stream an event like this direct from the event; you've got to have something to increase capacity.

Post reply on HN