Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video streaming at scale with Kubernetes and RabbitMQ

alexandreolive.medium.com

Post reply on HN
Print view View original

71–80 of 100 posts

Re: Video streaming at scale with Kubernetes and RabbitMQ

#71

Post by akbansa »

I am sure it's difficult for someone to build and scale video infrastructure. A few companies are doing it for you; plug in the APIs, and you're done.

Gumlet (https://www.gumlet.com): Per-title encoding (Netflix's approach) to optimize and transcode your videos to boost engagement rates. Moreover, securing your videos is easy with digital rights management solutions paired with Widevine and Fairplay. Made for developers, by developers.

Mux: Developer-friendly video infrastructure for your on-demand & live video needs.

I love Gumlet because of their pricing and support.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#72

Post by timc3 »

jackcviers3 wrote: post #70
dbrueck wrote: post #13

Earlier quoted context omitted.

Oh, this is just the tip of the iceberg. Many parts of on-demand video streaming are largely commoditized at this point. Add in support for linear (live) streaming and ad insertion and things start to get really interesting. :)

Article made me nostalgic. My very first job was writing streaming media services for radio. Integrating the ad services over events embedded into the stream identifying content was a pretty nifty solution. You send the metadata to your ad server, it selects the ads based on your user and media metadata, you swap out the player's media url to play the ad(s), and keep the stream going in the background. When the comme…

In 2007 and i would argue before that, live media encoding for streams was pretty much a solved problem but it was often very expensive, and out of reach of radio stations and the information wasn't readily accessible so you could potentially end up giving Microsoft or Real networks lots of money.

I was working on streaming in the late 90s - that was real wild west, had to write solutions myself, and it felt very duck taped together. Luckily it was an offshoot of a largish ISP so we had bandwidth and a lot of servers. incoming streams often used bonded ISDN lines.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#73

Post by timc3 »

grzes wrote: post #15
com2kid wrote: post #9

This is nice if you only have to deliver in one format, but as soon as you want to show up on TVs you are stuck delivering in a lot of formats, and life gets complicated quickly. Throw subtitles in multiple languages, and different audio tracks, into the mix, and all of a sudden streaming video becomes a nightmare. Finally, if you are dealing with copyrighted materials, you have to be aware as to what country your us…

"fancy asset catalog management system" - was thinking about building such solution lately - do you know any open-source solutions of this kind?

I’ve built or been involved in building 4 closed source media asset management systems ( including my current place of work and one I cofounded) and you can get relatively far in the first month.

But the devil is in the details and most people that attempt it don’t even get the database models right on the first or second time, they underestimate the complexity particularly when it comes to video and struggle when the use cases widen. Document asset management systems are way easier but usually don’t understand media as in depth as a media asset management system.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#74

Post by littlestymaar »

schott12521 wrote: post #6

I thoroughly appreciated this article as I've been building a short-form video content streaming service and the performance hasn't been what I expected. Granted, I knew that my service needs to be able to scale at different bottlenecks, but a lot of "build your own video service!" tutorials start with: - Build a backend, return a video file - Build a frontend, embed the video And that leaves a lot to be desired in t…

> Video Upload / Processing Service that saves the video into chunks

At this point you also need to chose what streaming protocol you want to use. You have mostly two choices, HLS if you want to get things done quick, or MPEG-DASH if you want more control (but you'd need a separate HLS pipeline for iOS anyway…)

> Build or use a video streaming library that can play video chunks as a stream

As someone who's worked on a web streaming player, I'd strongly recommend not to build one but to use an existing one (or, in short: use HLS.js)

Re: Video streaming at scale with Kubernetes and RabbitMQ

#75

Post by malux85 »

codenesium wrote: post #60
malux85 wrote: post #28

Earlier quoted context omitted.

Except kubernetes has a whole storage provisioning system that gives you redundancy and automatic failover, if you’re going to the trouble of running kubernetes why not just run your whole infra on it? I run https://atomictessellator.com solo, using kubernetes, and my database, Minio object store, application servers, quantum workers, everything is all on kubernetes, it’s self healing and much simpler to run all the…

Rabbit and most databases have their own failover strategy. Putting it all on k8s is fine for a toy app but idk why anyone would deploy a real system like that.

OK, I can only speak to my personal projects and 20+ years experience at work.

We run all of our stateful and stateless workloads on 10+ kubernetes clusters at work in multiple datacenters in multiple continents, and we serve 500 million users a month with it.

I wrote the first BORG version of DFP backend systems at Google, where we served billions of users billions of ads a day, and we used stateful infrastructure management on some of the first container runtime systems that inspired k8s during it's development.

Using rabbit and "most databases" native fallover strategy is fine for toy projects, but when you're operating at this scale, you need automated infrastructure provisioning and all of the automated tooling around it.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#76

Post by [deleted] »

dilyevsky wrote: post #63
jiggawatts wrote: post #31

Earlier quoted context omitted.

This post is somewhat unfairly voted down. Cloud services like S3 and Azure Storage were invented specifically for hosting images and video. That’s their origin story, their foundation, their very reason for being. Similarly, cloud functions / lambda were invented for background processing of blobs. The first demos were always of resizing images! Building out this infrastructure yourself is a little insane. Unless yo…

I actually worked on system just like OP is describing and we ran everything ourselves including DBs (on k8s) so I can offer some perspective. For egress heavy services such as video minimizing outrageous cloud egress fees is key. Having an ability to tell your cloud account manager you can easily run multi-cloud or on-prem makes it a lot easier to negotiate sane rates. Ultimately you’re competing with the likes of T…

[deleted]

Re: Video streaming at scale with Kubernetes and RabbitMQ

#77

Post by wouldbecouldbe »

schott12521 wrote: post #6

I thoroughly appreciated this article as I've been building a short-form video content streaming service and the performance hasn't been what I expected. Granted, I knew that my service needs to be able to scale at different bottlenecks, but a lot of "build your own video service!" tutorials start with: - Build a backend, return a video file - Build a frontend, embed the video And that leaves a lot to be desired in t…

I've been using commercial streaming services for in app (cloudflare, bunny, vimeo), and found performance & bandwith use terrible. The HLS protocol for iOS doesnt work wel for 5-10 second clips, since it needs 1-4 seconds. Now using compress mp4 with progressive loading. Way better.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#78

Post by ddorian43 »

wouldbecouldbe wrote: post #77
schott12521 wrote: post #6

I thoroughly appreciated this article as I've been building a short-form video content streaming service and the performance hasn't been what I expected. Granted, I knew that my service needs to be able to scale at different bottlenecks, but a lot of "build your own video service!" tutorials start with: - Build a backend, return a video file - Build a frontend, embed the video And that leaves a lot to be desired in t…

I've been using commercial streaming services for in app (cloudflare, bunny, vimeo), and found performance & bandwith use terrible. The HLS protocol for iOS doesnt work wel for 5-10 second clips, since it needs 1-4 seconds. Now using compress mp4 with progressive loading. Way better.

> and found performance & bandwith use terrible

Can you explain exactly what you mean?

Re: Video streaming at scale with Kubernetes and RabbitMQ

#79

Post by KronisLV »

pyrophane wrote: post #46
andrewstuart wrote: post #5

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

Why do you say RabbitMQ is heavyweight? What queues do you consider more lightweight and what would be your go-to in a situation like this?

They might be thinking of something like ZeroMQ, which is pretty well liked: https://zeromq.org/

That said, I wouldn't call RabbitMQ that heavyweight myself, at least when compared to something like Apache Kafka.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#80

Post by KronisLV »

sass_muffin wrote: post #51
malux85 wrote: post #12

Earlier quoted context omitted.

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes.

> It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes.

Is that still true?

I wouldn't call the parent comment charitable enough, because there definitely can be some reasons for running stateful workloads even outside of containers altogether (familiarity included), but at the same time it feels like a lot of effort has been invested into making that a non-issue.

For example, how many database Operators are now available for Kubernetes: https://operatorhub.io/?category=Database&capabilityLevel=%5...

Honestly, as long as you have storage and config setup correctly, it's not like you even need an Operator, that's for more advanced setups. I've been running databases in containers (even without Kubernetes) for years, haven't had that many issues at small/medium scale.

Post reply on HN