Live data from Hacker News

Video streaming at scale with Kubernetes and RabbitMQ

alexandreolive.medium.com

11–20 of 100 posts

Re: Video streaming at scale with Kubernetes and RabbitMQ

#11

Must be expensive to run on Google Cloud. Also looks pretty complex. The stabilization step presumably does a video encode …. that’s extremely expensive in terms of time, compute and money I wonder why it’s necessary.

I wonder if it wouldn't be cheaper to run an on-prem farm of BestBuy-grade "gamer PC" for smaller scale networks like that.

Slap one of these puppies in….

AMD Alveo MA35D Media Accelerator

https://www.xilinx.com/applications/data-center/video-imagin...

Re: Video streaming at scale with Kubernetes and RabbitMQ

#12

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

Re: Video streaming at scale with Kubernetes and RabbitMQ

#13
post #9

This is nice if you only have to deliver in one format, but as soon as you want to show up on TVs you are stuck delivering in a lot of formats, and life gets complicated quickly. Throw subtitles in multiple languages, and different audio tracks, into the mix, and all of a sudden streaming video becomes a nightmare. Finally, if you are dealing with copyrighted materials, you have to be aware as to what country your us…

Oh, this is just the tip of the iceberg. Many parts of on-demand video streaming are largely commoditized at this point. Add in support for linear (live) streaming and ad insertion and things start to get really interesting. :)

Re: Video streaming at scale with Kubernetes and RabbitMQ

#14

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

I believe loads of auxiliary microservices have been omitted for brevity. Of course, those also don’t require Kubernetes, but maybe they have some standardised deployment system which keeps things manageable. Don’t forget about Observability and whatnot.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#15
post #9

This is nice if you only have to deliver in one format, but as soon as you want to show up on TVs you are stuck delivering in a lot of formats, and life gets complicated quickly. Throw subtitles in multiple languages, and different audio tracks, into the mix, and all of a sudden streaming video becomes a nightmare. Finally, if you are dealing with copyrighted materials, you have to be aware as to what country your us…

"fancy asset catalog management system" - was thinking about building such solution lately - do you know any open-source solutions of this kind?

Re: Video streaming at scale with Kubernetes and RabbitMQ

#18

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

[deleted]

Re: Video streaming at scale with Kubernetes and RabbitMQ

#19

I thoroughly appreciated this article as I've been building a short-form video content streaming service and the performance hasn't been what I expected. Granted, I knew that my service needs to be able to scale at different bottlenecks, but a lot of "build your own video service!" tutorials start with: - Build a backend, return a video file - Build a frontend, embed the video And that leaves a lot to be desired in t…

I helped work on howvideo.works, fun to see it helping people! The world of video is, I'd argue, one of those technical spaces that is extremely iceberg-y. You can get decently far enough using S3 + the HTML5 video tag, which I think creates a perception among some that video is just images but a little bigger, but that couldn't be further from the truth. You can really pick just about any step along the video pipeline from production to playback and go as deep for as many years as you'd like.

This is both a semi-shameless plug and probably a few levels deeper than what you're looking for, but I organize a conference for video developers called Demuxed. The YouTube channel[1] has 8 years worth of conference videos about streaming video (and the 9th year is happening in a couple of weeks). The bullet points you mentioned are definitely covered across a few talks, but it's certainly not in any kind of "how to" format.

[1]: https://youtube.com/demuxed

Re: Video streaming at scale with Kubernetes and RabbitMQ

#20
post #3

Must be expensive to run on Google Cloud. Also looks pretty complex. The stabilization step presumably does a video encode …. that’s extremely expensive in terms of time, compute and money I wonder why it’s necessary.

I was thinking the same. CF on the front would improve on it but still. Hetzner or other bare metal providers would probably be a better idea.

CF meaning Cloudflare? If you’re serving video through them, then you’re in “enterprise plan” territory. You can’t do that on the free or “self-serve” paid plans. $5k+/m depending on bandwidth needs (and if you just need a cdn to push bits, CF won’t be competitive on price—their enterprise prices are tailored for companies that want all sorts of managed services and private networking stuff)
Post reply on HN