Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video streaming at scale with Kubernetes and RabbitMQ

alexandreolive.medium.com

Post reply on HN
Print view View original

21–30 of 100 posts

Re: Video streaming at scale with Kubernetes and RabbitMQ

#21

Post by robertlagrant »

malux85 wrote: post #12
andrewstuart wrote: post #5

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

Kubernetes loves stateless services. Zero wrong with moving RabbitMQ or a database outside of it.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#22

Post by goeiedaggoeie »

andrewstuart wrote: post #11
klaussilveira wrote: post #7

Earlier quoted context omitted.

I wonder if it wouldn't be cheaper to run an on-prem farm of BestBuy-grade "gamer PC" for smaller scale networks like that.

Slap one of these puppies in…. AMD Alveo MA35D Media Accelerator https://www.xilinx.com/applications/data-center/video-imagin...

Ive used xilinx a fair bit for encoding. once you get past the pain of compiling your tooling for it it does speed up VOD encode significantly.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#23

Post by baq »

malux85 wrote: post #12
andrewstuart wrote: post #5

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

There some of us who still perform four extra steps before putting any DB in k8s and we have good reasons.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#24

Post by dvliman »

I built a similar video pipeline, not on Kubernetes but using EC2 instances for those hungry FFMPEG encoder.

The system differ in that it was not user generated video content. It was coming from the cameras in our fitness studio.

Here is the article if anyone intereste to read about: https://dev.to/dvliman/building-a-live-streaming-app-in-cloj...

Re: Video streaming at scale with Kubernetes and RabbitMQ

#25

Post by mannyv »

Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud.

AWS is easier, but you can do it with anything. The basic steps are:

1. Upload the file somewhere 2. Transcode it 3. Put the parts somewhere 4. Serve the parts

You should really transcode everything into HLS. It's 2023, and everything that matters supports it. If you want 4k you can use HLS or the other thing (which I keep forgetting the acronym for).

If you want to get fancy you can do rendition audio, which not everything supports. Rendition audio means sharing one audio stream amongst N number of video streams.

You can use FFMPEG to transcode, but I'd suggest using AWS MediaConvert. It's cheap, fast, and probably does everything you want. Using FFmpeg directly works, but why bother. You will get an option wrong and screw everything up. You don't want your video to not work on some random device that 50k people are using in some country you didn't think about.

He's using RabbitMQ but you should use SQS, because SQS can trigger lambdas...which means no polling required. But use whatever queue you want.

You can kick the process off by attaching a Lambda to S3, which will start the process when the file is uploaded.

You can kick your "availability activation" off by attaching a Lambda to the S3 output bucket.

Background: I help run a streaming service and built the backend pipeline.

This omits the entire "metadata management and analytics" side as well. That's left as an exercise for the user.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#26

Post by alexandreolive »

andrewstuart wrote: post #2

Must be expensive to run on Google Cloud. Also looks pretty complex. The stabilization step presumably does a video encode …. that’s extremely expensive in terms of time, compute and money I wonder why it’s necessary.

Hello, I'm the writer of the article. Our solution gets videos from random people who present products we sent them. We get dodgy videos filmed on bad devices, and the process of contacting the user and getting him to re-upload another video in better quality is time-consuming for our team. We'd rather spend a little bit more in computing to try and save time overall. I hope this answers your question.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#27

Post by jonnycoder »

mannyv wrote: post #25

Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud. AWS is easier, but you can do it with anything. The basic steps are: 1. Upload the file somewhere 2. Transcode it 3. Put the parts somewhere 4. Serve the parts You should really transcode everything into HLS. It's 2023, and everything that matters supports it. If you want 4k you can use HLS or the other thing (which I keep…

What would you recommend using as an alternative to being locked into AWS?

Re: Video streaming at scale with Kubernetes and RabbitMQ

#28

Post by malux85 »

robertlagrant wrote: post #21
malux85 wrote: post #12

Earlier quoted context omitted.

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

Kubernetes loves stateless services. Zero wrong with moving RabbitMQ or a database outside of it.

Except kubernetes has a whole storage provisioning system that gives you redundancy and automatic failover, if you’re going to the trouble of running kubernetes why not just run your whole infra on it?

I run https://atomictessellator.com solo, using kubernetes, and my database, Minio object store, application servers, quantum workers, everything is all on kubernetes, it’s self healing and much simpler to run all the infrastructure the same.

Recently I had a node failure while I was sleeping and the whole system healed itself while I slept, the monitoring system didn’t even alarm me because the small blip of increased latency while the pods rebalanced wasn’t above the alert threshold so it didn’t even wake me up.

What happens in the article infra when the rabbitmq or database nodes fail? The whole system goes offline, which seems very silly setup when you have kubernetes sitting right there, who’s primary function is to handle all of this.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#29

Post by alexandreolive »

andrewstuart wrote: post #5

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

Hello, I'm the writer of the article. We are using Kubernetes for our whole architecture, consisting of around 40 microservices and cron jobs. I just wanted in this article to give an example of asynchronous architecture using Kubernetes and RabbitMQ.

We are using RabbitMQ because it's my company target solution. There might better so lighter solution that would fit us but having just one for every solution is easier to maintain.

Great comment about GPU hardware acceleration for encoding, I'm going to look this up.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#30

Post by alexandreolive »

mmcclure wrote: post #19
schott12521 wrote: post #6

I thoroughly appreciated this article as I've been building a short-form video content streaming service and the performance hasn't been what I expected. Granted, I knew that my service needs to be able to scale at different bottlenecks, but a lot of "build your own video service!" tutorials start with: - Build a backend, return a video file - Build a frontend, embed the video And that leaves a lot to be desired in t…

I helped work on howvideo.works, fun to see it helping people! The world of video is, I'd argue, one of those technical spaces that is extremely iceberg-y. You can get decently far enough using S3 + the HTML5 video tag, which I think creates a perception among some that video is just images but a little bigger, but that couldn't be further from the truth. You can really pick just about any step along the video pipeli…

I'm the writer of the article; I LOVE howvideo.works. It helped me quite a lot when I started working on video processing. I'm still a beginner and always fall back to it when I'm unsure about something fundamental. Thanks for your work. I'll take a look at your YouTube channel.
Post reply on HN