Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video streaming at scale with Kubernetes and RabbitMQ

alexandreolive.medium.com

Post reply on HN
Print view View original

51–60 of 100 posts

Re: Video streaming at scale with Kubernetes and RabbitMQ

#51

Post by sass_muffin »

malux85 wrote: post #12
andrewstuart wrote: post #5

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t me…

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#52

Post by c7DJTLrn »

mannyv wrote: post #25

Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud. AWS is easier, but you can do it with anything. The basic steps are: 1. Upload the file somewhere 2. Transcode it 3. Put the parts somewhere 4. Serve the parts You should really transcode everything into HLS. It's 2023, and everything that matters supports it. If you want 4k you can use HLS or the other thing (which I keep…

If I'm understanding correctly you're suggesting using Lambda for processing. Is that a good idea? Lambda is actually expensive for heavy workloads.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#53

Post by cedws »

thomasjudge wrote: post #33

OP mentions that "I would love to be a little mouse and peek at YouTube’s complete architecture to see how far we are from them." You can occasionally find posts -often linked here- from another player in streaming video which you might have heard of, discussing technical architecture. For example, this might be a little lower level that you may be interested in as it relates to kernel optimizations to jack bit throu…

I've heard somewhere that YouTube has their own transcoding hardware.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#54

Post by jrockway »

malux85 wrote: post #45
robertlagrant wrote: post #44

Earlier quoted context omitted.

What happens when your storage detaches from your k8s cluster? Your services start 503ing, hopefully, because you didn't design your system thinking that k8s == 100% uptime.

Anybody can invent random problems ad nauseam - that doesn’t prove anything. I’m not claiming that it’s totally bullet proof, I never said that - I’m saying that if you had a kubernetes cluster anyway why not benefit from its abilities? Especially when the alternative is single node, single points of failure, which is clearly inferior. The "what if the storage detaches" argument could easily apply to the single node…

Yeah, what happens when someone in AWS clicks "delete database" accidentally? It's the same thing that happens when K8s blows up in some weird way. You restore from your backup. (Fun fact: deleting the instance deletes the backups!)

Re: Video streaming at scale with Kubernetes and RabbitMQ

#55

Post by pyrophane »

c7DJTLrn wrote: post #52
mannyv wrote: post #25

Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud. AWS is easier, but you can do it with anything. The basic steps are: 1. Upload the file somewhere 2. Transcode it 3. Put the parts somewhere 4. Serve the parts You should really transcode everything into HLS. It's 2023, and everything that matters supports it. If you want 4k you can use HLS or the other thing (which I keep…

If I'm understanding correctly you're suggesting using Lambda for processing. Is that a good idea? Lambda is actually expensive for heavy workloads.

I believe the Lambda would just trigger a MediaConvert process, so it would actually be doing very little.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#56

Post by c7DJTLrn »

pyrophane wrote: post #55
c7DJTLrn wrote: post #52

Earlier quoted context omitted.

If I'm understanding correctly you're suggesting using Lambda for processing. Is that a good idea? Lambda is actually expensive for heavy workloads.

I believe the Lambda would just trigger a MediaConvert process, so it would actually be doing very little.

Ah, got it, thanks.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#57

Post by EricBurnett »

cedws wrote: post #53
thomasjudge wrote: post #33

OP mentions that "I would love to be a little mouse and peek at YouTube’s complete architecture to see how far we are from them." You can occasionally find posts -often linked here- from another player in streaming video which you might have heard of, discussing technical architecture. For example, this might be a little lower level that you may be interested in as it relates to kernel optimizations to jack bit throu…

I've heard somewhere that YouTube has their own transcoding hardware.

Indeed. https://blog.youtube/inside-youtube/new-era-video-infrastruc... , https://research.google/pubs/pub50300/ . (Search the paper title and you should be able to find the pdf itself elsewhere).

I'm not actually sure on balance how much transcode gets done in hardware vs software, since it's also very amenable to using batch compute that's otherwise idle. I'll guess that most or all live transcoding - streams, on-the-fly transcode into formats not pregenerated - are done in hardware, and transcoding new formats for the back catalog are probably done on a mixture of mechanisms where and when capacity is available. (Source: Googler, not on YouTube though.)

Re: Video streaming at scale with Kubernetes and RabbitMQ

#58

Post by mikeryan »

hotnfresh wrote: post #20
tehlike wrote: post #3

Earlier quoted context omitted.

I was thinking the same. CF on the front would improve on it but still. Hetzner or other bare metal providers would probably be a better idea.

CF meaning Cloudflare? If you’re serving video through them, then you’re in “enterprise plan” territory. You can’t do that on the free or “self-serve” paid plans. $5k+/m depending on bandwidth needs (and if you just need a cdn to push bits, CF won’t be competitive on price—their enterprise prices are tailored for companies that want all sorts of managed services and private networking stuff)

Um. Cloudflare stream starts at $5 per month and you don’t pay for encoding only storage and bandwidth. You can serve a decent video library for $500 per month.

https://www.cloudflare.com/products/cloudflare-stream/

Re: Video streaming at scale with Kubernetes and RabbitMQ

#59

Post by whalesalad »

mannyv wrote: post #25

Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud. AWS is easier, but you can do it with anything. The basic steps are: 1. Upload the file somewhere 2. Transcode it 3. Put the parts somewhere 4. Serve the parts You should really transcode everything into HLS. It's 2023, and everything that matters supports it. If you want 4k you can use HLS or the other thing (which I keep…

serverless on aws is underrated if your workload can fit on it. i have an app that hasn't received a single ounce of maintenance in like 3 years that still "just works", collects revenue from stripe, does all the business logic on lambdas, generates downloadable print-friendly pdf's on lambdas, etc. the supporting tech is dynamo + triggers for lambda, s3 and related triggers for lambda. but it would be hard for a non-expert aws user to fathom this sort of architecture, so i don't fault others for falling down the nih rabbit hole.

Re: Video streaming at scale with Kubernetes and RabbitMQ

#60

Post by codenesium »

malux85 wrote: post #28
robertlagrant wrote: post #21

Earlier quoted context omitted.

Kubernetes loves stateless services. Zero wrong with moving RabbitMQ or a database outside of it.

Except kubernetes has a whole storage provisioning system that gives you redundancy and automatic failover, if you’re going to the trouble of running kubernetes why not just run your whole infra on it? I run https://atomictessellator.com solo, using kubernetes, and my database, Minio object store, application servers, quantum workers, everything is all on kubernetes, it’s self healing and much simpler to run all the…

Rabbit and most databases have their own failover strategy. Putting it all on k8s is fine for a toy app but idk why anyone would deploy a real system like that.
Post reply on HN