The Evolution of Container Usage at Netflix • Hacker News Board

Re: The Evolution of Container Usage at Netflix

#141

Post by lima » Fri, Apr 21, 2017, 10:39 PM UTC

jakon89 wrote: post #95

lima wrote: post #48

Earlier quoted context omitted.

Containers don't actually run directly on hardware with GCE - there's still a virtualization layer in-between. I'm 99% sure that Google runs one VM per container because that's the only way to make it safe. Anything else would be insane.

1 container per VM? What's the point? If you can bypass container sandbox it's very likely that you can do the same with VM.

> If you can bypass container sandbox it's very likely that you can do the same with VM.

Hypervisors are much, much harder to break out of than a Linux container.

Re: The Evolution of Container Usage at Netflix

#142

Post by lima » Fri, Apr 21, 2017, 10:40 PM UTC

outworlder wrote: post #110

lima wrote: post #48

Earlier quoted context omitted.

Containers don't actually run directly on hardware with GCE - there's still a virtualization layer in-between. I'm 99% sure that Google runs one VM per container because that's the only way to make it safe. Anything else would be insane.

> I'm 99% sure that Google runs one VM per container I'm 100% sure you are wrong. You might as well just use VMs. Containers are not only about safety, you know.

I stand corrected, it's a bunch of VMs per customer but still no multi-tenancy.

Re: The Evolution of Container Usage at Netflix

#143

Post by orf » Fri, Apr 21, 2017, 11:37 PM UTC

user5994461 wrote: post #138

orf wrote: post #109

Earlier quoted context omitted.

"Responsible 20% of the entire us internet traffic? Solved problem mate, i could do it on a single machine with mongodb. Overengineered!"

Forget about the joke for a minute and let's be realistic. It is entirely possible and reasonable to prototype a streaming service serving 20% of the us traffic with a single machine and mongodb. Well, a single machine, mongodb AND akamai.

Well of course, mongodb is web scale don't ya know

Re: The Evolution of Container Usage at Netflix

#144

Post by [deleted] » Fri, Apr 21, 2017, 11:55 PM UTC

sheeshkebab wrote: post #131

johansch wrote: post #125

Earlier quoted context omitted.

Bullcrap. The tech is good enough (but not that much better beyond that level). >90% of the success comes from the content creation/licensing strategy/execution and marketing. I.e. non-tech. At this point it's a bit like arguing that Disney's great success in Bluray sales comes from their encoding/Bluray mastering expertise.

Tech part was "good enough" for last 20+ years - it's why most companies call their tech teams "IT department". There is however something about that last 5-10% where moving a few bits around the right way gives you business advantage, over places like Disney. That said, it does seem like some of this Netflix stuff is overbuilt, although I don't need to run tens of 1000's of ec2, so maybe it's worth it.

[deleted]

Re: The Evolution of Container Usage at Netflix

#145

Post by johansch » Fri, Apr 21, 2017, 11:56 PM UTC

sethev wrote: post #132

johansch wrote: post #125

Earlier quoted context omitted.

Bullcrap. The tech is good enough (but not that much better beyond that level). >90% of the success comes from the content creation/licensing strategy/execution and marketing. I.e. non-tech. At this point it's a bit like arguing that Disney's great success in Bluray sales comes from their encoding/Bluray mastering expertise.

You seem to stumbling into the distinction between functional and non-functional requirements. Netflix spends massively more on content then tech so that's consistent with what you're saying. To use you're Disney example though, yes it would effect their success if they weren't able to deliver their content because their Bluray mastering expertise was lacking.

Exactly. The tech division, while overfunded and overengineered is fulfilling its role in a good enough way.

Re: The Evolution of Container Usage at Netflix

#146

Post by nailer » Sat, Apr 22, 2017, 12:47 AM UTC

brendangregg wrote: post #124

nailer wrote: post #76

Earlier quoted context omitted.

Yep but Netflix also has a decade of large scale arch wielded to AWS that we don't. They might not be deploying containers on VMs on bare metal because they want to, but because they /have/ to.

Mind quantifying this overhead you assume containers-on-metal vs VMs will solve?

It's not assumed, it's widely known VMs have poor IO performance, that one of the main points for containers in the first place. Short ver: GIYF.

Re: The Evolution of Container Usage at Netflix

#147

Post by paulddraper » Sat, Apr 22, 2017, 1:17 AM UTC

user5994461 wrote: post #138

orf wrote: post #109

Earlier quoted context omitted.

"Responsible 20% of the entire us internet traffic? Solved problem mate, i could do it on a single machine with mongodb. Overengineered!"

Forget about the joke for a minute and let's be realistic. It is entirely possible and reasonable to prototype a streaming service serving 20% of the us traffic with a single machine and mongodb. Well, a single machine, mongodb AND akamai.

Prove it.

Re: The Evolution of Container Usage at Netflix

#148

Post by Gelob » Sat, Apr 22, 2017, 1:21 AM UTC

RcouF1uZ4gsC wrote: post #52

Netflix seems so over engineered to me. They basically have a catalog of a few thousand movies that are negotiated months in advance of actual use. Basically, they just need to encode them and put them on a box and ship them to edge caches. Caching immutable data scales incredibly well.I would also bet that 99.99% of the movies people actually watch on Netflix, would fit on a single box. In regard to the analytics, t…

While the contracts may be negotiated months in advance the videos aren't delivered until very close to the release date, the same for Google and amazon. In some cases it may be a few days early but production schedules are tight and even if the post house has it done early contracts and controls forbid them from sending it to Netflix, etc too early.

Re: The Evolution of Container Usage at Netflix

#149

Post by brendangregg » Sat, Apr 22, 2017, 3:03 AM UTC

nailer wrote: post #146

brendangregg wrote: post #124

Earlier quoted context omitted.

Mind quantifying this overhead you assume containers-on-metal vs VMs will solve?

It's not assumed, it's widely known VMs have poor IO performance, that one of the main points for containers in the first place. Short ver: GIYF.

If you can't provide numbers, then please talk about what technologies you're assuming are involved that create this overhead.

Re: The Evolution of Container Usage at Netflix

#150

Post by toast0 » Sat, Apr 22, 2017, 3:06 AM UTC

emptythought wrote: post #91

zaidos wrote: post #88

Earlier quoted context omitted.

That is the core of their product. However, there are a lot of by-products and back office software needed to run the company. As a consumer, you most likely are only perceiving a small percentage of the technology running a company. - Applications for all devices Netflix is on (TVs, Phones, Displays, Roku, Apple TV, etc.) and all the support around it. - Recommendation and analytical software within Netflix. Not eve…

Another thing worth noting is that they still support basically every piece of hardware they ever did. Think built in apps on old smart TVs or early blu-ray decks that haven't been patched in YEARS, devices from before 2009~ even. You can still watch netflix on an original Wii. That hasn't gotten any OS or software patches since like 2011. That's a long tail of legacy clients to have to deal with.

Given that streaming is basically competing with discs, it makes sense not to drop any devices ever; a DVD produced today is expected to work on a DVD player from however long ago, provided it's still operational.