Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Build Code at Netflix

techblog.netflix.com

Post reply on HN
Print view View original

41–50 of 140 posts

Re: How We Build Code at Netflix

#41

Post by goalieca »

riskable wrote: post #22
mattiemass wrote: post #2

Very cool article. Amazing how much tooling Netflix has built themselves.

What's amazing is that Netflix let their teams develop solutions from scratch. Usually at big companies when their developers say something like, "can't we just build a solution ourselves?" they're laughed out of the room or, more likely, marked down as candidates in the next round of layoffs.

"let's wait for the community to fix the opensource code we rely on".

I've heard this a lot in my career (even academia)

Re: How We Build Code at Netflix

#42

Post by je42 »

jfindley wrote: post #35
riskable wrote: post #29

Earlier quoted context omitted.

I'm on a team developing an immutable infrastructure solution with (Docker) containers at a large financial institution. The biggest downsides so far: * It can get complicated very quickly since it involves a lot of stages, systems, and loads of brand new tools where you're unlikely to be able to hire existing talent (you need people who can just read the docs on a complicated tool/architecture like Kubernetes and th…

We are doing something very similar (mesos rather than k8s), also in a financial organisation. Our main issue is also security - as docker containers aren't very good at actually containing, from a security PoV, we're using SELinux MLS, assigning each container a unique SELinux level/category (fedora-atomic sets this up nicely for you if anyone wants to try). This is great for security, but breaks so many things - lo…

Do you have links pointing at what docker containers leak from a security PoV ?

Re: How We Build Code at Netflix

#43

Post by pyvpx »

riskable wrote: post #29
chei0aiV wrote: post #18

Earlier quoted context omitted.

> Immutable infrastructure is the future Are there any downsides to immutable infrastructure?

I'm on a team developing an immutable infrastructure solution with (Docker) containers at a large financial institution. The biggest downsides so far: * It can get complicated very quickly since it involves a lot of stages, systems, and loads of brand new tools where you're unlikely to be able to hire existing talent (you need people who can just read the docs on a complicated tool/architecture like Kubernetes and th…

I'm a network engineer with a strong interest in container networking -- where does anycast come into play in your immutable infrastructure? I am highly curious!

Re: How We Build Code at Netflix

#44

Post by riskable »

pyvpx wrote: post #43
riskable wrote: post #29

Earlier quoted context omitted.

I'm on a team developing an immutable infrastructure solution with (Docker) containers at a large financial institution. The biggest downsides so far: * It can get complicated very quickly since it involves a lot of stages, systems, and loads of brand new tools where you're unlikely to be able to hire existing talent (you need people who can just read the docs on a complicated tool/architecture like Kubernetes and th…

I'm a network engineer with a strong interest in container networking -- where does anycast come into play in your immutable infrastructure? I am highly curious!

Right now for us it mostly just DNS using anycast but we may use anycast as part of a (forthcoming) microservices architecture.

The current popular method is to simply query a service discovery resolver of some sort (e.g. https://docs.docker.com/swarm/discovery/) to figure out which server to query, say, the weather (very important in finance! haha =). That's great and necessary if your architecture spans across your internal cloud, Amazon, and cloud-provider-of-the-day but if you have a fixed set of data centers where you know your applications will always be running (say, because of legal reasons =) then you can just use anycast as long as you keep ports consistent.

For most microservices deployments I'd imagine they would want their stuff to be accessible via a global DNS name and if they're going to do that then they might as well take the next step and just standardize on a port as well. If you have a single (global) name and a single port for an API why do you need service discovery (for that)? You don't. You just need to make sure that when clients query your API that they don't wind up using that one server you've got on the other side of the world on a 56k ISDN line (sadly yes, that does happen).

Having said that we may end up using popular service discovery tools anyway because they're convenient and we can use them in horrible, do-as-I-say-not-as-I-do ways to hack legacy applications into containers =D

Re: How We Build Code at Netflix

#46

Post by aaronblohowiak »

Scarbutt wrote: post #24

What are the reasons for Netflix choosing nodejs for their front-end server and not java like in their back-end?

Freedom and Responsibility! Teams are free to choose the tools that work for them. Netflix technical direction operates more like an ecosystem than a hierarchy.

Re: How We Build Code at Netflix

#47

Post by riskable »

je42 wrote: post #42
jfindley wrote: post #35

Earlier quoted context omitted.

We are doing something very similar (mesos rather than k8s), also in a financial organisation. Our main issue is also security - as docker containers aren't very good at actually containing, from a security PoV, we're using SELinux MLS, assigning each container a unique SELinux level/category (fedora-atomic sets this up nicely for you if anyone wants to try). This is great for security, but breaks so many things - lo…

Do you have links pointing at what docker containers leak from a security PoV ?

This largely depends on how you're implementing them. If you're using Kubernetes, for example, all Docker containers being managed by Kubernetes can talk to each other without restrictions. From each container's perspective they're part of a big, flat network with no firewalls between them.

So if you have a container running the database for client A that means that when client B spins up their little malicious container they can directly connect to client A's DB.

That's one way that Docker containers can "leak" (sort of). Another way is that the filesystem for running containers is directly accessible from outside the container. That means that anyone with the power to start/stop containers has the power to read the filesystems for all other containers.

There used to be a problem with, "root inside the container is the same as root outside the container" but that was fixed with Docker 1.9 via the new UID mapping feature. So the filesystem and processes running as UID 0 inside the container will actually be some high UID in the parent OS. So unless you're running Red Hat's version of Docker (which is currently hard set to be Docker 1.8!) you can upgrade and solve that problem pretty quickly with a few tweaks.

Re: How We Build Code at Netflix

#48

Post by error54 »

Scarbutt wrote: post #24

What are the reasons for Netflix choosing nodejs for their front-end server and not java like in their back-end?

Since all Frontend code is written in JavaScript, it makes sense to have the server code written in JavaScript as well as we do quite a bit of work on the server. Also, we do serve isomorphic views so Node is a big help in that regards.

Re: How We Build Code at Netflix

#49

Post by dominotw »

salehenrahman wrote: post #25
Scarbutt wrote: post #24

What are the reasons for Netflix choosing nodejs for their front-end server and not java like in their back-end?

I'm guessing it's their use of React both on the front-end and back-end.

there is also this https://github.com/Netflix/falcor

Re: How We Build Code at Netflix

#50

Post by riskable »

chatmasta wrote: post #27
riskable wrote: post #22

Earlier quoted context omitted.

What's amazing is that Netflix let their teams develop solutions from scratch. Usually at big companies when their developers say something like, "can't we just build a solution ourselves?" they're laughed out of the room or, more likely, marked down as candidates in the next round of layoffs.

It helps that they have repeatedly proven this strategy works. When the CTO goes to the CEO and says "we're going to build this, not buy it," the CEO obviously trusts his decision making because he's seen him successfully execute his strategy so many times. It sounds like Netflix engineering culture is built a lot around trust. Management trusts that only top engineering talent works there, and the sink-or-swim cultu…

> It helps that they have repeatedly proven this strategy works.

I laughed at this because you can't "repeatedly prove" you're capable of developing your own solutions if you're never allowed to do it in the first place!

The enterprise mantra is: COTS or it doesn't happen.

Post reply on HN