Live data from Hacker News

Kubernetes Networking: Behind the Scenes

itnext.io

31–34 of 34 posts

Re: Kubernetes Networking: Behind the Scenes

#31

Earlier quoted context omitted.

Because ipv6 is not a simple addition, but a complex suite of new protocols meant to replace v4, except that is never going to happen in our lifetimes. v6 should've simply been v4 with more bits

IPv6 should not be an addition to Kubernetes, but the default. IPv4 could then be an addition for when you need external communication to IPv4 networks. There's no reason why IPv6 isn't the default for internal communication between containers. We've been deploying IPv6 for at least 10 years and have hosts that are exclusively IPv6. Then along comes Docker and Kubernetes with their retro IPv4 only stack and now custo…

Well, since k8s is developed by mere humans on simple notebooks, and initially it got deployed to stock Linux they started with v4. And they had to start with v4. And to get more contributors they also needed v4.

Yes, the network effects are strong with this one.

But soon. Bit by bit, I hope the true dualstack will come, and then the v4 will fade away over generations.

Re: Kubernetes Networking: Behind the Scenes

#32

Earlier quoted context omitted.

Because ipv6 is not a simple addition, but a complex suite of new protocols meant to replace v4, except that is never going to happen in our lifetimes. v6 should've simply been v4 with more bits

No backwards compatibility could be IPv6's biggest mistake. Now everyone is forced to maintain 2 IP addresses.

So far no one ever managed to come up with a plan that is backward compatible and does enough. (Aka the fundamental problem is that if you change v4 semantics you have to deploy new software/firmware/hardware, and if you are doing that anyway, a clean slate seemed like the easier option.) And don't forget v6 has well defined semantics with regards to v4 interoperability for a lot of things (::ffff:, NAT64, DNS64), so you can be on v6 and reach all of v4 via a gateway.)

Re: Kubernetes Networking: Behind the Scenes

#33
On the Nomad project we've taken a distinct approach to networking: treat the IP layer as an implementation detail and attempt to abstract it away. This has led to a few design decisions:

1. Global node addressing - Given a region + node ID you can query any node from any other node.

2. Global allocation (pod) addressing - Given a region + allocation ID you can interact with individual allocations and tasks (metrics, files, logs, remote exec, etc).

3. Global name-based service addressing (via Consul) - whether via Consul's DNS interface, catalog API, or Connect service mesh, services discover one another by name instead of by IP or port. The service's address (ip+port) is hopefully an opaque implementation detail that never needs to be externally observed.

With all 3 of these taken together there's no need for overlays, distinct subnets, or unique IPs-per-allocation. Datacenters may even have overlapping subnets and Consul Connect will still allow services to communicate via global name based addressing. Even without Consul, almost all Nomad commands will route to a remote region and execute on the correct node regardless of subnet overlapping.

This does cause some friction at the edge: when translating from external IP based routing to internal services (aka ingress). However, there are lots of load balancers that integrate with Consul and therefore translate between IP and name based routing adequately.

Really any service unhappy with our "route by name, not by ip" design is going to be a bit awkward to use, but routable IPs and ports are selected and exposed by Nomad's scheduler - or advertised in Consul if the address is overridden at runtime.

Re: Kubernetes Networking: Behind the Scenes

#34

Earlier quoted context omitted.

No backwards compatibility could be IPv6's biggest mistake. Now everyone is forced to maintain 2 IP addresses.

I think that’s intended as a feature. IPv4 is just not good enough, we will run out of addresses. I think they want to kill IPv4 and replace it rather than improving upon it. At some point programs will ignore IPv4 altogether and our IP addresses will not be fundamentally flawed anymore.

layers are only ever added, never removed
Post reply on HN