Kubernetes Networking: Behind the Scenes • Hacker News Board

Re: Kubernetes Networking: Behind the Scenes

#21

Post by justinsaccount » Wed, Oct 16, 2019, 7:09 PM UTC

atombender wrote: post #17

justinsaccount wrote: post #16

Earlier quoted context omitted.

That is basically how consul connect works: https://www.consul.io/docs/connect/native/go.html

Not as I understand it. It uses DNS resolution, so you connect to something like foo.web.service.consul. Kubernetes's KubeDNS works exactly the same way with Kubernetes service objects.

From that page:

  // Create an instance representing this service. "my-service" is the
  // name of _this_ service. The service should be cleaned up via Close.
  svc, _ := connect.NewService("my-service", client)

and when http is used:

> The hostname used in the request URL is used to identify the logical service discovery mechanism for the target. It's not actually resolved via DNS but used as a logical identifier for a Consul service discovery mechanism. It has the following specific limitations:

Re: Kubernetes Networking: Behind the Scenes

#22

Post by atombender » Wed, Oct 16, 2019, 7:32 PM UTC

justinsaccount wrote: post #21

atombender wrote: post #17

Earlier quoted context omitted.

Not as I understand it. It uses DNS resolution, so you connect to something like foo.web.service.consul. Kubernetes's KubeDNS works exactly the same way with Kubernetes service objects.

From that page: // Create an instance representing this service. "my-service" is the // name of _this_ service. The service should be cleaned up via Close. svc, _ := connect.NewService("my-service", client) and when http is used: > The hostname used in the request URL is used to identify the logical service discovery mechanism for the target. It's not actually resolved via DNS but used as a logical identifier for a C…

Ah, I see. That's presumably possible because Go has its own DNS resolver, and it (net.DefaultResolver) can be overridden by libraries; no LD_PRELOAD needed.

Re: Kubernetes Networking: Behind the Scenes

#23

Post by kgersen75 » Wed, Oct 16, 2019, 7:44 PM UTC

forgot-my-pw wrote: post #14

sascha_sl wrote: post #10

Earlier quoted context omitted.

Because ipv6 is not a simple addition, but a complex suite of new protocols meant to replace v4, except that is never going to happen in our lifetimes. v6 should've simply been v4 with more bits

No backwards compatibility could be IPv6's biggest mistake. Now everyone is forced to maintain 2 IP addresses.

that's not true. you can have IPv6 only in the backend and expose IPv4 only at the very frontend (like Cloudfare does the other way around).

Dual stack are not really needed and usually add complexity for few little benefits.

The main problem of IPv6 is the lack of people willing to learn it, teach it and adopt it.

Something brand new like Kubernetes should have from the start been IPv6 only , with IPv4 at the egress parts only.

Re: Kubernetes Networking: Behind the Scenes

#24

Post by justinsaccount » Wed, Oct 16, 2019, 8:06 PM UTC

atombender wrote: post #22

justinsaccount wrote: post #21

Earlier quoted context omitted.

From that page: // Create an instance representing this service. "my-service" is the // name of _this_ service. The service should be cleaned up via Close. svc, _ := connect.NewService("my-service", client) and when http is used: > The hostname used in the request URL is used to identify the logical service discovery mechanism for the target. It's not actually resolved via DNS but used as a logical identifier for a C…

Ah, I see. That's presumably possible because Go has its own DNS resolver, and it (net.DefaultResolver) can be overridden by libraries; no LD_PRELOAD needed.

it's because connect.NewService is the consul client, not something like net.DialTCP

Re: Kubernetes Networking: Behind the Scenes

#25

Post by mrweasel » Wed, Oct 16, 2019, 8:21 PM UTC

sascha_sl wrote: post #10

Already__Taken wrote: post #8

It's surprising k8s being so new seems to have no concept of ipv6

Because ipv6 is not a simple addition, but a complex suite of new protocols meant to replace v4, except that is never going to happen in our lifetimes. v6 should've simply been v4 with more bits

IPv6 should not be an addition to Kubernetes, but the default. IPv4 could then be an addition for when you need external communication to IPv4 networks. There's no reason why IPv6 isn't the default for internal communication between containers.

We've been deploying IPv6 for at least 10 years and have hosts that are exclusively IPv6. Then along comes Docker and Kubernetes with their retro IPv4 only stack and now customer wonder why we're struggling to make containers work or suppliers that can't understand why we can't easily deploy the latest version of their software, which is only distributed as a Docker image.

Re: Kubernetes Networking: Behind the Scenes

#26

Post by atombender » Wed, Oct 16, 2019, 8:25 PM UTC

justinsaccount wrote: post #24

atombender wrote: post #22

Earlier quoted context omitted.

Ah, I see. That's presumably possible because Go has its own DNS resolver, and it (net.DefaultResolver) can be overridden by libraries; no LD_PRELOAD needed.

it's because connect.NewService is the consul client, not something like net.DialTCP

Oh, they require all TCP connections to go through that. That's a bit heavy-handed — requires rewriting all client code to adapt to that pattern. The nice thing about KubeDNS is that you don't have to. Same with Istio.

Re: Kubernetes Networking: Behind the Scenes

#27

Post by peterwwillis » Wed, Oct 16, 2019, 9:29 PM UTC

atombender wrote: post #26

justinsaccount wrote: post #24

Earlier quoted context omitted.

it's because connect.NewService is the consul client, not something like net.DialTCP

Oh, they require all TCP connections to go through that. That's a bit heavy-handed — requires rewriting all client code to adapt to that pattern. The nice thing about KubeDNS is that you don't have to. Same with Istio.

Integrating with app code is optional. The default is to use a sidecar.

Re: Kubernetes Networking: Behind the Scenes

#28

Post by justinsaccount » Wed, Oct 16, 2019, 9:29 PM UTC

atombender wrote: post #26

justinsaccount wrote: post #24

Earlier quoted context omitted.

it's because connect.NewService is the consul client, not something like net.DialTCP

Oh, they require all TCP connections to go through that. That's a bit heavy-handed — requires rewriting all client code to adapt to that pattern. The nice thing about KubeDNS is that you don't have to. Same with Istio.

I brought up the native client because you mentioned "they should have defined a new API".

Re: Kubernetes Networking: Behind the Scenes

#29

Post by atombender » Wed, Oct 16, 2019, 9:33 PM UTC

justinsaccount wrote: post #28

atombender wrote: post #26

Earlier quoted context omitted.

Oh, they require all TCP connections to go through that. That's a bit heavy-handed — requires rewriting all client code to adapt to that pattern. The nice thing about KubeDNS is that you don't have to. Same with Istio.

I brought up the native client because you mentioned "they should have defined a new API".

That wasn't me. I'd argue the opposite. Though I'd also maybe argue that SRV records is what we should have standardized on for this stuff a long time ago.

Re: Kubernetes Networking: Behind the Scenes

#30

Post by pas » Wed, Oct 16, 2019, 10:06 PM UTC

londons_explore wrote: post #13

I feel like k8s networking is 10x more complex than it needs to be... I think it's because they still want to use the socket API, but to "pretend" you can open a socket to a pod, or to a service, etc. Instead, they should have defined a new API, like "connect('k8s:service:foobar?selectionstrategy=leastloaded')" Then they could have made simple LD_PRELOAD shims to get their magic API to work with existing software wit…

It's complex because K8S is used in a wide variety of complex environments.

Secure dynamic overlay networking, plus egress is inherently not a trivial thing. Then add customization/extendability on top.

The important thing to get is that it uses really simple building blocks, very UNIX-y. Every time a new container starts it starts a program (the CNI driver) and pipes in a JSON. And the driver does what it wants (executes iptables, BGP announces a prefix, commits something to its internal DB) and spits out some JSON that tells what IP the container got.

And .. that's it. No magic. Everything has to be configured, arguments passed to some daemon at cluster setup time, and so on. Yes, kubelet and apiserver both have hundreds of command line args, but [almost] all of them are very clearly associated with tuning a functionality (or passing some data forward to some daemon/driver/component) - and thus if you don't touch it but it works, yaay, you don't have to care about it (though of course it might turn out that you'll need to scale up/down/faster/lighter).

The socket API is fine, even with a new shiny API the network fundamentals would have to be managed just the same way. The Linux kernel would be the same, just as Ethernet, IP and so on. Reinventing all of those wheels just to provide transparent pod-to-pod networking doesn't seem to ever worth it.