Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Sisyphean Task of DNS Client Config on Linux

tailscale.com

Post reply on HN
Print view View original

21–30 of 86 posts

Re: The Sisyphean Task of DNS Client Config on Linux

#21

Post by friseurtermin »

tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

Funnily enough, that is the _exact_ same problem I'm facing right now, down to Firecracker and a custom internal TLD. I'm excited to see the solutions. I think the only difference is that I need to run this DNS service on the same host as my VMs, so I will need to use a different port than systemd-resolve.

Re: The Sisyphean Task of DNS Client Config on Linux

#22

Post by tptacek »

dnr wrote: post #17
tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

systemd-resolved can do this, though I haven't played with it too much yet. Here's a random post I just found: https://gist.github.com/brasey/fa2277a6d7242cdf4e4b7c720d42b... (Ha, I did the thing where I read the comments before the post, and the post describes how to do this. So what's still missing?)

I'm not sure how reasonable it is for us to run systemd-resolved (or systemd itself) on our VMs. We're trying to provide a clean environment for any random container to run on; we provide our own init, and that's almost the whole of it.

What, I think, you really care about here is glibc's behavior. But then, you can't depend on any one libc, either.

Re: The Sisyphean Task of DNS Client Config on Linux

#23

Post by jeffbee »

tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

Feels like the actual clean way to resolve this is to not use DNS for discovery of named services.

Re: The Sisyphean Task of DNS Client Config on Linux

#24

Post by recuter »

tptacek wrote: post #22
dnr wrote: post #17

Earlier quoted context omitted.

systemd-resolved can do this, though I haven't played with it too much yet. Here's a random post I just found: https://gist.github.com/brasey/fa2277a6d7242cdf4e4b7c720d42b... (Ha, I did the thing where I read the comments before the post, and the post describes how to do this. So what's still missing?)

I'm not sure how reasonable it is for us to run systemd-resolved (or systemd itself) on our VMs. We're trying to provide a clean environment for any random container to run on; we provide our own init, and that's almost the whole of it. What, I think, you really care about here is glibc's behavior. But then, you can't depend on any one libc, either.

The cliffs of cloud native software defined networking insanity await you:

https://github.com/containernetworking/cni

https://github.com/firecracker-microvm/firecracker-go-sdk

  Firecracker, by design, only supports Linux tap devices. The SDK provides facilities to:

  Attach a pre-created tap device, optionally with static IP configuration, to the VM. This is referred to as a "static network interface".
  Create a tap device via CNI plugins, which will then be attached to the VM automatically by the SDK. This is referred to as a "CNI-configured network interface"

Re: The Sisyphean Task of DNS Client Config on Linux

#25

Post by YarickR2 »

friseurtermin wrote: post #21
tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

Funnily enough, that is the _exact_ same problem I'm facing right now, down to Firecracker and a custom internal TLD. I'm excited to see the solutions. I think the only difference is that I need to run this DNS service on the same host as my VMs, so I will need to use a different port than systemd-resolve.

I'd recommend to run it on the same port, but different IP (127.0.1.2 , f.e.) , due to inability of some programs to use non-standard port

Re: The Sisyphean Task of DNS Client Config on Linux

#26

Post by js2 »

tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

On macOS, you could do this trivially by creating /etc/resolver/internal with the name servers in it.

Linux should copy that.

Re: The Sisyphean Task of DNS Client Config on Linux

#27

Post by YarickR2 »

mjevans wrote: post #16
tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

Couldn't you run dnsmasq, unbound, or some other configurable resolver on the localhost? Though that's probably already what's done with the internal smart resolver.

This is proper advice. My recursor of choice is pdns-recursor , but bind in forward-only mode with several forwarders for different zones will work too

Re: The Sisyphean Task of DNS Client Config on Linux

#28

Post by YarickR2 »

tptacek wrote: post #22
dnr wrote: post #17

Earlier quoted context omitted.

systemd-resolved can do this, though I haven't played with it too much yet. Here's a random post I just found: https://gist.github.com/brasey/fa2277a6d7242cdf4e4b7c720d42b... (Ha, I did the thing where I read the comments before the post, and the post describes how to do this. So what's still missing?)

I'm not sure how reasonable it is for us to run systemd-resolved (or systemd itself) on our VMs. We're trying to provide a clean environment for any random container to run on; we provide our own init, and that's almost the whole of it. What, I think, you really care about here is glibc's behavior. But then, you can't depend on any one libc, either.

It's unreasonable to bring the whole cow to just get a gallon of milk . Use standalone resolver.

Re: The Sisyphean Task of DNS Client Config on Linux

#29

Post by ptomato »

tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

... I realize I'm a monster, but aren't y'all already extensively using BPF for things? Sticking in a BPF egress filter on the VM that rewrites outbound DNS packets with .internal in the question to point at your DNS server seems like it would be lighter weight than just handling all queries recursively.

Re: The Sisyphean Task of DNS Client Config on Linux

#30

Post by YarickR2 »

bonzini wrote: post #2

The most interesting takeaway from this article is that, according to people who actually do the work, NetworkManager and systemd-resolved do get things right.

Mostly right. Article never mentions /etc/hosts , which is still largely a thing, and works wonders in difficult cases ( and makes other difficult cases much worse to debug)
Post reply on HN