Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Sisyphean Task of DNS Client Config on Linux

tailscale.com

Post reply on HN
Print view View original

41–50 of 86 posts

Re: The Sisyphean Task of DNS Client Config on Linux

#41

Post by mwcampbell »

tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

> This sucks; we shouldn't have to be inline for arbitrary customer DNS.

Why not? Someone has to do it, and if I were paying Fly for service, I'd rather have Fly handle my DNS queries than be another freeloader on Google or Cloudflare.

Re: The Sisyphean Task of DNS Client Config on Linux

#42

Post by kevinoid »

bonzini wrote: post #2

The most interesting takeaway from this article is that, according to people who actually do the work, NetworkManager and systemd-resolved do get things right.

I agree! As a non-expert, I've also had good luck with NetworkManager, but wouldn't recommend systemd-resolved yet if you use DNSSEC:

https://github.com/systemd/systemd/issues/6490 (fixed in v248)

https://github.com/systemd/systemd/issues/8451

https://github.com/systemd/systemd/issues/9867

https://github.com/systemd/systemd/issues/12388

Re: The Sisyphean Task of DNS Client Config on Linux

#43

Post by vetinari »

LukeShu wrote: post #10
daenney wrote: post #6

Earlier quoted context omitted.

It’s probably omitted as nsswitch doesn’t affect DNS client configuration. It affects which sources are consulted to lookup a name (including maybe asking DNS), but it doesn’t configure things like which DNS servers to ask or what options to set. > The Name Service Switch (NSS) configuration file, /etc/nsswitch.conf, is used by the GNU C Library and certain other applications to determine the sources from which to ob…

nsswitch.conf is what tells it whether to use the traditional nss_dns (the thing that looks at /etc/resolv.conf), or whether to use the newer nss_resolve (the thing that talks to systemd-resolved). So yeah, if the article is contrasting resolv.conf vs systemd-resolved, nsswitch.conf is how you select which of those approaches is used.

Even with nss_resolve, you still have to have /etc/resolv.conf correctly populated, because some apps will ignore gethostbyname() and just parse resolv.conf and do DNS by themselves. For example, golang stdlib does that (but to the credit of golang runtime, it checks whether nsswitch.conf is in expected state and falls back to glibc if it is not).

Some apps go even further and do their DNS entirely on their own (Firefox DoH controversy).

nsswitch however doesn't just configure mechanism for public resolver, it is a hook where alternate mechanisms like nss-mdns or nss-mymachines can hook up.

Re: The Sisyphean Task of DNS Client Config on Linux

#44

Post by dave_universetf »

tptacek wrote: post #13

Tailscale is awesome, you should use it for everything. This Linux DNS stuff drives us batty at Fly.io. We run user containers as Firecracker VMs; users belong to "organizations", and organizations share a private IPv6 network. We do DNS for that private network under the fake "internal" TLD, so if you have an app "phoenix-frontend" and another app "rabbitmq-cluster", they can see each other at "phoenix-frontend.inte…

This is also what we're trying to do in Tailscale (to grab your MagicDNS domain, and whatever corporate split DNS you set, but not anything else). And yeah, on linux, basically systemd-resolved is the only thing that gets this right (with dynamic config - with fully static config, other recursors are an option, with varying tradeoffs), everything else assumes "one resolver should be enough for everything".

So, your solution to proxy all the traffic and split out as needed is the right way to do it without adding more software into each VM :(

Doing stuff via LD_PRELOAD would be hilarious, you should definitely do that and report back ducks behind the blast shield

Re: The Sisyphean Task of DNS Client Config on Linux

#45

Post by vetinari »

kelnos wrote: post #34
denysvitali wrote: post #15

Earlier quoted context omitted.

FYI, if you don't bother opening the link, systemd-resolv sets the /etc/resolv.conf to have only 127.0.0.1 as a DNS server. You can see how things get bad when the CoreDNS pod tries to get the "upstream" DNS servers from the host /etc/resolv.conf

I guess it shouldn't be doing that? I mean, that also breaks in the case where you're running something like dnsmasq as your local resolver, which will also require 127.0.0.1 in your resolv.conf. As much as I'm not always comfortable with the larger complexity, the CoreDNS pod really needs to be doing something like the flowchart described in the article. If systemd-resolved or NetworkManager are in play, it should b…

> I mean, that also breaks in the case where you're running something like dnsmasq as your local resolver, which will also require 127.0.0.1 in your resolv.conf.

Dnsmasq DNS support is redundant when you are running systemd-resolved. It can do everything dnsmasq does and more.

> resolv.conf is an old idea that is too inflexible for today's DNS needs; a more complex solution with more complex interface is unfortunately warranted here.

That's exactly what systemd-resolved is.

Re: The Sisyphean Task of DNS Client Config on Linux

#47

Post by mindslight »

Koffiepoeder wrote: post #35

To be honest I hate all of the aforementioned programs trying to battle over /etc/resvolv.conf. That's why by default I have the file marked as immutable, and pointing to 127.0.0.1. This way I cannot have accidental DNS traffic leaking. If I need local network DNS I send a manual dhcp command to get the wifi's DNS and add it temporarily to my dnscrypt config. Similar thing for VPN DNS.

I do something similar whereby I just point DNS traffic at the router itself and configure it per-host by DNAT, as part of the larger WAN horizon zone (eg direct goes to local recursor, wireguard tunnel goes to the recursor on the other side of the tunnel or suitable public resolver, commercial VPN goes to whatever their recommended resolver is, eventually pihole for guest network). Running multiple apps / security contexts / nyms (or whatever else you want to call them) on the same OS instance is just asking for trouble.

Re: The Sisyphean Task of DNS Client Config on Linux

#48

Post by tgbugs »

An enlightening article that explains why I think I am losing my mind every time I try to get the network configured correctly on any number of distros. My main take home is to never install NetworkManager or a resolvconf and to avoid systemd if at all possible, unless I absolutely know that I am going to need those state of the art DNS capabilities.

I have mostly managed to avoid resolv.conf issues since the default Gentoo image has a sane setup (even if using dhcp on a laptop and switching wifi and wired). However, every single time I have tried to set up a system using another distro something has gone wrong, and in support of the theory that NetworkManager is a major contributor to the problem the one Gentoo system with NetworkManager installed had the same issues.

To echo the plea in the original article, in nearly every case the primary challenge has been to figure out exactly what documentation actually applies to the system at hand because distros seem to change this completely out of sync with any attempt to correct or align the documentation.

I suspect that on an individual user level this leads to a happy path situation where everyone who does the right thing by accident is quiet and the ones who need something slightly different are never heard from because they weren't able to even connect to the internet (probably not quiet that bad).

Re: The Sisyphean Task of DNS Client Config on Linux

#49

Post by vetinari »

liveoneggs wrote: post #20
corty wrote: post #5

Earlier quoted context omitted.

Problem is, modern browsers do things differently yet again. So the insanity continues, just on different levels.

browsers are operating systems in user space. They will continue to replace kernel parts.

Technically, the existing dns resolving mechanism is a part of glibc, so it is userspace already.

Re: The Sisyphean Task of DNS Client Config on Linux

#50

Post by vetinari »

tgbugs wrote: post #48

An enlightening article that explains why I think I am losing my mind every time I try to get the network configured correctly on any number of distros. My main take home is to never install NetworkManager or a resolvconf and to avoid systemd if at all possible, unless I absolutely know that I am going to need those state of the art DNS capabilities. I have mostly managed to avoid resolv.conf issues since the default…

That's exactly the wrong take home; the only sane way to handle dns is systemd-resolved, as the article said. If you use your machine as anything resebling desktop/laptop, you should configure your network using Network Manager, especially if you have connections that come and go. Switching between Lan and wifi with dhcp is really low bar.

Anything else is just prolonging the agony that the linux networking configuration had to endure for years.

Post reply on HN