Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DNS is for people, not for IT infrastructure

louwrentius.com

Post reply on HN
Print view View original

91–100 of 102 posts

Re: DNS is for people, not for IT infrastructure

#91

Post by sshine »

nemothekid wrote: post #87
JdeBP wrote: post #83

Earlier quoted context omitted.

One does not even need to squint. The first page of RFC 882 explains outright that the DNS came about in the first place because the mechanisms for updating a HOSTS.TXT file and publishing it to loads of places did not scale. That's still just as true for the intranets of the 2020s with thousands of machines all downloading a HOSTS file several times a day (or even hour/minute) as it was for the Internet of July 1983…

> The first page of RFC 882 explains outright that the DNS came about in the first place because the mechanisms for updating a HOSTS.TXT file and publishing it to loads of places did not scale. Great piece of history. The RFC is a bit older than I am so I've never studied it. Looking at it that way, then OP has just re-invented DNS.

And what a great invention DNS is.

If you need to eliminate DNS and convince the internet it's largely unnecessary for the use-cases we have today...

...only to completely reinvent DNS for those use-cases with inferior technology that eventually becomes DNS...

...then you have achieved wisdom. I applaud the author for being on this journey.

Re: DNS is for people, not for IT infrastructure

#92

Post by adrian_b »

JdeBP wrote: post #83
nemothekid wrote: post #81

> Fortunately, we still have /etc/hosts, which we can easily provision This is the kind of thing you read in a post-mortem and wonder how they designed something so fiendishly wonderful. At 2:00am our MySQL master failed and failed over successfully to our secondary server. As part of post-failover ops, ansible playbook proceeded to login to 1000 instances to update the hosts file for the new master. This caused traf…

One does not even need to squint. The first page of RFC 882 explains outright that the DNS came about in the first place because the mechanisms for updating a HOSTS.TXT file and publishing it to loads of places did not scale. That's still just as true for the intranets of the 2020s with thousands of machines all downloading a HOSTS file several times a day (or even hour/minute) as it was for the Internet of July 1983…

This is true, but TFA argued that the network services themselves should not use /etc/hosts or other similar translation database, so a not yet updated file should not cause a network outage.

TFA proposed that /etc/hosts or the like should be used only for the benefit of administrators, to allow manual connections by name instead of by address, and presumably to make easy to interpret the activity logs. This is a desirable feature, but the network should work fine even when the name-to-address translation is temporarily unavailable, because of not-yet-updated /etc/hosts files.

Actually I have used for decades a system similar to what TFA proposes, avoiding to do DNS queries for the internal networks, while using my own DNS caching resolver for the Internet, but this was done only in relatively small networks, with a few hundred nodes at most, and where the IP addresses were changed infrequently. Thus I have no idea whether in a big network with frequently changed addresses there would be scaling problems.

Re: DNS is for people, not for IT infrastructure

#93

Post by trumpdong »

nemothekid wrote: post #81

> Fortunately, we still have /etc/hosts, which we can easily provision This is the kind of thing you read in a post-mortem and wonder how they designed something so fiendishly wonderful. At 2:00am our MySQL master failed and failed over successfully to our secondary server. As part of post-failover ops, ansible playbook proceeded to login to 1000 instances to update the hosts file for the new master. This caused traf…

You can, and we did, use VRRP, anycast, or similar protocols/techniques to have an IP address that is always the current MySQL master.

Re: DNS is for people, not for IT infrastructure

#94

Post by trumpdong »

solatic wrote: post #75
monkpit wrote: post #72

Earlier quoted context omitted.

Isn’t this just DNS with extra steps, anyways? Now Ansible is the DNS server, basically.

It replaces DNS's pull-based architecture (contact a DNS server to get the IP address) with a push-based one (push the IP addresses to each /etc/hosts file). Suggesting that a push-based, Ansible-based architecture will scale to hundreds of thousands of targets, with such pushes happening hundreds if not thousands of times a day, is a junior-level idea at best, dark comedy if I'm being charitable, and professional ma…

This sounds a bit like saying: don't use MySQL, because it can't scale to one billion requests per second. How many applications are actually running at that scale?

Re: DNS is for people, not for IT infrastructure

#95

Post by trumpdong »

protocolture wrote: post #40
louwrentius wrote: post #32

Earlier quoted context omitted.

Why would some form of service discovery be required? No need to discover things if you can push said information in configuration updates using tools like Ansible, pyinfra, and so on?

How does your convoluted Ansible system know which systems and services to maintain. If its a list of IP addresses, having a list of ip addresses is a crude service discovery protocol. Tasking developers (because lets be absolutely clear, the idea of removing DNS from production environments is something only a developer could come up with, no competent engineer would ever raise) with maintaining ordered lists of ser…

How does your DNS system know? There's always a list of all systems somewhere.

Re: DNS is for people, not for IT infrastructure

#96

Post by trumpdong »

spragl wrote: post #37
louwrentius wrote: post #22

Earlier quoted context omitted.

Why is that madness and not amazing? Isn’t the simplicity beautiful? Managing /etc/hosts with a tool like Ansible?

/etc/hosts scales like a lead balloon. For small groups of servers, with limited egress communication, it might nevertheless make sense. And then go for it, by all means. As a general replacement for DNS, not likely. It is hard to see how Ansible should be simpler than DNS. Maybe if you have worked with Ansible and not DNS, you might think so.

You're using Ansible anyway. I think the comparison is DNS+Ansible vs just Ansible.

Re: DNS is for people, not for IT infrastructure

#97

Post by solatic »

trumpdong wrote: post #94
solatic wrote: post #75

Earlier quoted context omitted.

It replaces DNS's pull-based architecture (contact a DNS server to get the IP address) with a push-based one (push the IP addresses to each /etc/hosts file). Suggesting that a push-based, Ansible-based architecture will scale to hundreds of thousands of targets, with such pushes happening hundreds if not thousands of times a day, is a junior-level idea at best, dark comedy if I'm being charitable, and professional ma…

This sounds a bit like saying: don't use MySQL, because it can't scale to one billion requests per second. How many applications are actually running at that scale?

Did you read the original article?

> The Facebook / Meta outage was so significant

The author specifically called out the Meta outage, as if he was offering a prescription ("It's easy to configure systems with tools like Ansible or pyinfra at scale") that would have prevented Meta (at Meta's scale) from suffering an outage. The argument that Meta should not have used DNS except that Meta runs at a scale where DNS is necessary... who comes up with these arguments?

Re: DNS is for people, not for IT infrastructure

#98

Post by patmorgan23 »

necovek wrote: post #50
louwrentius wrote: post #30

Earlier quoted context omitted.

I do state in the article that in the examples DNS isn't the root-cause, but the blast radius is very significant. Regardless of the topic of external/internal services, isn't it remarkable that a group of very smart and well-paid people create such circular dependancies? Yet, I'm not arguing for Facebook or similar size companies to ditch DNS internally. I'm making the argument for much smaller organisations to paus…

As said, I believe you are simplifying the problem significantly and thus making general claims which do not hold water. Eg. even if you are DNS based but have direct SSH access to the system which has a query cached and root access on it (you need to manage all this too!), you can temporarily edit /etc/hosts or /etc/resolv.conf to workaround the cached value. So my suggestion remains to keep working on a better argu…

Or you can just clear the cache!

Re: DNS is for people, not for IT infrastructure

#99

Post by Melatonic »

louwrentius wrote: post #30
necovek wrote: post #21

Earlier quoted context omitted.

The examples you cite (eg. 2021 Facebook outage) have nothing to do with DNS being used for internal infrastructure. In the other example (Amazon DynamoDB issue), the problem is with dynamically choosing from a large dynamic pool of IP addresses for a service — DNS is but one mechanism to do it. If it wasn't DNS, it could have been something else that did that job that was broken. Even /etc/hosts if it was updated wi…

I do state in the article that in the examples DNS isn't the root-cause, but the blast radius is very significant. Regardless of the topic of external/internal services, isn't it remarkable that a group of very smart and well-paid people create such circular dependancies? Yet, I'm not arguing for Facebook or similar size companies to ditch DNS internally. I'm making the argument for much smaller organisations to paus…

I think what you are really arguing for is more people that properly understand (and implement) DNS.

A smaller organisation should have a much easier time implementing internal DNS and it should be pretty damn stable and reliable. Unfortunately a lot of people dont properly understand it (not that you need to be a complete expert - just competent) and hence we always have the mantra "Its always DNS" when something goes down.

Usually complicated beskoke systems engineered for internal use are better left for really large orgs that can hire the talent to maintain and properly implement it (and have the manpower to have enough people in the first place always on staff to maintain it when the first person gets sick or something)

Re: DNS is for people, not for IT infrastructure

#100

Post by Melatonic »

louwrentius wrote: post #17
gfody wrote: post #10

> we'll just use /etc/hosts no DNS required! this is classic "easy vs. simple" folly, witness how someone too lazy to [learn how to] setup proper DNS for their infrastructure will do 10x the work hacking something "easy"

Serious response: how is templating out /etc/hosts with Ansible not 10x simpler than setting up an additional service that only introduces additional risk?

What happens when the person who is your Ansible expert gets sick and is out for a week?

What happens if you need to scale up (either a lot or a little) and you need to hire new people?

People are often the most vulnerable chain in your infrastructure.

Post reply on HN