Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop using low DNS TTLs (2019)

blog.apnic.net

Post reply on HN
Print view View original

41–50 of 110 posts

Re: Stop using low DNS TTLs (2019)

#41

Post by dspillett »

Tepix wrote: post #35
8organicbits wrote: post #8

The scenario I think about is, what happens if the DNS entry I just pushed had a typo in it? If my TTL is 60s, then I push a fixed entry and my site is back up quickly. If the TTL was 1hr, then anyone who cached that bad entry can't connect for about that long. 60s is better for disaster recovery, 1hr is better for performance. I've seen this happen on internal networks where IT support tries to guide end users throu…

Quoting from https://cr.yp.to/djbdns/tinydns-data.html : Beware that cache times below 300 seconds will be treated as 300 by some clients, and NS cache times below 2 seconds can cause lookup failures.

I wonder what those clients are, none of these warnings ever give examples in my experience so I can't tell if this is old out-of-date information being repeated or it is an active modern problem.

I know there was a DNS resolver that did worse than impose a minimum like that, it treat short TTLs (My current attitude is that if I have the need for short TTLs then I'm going to use them (though 300s is more than short enough for anything I envisage doing ATM & 300s seems to be reliable) and if someone's DNS cache is broken that is their problem, much like I don't pay any attention to issues Internet Explorer users might have looking at some HTML+CSS+JS I've inexpertly strung together.

Re: Stop using low DNS TTLs (2019)

#42

Post by bagels »

Is there a push mechanism in DNS?

My uninformed model is that it's all pull based, and for changes to propagate you basically have to wait for cached entries to expire. It seems to me that if there were a cache invalidation mechanism you could have fast updates, but long cache TTLs.

(I know that if this doesn't exist, adding it would be basically impossible, since some servers won't update to new protocols)

Re: Stop using low DNS TTLs (2019)

#43

Post by bagels »

cmeacham98 wrote: post #40
ikiris wrote: post #23

Earlier quoted context omitted.

And yet most of that doesn't matter, because you almost always have local resolvers that do cache properly.

Your average consumer router is a pile of garbage that barely works. I'll admit I don't have the stats in front of me, but I'd bet over half of them either don't attempt caching at all or do a horrible job at it. Of course, your upstream resolver (hopefully) does do some caching, but as most of the latency is getting to this resolver in the first place that doesn't really matter.

Consumer router doesn't seem to need to given that they typically have such low client counts. Your computer is doing caching, so there's not much benefit for your home router to also do caching.

Re: Stop using low DNS TTLs (2019)

#44

Post by donavanm »

toast0 wrote: post #16
jedberg wrote: post #13

DNS TTLs are fraught with peril. For one, a lot of clients ignore them. [0] Secondly, a lot of ISPs ignore them too. At every place I've worked, we've had trouble where ISPs just ignore TTLs and set them arbitrarily high. When I worked at reddit, it took a month for traffic to shed off of the old load balancer, despite me lowering the TTL to 5 seconds. And even then there was still some traffic, so I just had to cut…

I'd say about 80% of traffic moves within 2, maybe 3 TTLs. That 20% of traffic though... I've never really seen traffic on old servers go to zero real looking traffic, even after waiting months. Sometimes there's a dropoff when you send RSTs to SYNs, but sometimes not. If you're using a DNS service that charges per query, set the TTLs high! Otherwise, 60s is probably good enough for most failover. I wouldn't go too f…

From the CDN & authoritative name server side Ive mostly dealt with eyeballs and a smattering of devices. With ~1min TTLs itd be 80% in 5min, 95% in 20-30min, 99% at an hour, 99.9% 1day, the remainder entirely variable. Literally stopped announcing BGP prefixes for years, turn them back up, get immediate client traffic to 80 & 443. Have done down to 10s and lower and its fine (as far as we were concerned).

Re: Stop using low DNS TTLs (2019)

#45

Post by pastage »

dspillett wrote: post #41
Tepix wrote: post #35

Earlier quoted context omitted.

Quoting from https://cr.yp.to/djbdns/tinydns-data.html : Beware that cache times below 300 seconds will be treated as 300 by some clients, and NS cache times below 2 seconds can cause lookup failures.

I wonder what those clients are, none of these warnings ever give examples in my experience so I can't tell if this is old out-of-date information being repeated or it is an active modern problem. I know there was a DNS resolver that did worse than impose a minimum like that, it treat short TTLs ( My current attitude is that if I have the need for short TTLs then I'm going to use them (though 300s is more than short…

Getting DNS right is non trivial, and UDP will be dropped at times. If you have to lookup the name every two seconds means you have a high risk of getting failed lookups even if your infra team is competent. (10 minutes should be ok)

Re: Stop using low DNS TTLs (2019)

#46

Post by vinay_ys »

Tepix wrote: post #34

One cool thing about DJBs tinydns is that you can configure TTLs if you plan to roll over to another system at a particular point in time. I.e. you can have multiple records each with its own time range where it is valid. From the tinydns-data documentation: -- You may include a timestamp on each line. If ttl is nonzero (or omitted), the timestamp is a starting time for the information in the line; the line will be i…

Yeah, this works if it is a planned cutover. If you are having to ditch your current IP network in a hurry, then this approach won't work.

Re: Stop using low DNS TTLs (2019)

#47

Post by kazen44 »

occamrazor wrote: post #26
jiggawatts wrote: post #14

Earlier quoted context omitted.

We should do the right thing even if other people do the wrong thing. Traffic signs shouldn't be taken down just because some people speed regardless of the posted speed limit. Fundamentally however, giving clients control over when failover occurs means giving up control over failover. Literally just a couple of hours ago, I was patiently explaining this to some Azure techs from Microsoft. The conversation went like…

Aren’t routing tables also cached (not by clients, but by ISPs)? What is their typical TTL?

routing tables usually aren't "cached" in the same sense that DNS caches entries.

however. Inside ISP;s routes are usually distributed by an IBGP with a couple of route reflectors, and route reflectors will drop routes if no router within the IBGP domain is advertising the route.

Routes between ISP's are usually fairly static thanks to mechanisms like graceful restart, which makes sure routes do not get dropped, even if BGP has a restart between routers. THere are various other mechanism in which to achieve this. (look up route flap dampening if you want to learn more).

Re: Stop using low DNS TTLs (2019)

#48

Post by School-Cotton »

bagels wrote: post #42

Is there a push mechanism in DNS? My uninformed model is that it's all pull based, and for changes to propagate you basically have to wait for cached entries to expire. It seems to me that if there were a cache invalidation mechanism you could have fast updates, but long cache TTLs. (I know that if this doesn't exist, adding it would be basically impossible, since some servers won't update to new protocols)

How would that even work, since you can’t in general know who has cached the entries you control? Broadcast updates to the whole internet?

Re: Stop using low DNS TTLs (2019)

#49

Post by blueflow »

sofixa wrote: post #27
blueflow wrote: post #5

Earlier quoted context omitted.

It made as much sense as it does now, but differently from back then, people today neglect the performance and reliability impact it has.

Not really, because infrastructure is much more dynamic today. Maybe if your IPs never changed long TTLs are OK, but when you're running behind a cloud load balancer, with changing IPs and failover to different regions/providers, low TTLs make more sense.

Its understandable that you need to change your routing dynamically, but DNS is not the right tool for that.
Post reply on HN