Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

Post reply on HN
Print view View original

51–60 of 304 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#51

Post by rfgplk »

Frankly weird that they were resorting to high level containers for this in the first place. Also, this line struck me as odd

> Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads.

jemalloc multithreaded performance is actually poor(ish) compared to other modern allocators, which makes it a weird choice. But even weirder is why they're even using an allocator in the first place compared to a va MAP_ANON | MAP_NORESERVE arena carveout approach? You can also do punning that way too, which I'm not even certain if Rust supports?

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#52

Post by MayeulC »

bhouston wrote: post #32

I've run into issues with using public wifi when I override my MacBook's DNS server to 1.1.1.1 or 8.8.8.8. I believe this is because captive portals require custom resolution of the name captive.apple.com. And external DNS servers will not resolve that correctly to the local gateway's authorization page.

AFAIK (at least it worked like that some 10 years ago) the captive portal just intercepts the HTTP page load and inserts its own content (most often a 302). So it just has to be a http web page. Firefox uses http://detectportal.firefox.com/canonical.html

Relevant support page, though light in details: https://support.mozilla.org/en-US/kb/captive-portal

Edit: ah, yes, DNS can be hijacked too (requires intercepting outgoing traffic on port 53 therefore incompatible with DoH), that may require fewer computing resources. Still need http otherwise the server cannot use the correct cert chain.

Edit 2: Wikipedia says both methods are used: https://en.wikipedia.org/wiki/Captive_portal and also mentions RFC 8910. I suspected something like that existed, hence my initial disclaimer.

My point was: that domain is not treated any differently from other domains.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#53

Post by sergq »

0xAstro wrote: post #49

It's weird that it took so long for these trivial optimizations but it might just be that they were working on optimizing other stuff.

this applies to more than DNS caches. In 1998 I mailed Microsoft a proposal to replace search engine crawlers with a push-based filesystem monitor (detect change → extract → compress → push to index). Got a 5-line rejection letter. They built the same thing 20 years later as IndexNow. Full story with the original letter: https://dev.to/andrew_vl/in-1998-i-proposed-push-based-searc...

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#54

Post by otterley »

pbhjpbhj wrote: post #45
seiferteric wrote: post #29

Earlier quoted context omitted.

You have to cache, cloudflare doesn't know all the records ahead of time, they have to do recursive lookups to the authoritative servers that own the records and that is only good for the period of the TTL of the record. There is no "global" DNS record database or something like that.

>that is only good for the period of the TTL of the record. Not really, TTLs are often short, but IPs might not change for years. You can probably generate your own TTL, at scale, and avoid many DNS requests.

In DNS, the owner of each record has full control over its TTL. Intermediary DNS servers are required to honor them and are not permitted to replace TTLs with their own.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#55

Post by [deleted] »

eviks wrote: post #44
r3trohack3r wrote: post #37

Earlier quoted context omitted.

Rob Pikes 5 Rules of Programming: Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is. Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. Rule 3. Fancy algorithms are slow when n is…

> Data structures, not algorithms, are central to programming So you agree that they should've designed the system to use the appropriate data structure from the beginning?

[deleted]

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#56

Post by otterley »

mannyv wrote: post #23

One question the article doesn't answer is: why are they cacheing at all? If your cache is that big it isn't a cache. How much bigger is the dataset in question? There are 250 billion entries. Assuming 80/20, that implies 1.25 trillion records? What's the speed of service/response time relative to the data source? At that point it might be enough to replace your multiple caches with fewer in-RAM databases? It's an in…

The simple answer is that if you didn't cache, DNS traffic would skyrocket, and the load would pile up on the authoritative servers, which were intended to be small, and during the early days of the Internet, were frequently on bandwidth-constrained links.

DNS is designed to distribute query load to the edge as much as possible, and that's enabled by caching. It just so happens that "the edge" is now becoming concentrated among a small set of providers because they wanted to make a business out of it.[1] They knew that this would be expensive going in, though.

[1] Nobody has to use 8.8.8.8 or 1.1.1.1. Most people can use their ISP's cache or a local cache instead without any noticeable difference in behavior.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#57

Post by tehlike »

sdcfgy wrote: post #27
irdc wrote: post #4

This is why system programming still matters. Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.

System programming always matters. Things are cheap until they aren't one day.

things are cheap until you reach a scale.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#58

Post by win311fwg »

eviks wrote: post #44
r3trohack3r wrote: post #37

Earlier quoted context omitted.

Rob Pikes 5 Rules of Programming: Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is. Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. Rule 3. Fancy algorithms are slow when n is…

> Data structures, not algorithms, are central to programming So you agree that they should've designed the system to use the appropriate data structure from the beginning?

The existence of 1.1.1.1 speaks to a much larger design problem. If you want to talk about what should have been done, you need to step much, much further back.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#59

Post by ButlerianJihad »

otterley wrote: post #54
pbhjpbhj wrote: post #45

Earlier quoted context omitted.

>that is only good for the period of the TTL of the record. Not really, TTLs are often short, but IPs might not change for years. You can probably generate your own TTL, at scale, and avoid many DNS requests.

In DNS, the owner of each record has full control over its TTL. Intermediary DNS servers are required to honor them and are not permitted to replace TTLs with their own.

Actually that is not true. The IETF has expanded the definition of “TTL” and explicitly permits resolvers to serve “stale” RRs beyond their expiration time.

https://www.rfc-editor.org/info/rfc8767/

As a corollary, there is obviously no floor on refetching unexpired RRs, of course, except for efficiency concerns.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#60

Post by senderista »

rfgplk wrote: post #51

Frankly weird that they were resorting to high level containers for this in the first place. Also, this line struck me as odd > Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads. jemalloc multithreaded performance is actually poor(ish) compared to other modern allocators, which makes it a weird choice. But even weirder is why they're even using an allocator in the first…

I would also have instinctively reached for a large VM reservation to exploit demand paging. I have used that pattern a lot in C++ but not in Rust, so I don't know how difficult it would be to implement there.
Post reply on HN