Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
blog.cloudflare.com
Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
1–10 of 303 posts
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#2> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec
Were there no design discussions/reviews when the system was setup to catch trivial things like this?
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#3> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this?
it was working so no one thought to check
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#4This is why system programming still matters.
Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#5> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this?
Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#6> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this?
It is often not worth optimising in the early days. You don't know how popular it will become, you might not know how many DNS records you will hold, it was possibly written in an earlier language and ported as-is.
At the point someone queries the 100TB of RAM, then maybe it is worth revisiting but even that has risks. You have to design the migration path, have fallback mechanisms etc.
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#7> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this?
Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.
How does that fit? What would be the evil of not wasting memory for many years at 1x?
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#8Earlier quoted context omitted.
Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.
How does that fit? What would be the evil of not wasting memory for many years at 1x?
One of the "evils" of premature optimization is how much time you spend on the optimization vs. the benefit you get from it. If your goal is correctness and shipping fast and you're not memory constrained then spending time using the least amount of memory is a waste of time specifically because you want to ship fast.
Another interesting thing that happens is you don't necessarily know what form your actual optimizations will need to take. Later when your systems grow you discover the suboptimal parts you hadn't optimized for.
Very early on at Cloudflare I worked on part of the DNS infrastructure that took DNS records from the UI and got them in a state for actual authoritative serving. The system had been constructed anticipating Cloudflare having millions of customers with unique domains, but it had not been constructed for a single customer with a single domain with millions of records. This caused a periodic slow down in DNS record updating while the system churned on that one customer.
In a different job I worked on a piece of optimization software that needed to keep track of "node" A is reachable from node "B". This had been implemented as a matrix (literally a malloced NxN matrix of ints storing 0 or 1) which worked really well for small systems. But you'd be out of memory really fast on a large project. I replaced the matrix with a hash table and all was good because the matrix was actually really sparse.
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#9Earlier quoted context omitted.
Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.
How does that fit? What would be the evil of not wasting memory for many years at 1x?
Engineers are expensive, especially good system engineers who are trained in your code base. Very possible that this just hadn't gotten to the top of the priority list.
Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
#10This is why system programming still matters. Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.
For the curious, this is technically possible in Rust using a dynamically sized type [1], but in practice is difficult and doesn't really play nice with the rest of the language. The nomicon entry concludes with "Yes, custom DSTs are a largely half-baked feature for now." [2]
[1] https://doc.rust-lang.org/reference/dynamically-sized-types....