Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

Post reply on HN
Print view View original

11–20 of 305 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#11

Post by eviks »

lbriner wrote: post #6
eviks wrote: post #2

> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this?

It is often not worth optimising in the early days. You don't know how popular it will become, you might not know how many DNS records you will hold, it was possibly written in an earlier language and ported as-is. At the point someone queries the 100TB of RAM, then maybe it is worth revisiting but even that has risks. You have to design the migration path, have fallback mechanisms etc.

It's also often that you can avoid all those future migration/fallback risks and pains if you invest a little bit of design thinking upfront.

So how would you decide which path to take in situations like this?

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#12

Post by strenholme »

With my own MaraDNS, I aggressively optimized the memory usage of blacklist entries by having a single really big malloc() to allocate the memory for the entries, then traversing that memory block for potentially blacklisted entries.

When I was using one malloc() per entry, a large blacklist took up 237 megabytes of memory. The same blacklist, once optimized to be loaded with a single malloc() call, only took up 9.5 megabytes of memory.

https://samboy.github.io/blog/entries/MaraDNS.html#BlogEntry...

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#13

Post by cobalt »

irdc wrote: post #4

This is why system programming still matters. Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.

less ergonomic, but still totally doable

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#14

Post by eviks »

gbear605 wrote: post #9
eviks wrote: post #7

Earlier quoted context omitted.

How does that fit? What would be the evil of not wasting memory for many years at 1x?

Engineers are expensive, especially good system engineers who are trained in your code base. Very possible that this just hadn't gotten to the top of the priority list.

I don't understand why you need training on your code base to design a cache format for read only vs rw workloads, but anyway yours is a comment about neglect, not the "evil" that would happen if you did that design

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#15

Post by stickfigure »

jgrahamc wrote: post #8
eviks wrote: post #7

Earlier quoted context omitted.

How does that fit? What would be the evil of not wasting memory for many years at 1x?

One of the "evils" of premature optimization is how much time you spend on the optimization vs. the benefit you get from it. If your goal is correctness and shipping fast and you're not memory constrained then spending time using the least amount of memory is a waste of time specifically because you want to ship fast. Another interesting thing that happens is you don't necessarily know what form your actual optimizat…

Absolutely true, but I will say that LLMs have changed the equation somewhat.

With a rather short prompt, claude/codex will take your code, write a harness, profile it, build experiments, profile those, and give some pretty solid advice which one to pick. Then integrate the changes. It's the kind of goal-directed, bite-sized job that LLMs excel at. Extremely low-commitment.

Except for the whole "making changes in production at scale" problem, of course.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#16

Post by toast0 »

mhitza wrote: post #5
eviks wrote: post #2

> Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this?

Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.

Using obviously better data structures the first time isn't premature optimization.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#17

Post by OptionOfT »

> we store the records as a single Box containing each record encoded as a 2-byte length prefix followed by its raw bytes.

Interestingly this is exactly how netlink works-ish: https://manpages.ubuntu.com/manpages/focal/man3/netlink.3.ht...

You start, get the type & length, and then that is how many bytes you read.

Some issues with that when you deserialize, from a raw stream in to `[u8; 4096]` buffer, the alignment is only guaranteed to be on 1 byte, not 4 bytes.

In practice it is 4 bytes, but if you run those tests with Miri, you'll get yelled at. So the fix there is to declare the buffer with a type that mandates the alignment of the largest type that you're going to be deserializing.

So then you start your buffer as follows: `[u32; 1024]`, and with `slice::from_raw_parts` you get to turn that into `[u8; 4096]` with the expected alignment.

As an exercise I wrote a streaming parser for netlink, the current existing package serializes everything, all at once.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#18

Post by win311fwg »

eviks wrote: post #14
gbear605 wrote: post #9

Earlier quoted context omitted.

Engineers are expensive, especially good system engineers who are trained in your code base. Very possible that this just hadn't gotten to the top of the priority list.

I don't understand why you need training on your code base to design a cache format for read only vs rw workloads, but anyway yours is a comment about neglect, not the "evil" that would happen if you did that design

> I don't understand why you need training on your code base to design a cache format

Because anyone willing to come in just to design your cache format is going to expect payment that is many multiples more than the engineers you already cannot afford? Long-term employees cost less, which brings them closer to being affordable, but you have to be able to keep them busy for long periods of time to realize that reduction in cost. A engineer who doesn't understand your codebase isn't going to be useful for very long.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#19

Post by mannyv »

toast0 wrote: post #16
mhitza wrote: post #5

Earlier quoted context omitted.

Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.

Using obviously better data structures the first time isn't premature optimization.

There was a reason for that field, but that reason never panned out.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#20

Post by suriyaG »

eviks wrote: post #11
lbriner wrote: post #6

Earlier quoted context omitted.

It is often not worth optimising in the early days. You don't know how popular it will become, you might not know how many DNS records you will hold, it was possibly written in an earlier language and ported as-is. At the point someone queries the 100TB of RAM, then maybe it is worth revisiting but even that has risks. You have to design the migration path, have fallback mechanisms etc.

It's also often that you can avoid all those future migration/fallback risks and pains if you invest a little bit of design thinking upfront. So how would you decide which path to take in situations like this?

It only looks super obvious in hindsight and the well explained blog post. when a team of 5 is tasked with getting a completely new DNS up at the scale and integrate well with cloudflare.

if you spend cycles on nitty gritty opinions like this time to market goes out further and further out. some napkin math, 130 gen13 servers cost "only" ~$2.6M. relative to the importance of the 1.1.1.1 and the market at the time. that is nothing to cloudflare.

this is not to say good system design does not matter. it very much does, but making that call at that time would've butchered the prodcut very much similar to google+, youtube etc.

Post reply on HN