Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

Post reply on HN
Print view View original

181–190 of 304 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#181

Post by kibwen »

f311a wrote: post #70
irdc wrote: post #4

This is why system programming still matters. Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.

Unfortunately, Rust is not a good choice for this kind of tricks. This is where Zig shines. In Rust, you can’t even use proper arenas, which can help a ton with allocations. Cloudflare started to pick Zig recently, for projects, that have memory constraints.

Rust supports arenas just fine ( https://crates.io/crates/bumpalo ), and if you mean the support for using custom allocators in the standard library collections, that's as stable as Zig is.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#182

Post by hdjrudni »

catlifeonmars wrote: post #159
8n4vidtmkvmk wrote: post #152

Earlier quoted context omitted.

Not from what I've seen. The desirable neighborhoods are 500-600k. Suburbs outside major city

Desirable neighborhoods are by definition expensive. The trick is to find a neighborhood you like where your home can just be a home and not a top-heavy investment. Acknowledging this isn’t always easy or possible, but just pointing out that this is a self reinforcing problem.

> The trick is to find a neighborhood you like where your home can just be a home and not a top-heavy investment

I meant desirable for me to live there, not as an investment. Who wants to buy a home in place they don't want to live?

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#183

Post by hdjrudni »

esseph wrote: post #158
8n4vidtmkvmk wrote: post #152

Earlier quoted context omitted.

Not from what I've seen. The desirable neighborhoods are 500-600k. Suburbs outside major city

You didn't read what I said. I mentioned acres of land. You normally don't have multiple acres of land in the suburbs.

Well then you didn't read parent's comment

> is it possible to buy a reasonably nice home located in a reasonably nice amerikkkan city… for $300k in 2026?

Who wants acreage? We want homes.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#184

Post by switchbak »

sandeepkd wrote: post #148
lpapez wrote: post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

Its a yes if you do not know the domain space, query patterns well enough and also if the cost of optimization or time for optimization may have detrimental impact to business. In this case it most likely means that the crowd in the room did not anticipate much on this in early phases and no one in the room pointed these things out. The irony is that these performance and disk numbers are heavily discussed as a part…

“changing data once its running in production is hard, changing the data structure is even harder”

100% agreement on this. There are a class of optimizations that can happen transparently. Those can happen at any time, and are fine to defer. Not all profiling and scalability improvements fall into this bucket. Some are very expensive to roll out, and ignoring these concerns can cause huge headaches down the line. Not fun to hear, but it’s definitely true. Even with LLMs, this can still be a huge challenge.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#185

Post by deathanatos »

bhouston wrote: post #32

I've run into issues with using public wifi when I override my MacBook's DNS server to 1.1.1.1 or 8.8.8.8. I believe this is because captive portals require custom resolution of the name captive.apple.com. And external DNS servers will not resolve that correctly to the local gateway's authorization page.

Dumb captive portals, which do still exist in some places, usually do MitM attacks on the connection, so you need some http(no-s) site that you can abuse as "yeah, this can get attacked by the WiFi" to then answer the portal.

The right way is that there's DHCP option for the network to signal "I have a captive portal", that's been standardized for over a decade.

… or … IDK … just stop shoving ads down people's throats just because they want WiFi.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#186

Post by sophacles »

toast0 wrote: post #175
sophacles wrote: post #116

Earlier quoted context omitted.

Imagine you're an engineer at cloudflare, an 8 year old (at the time of launch of 1.1.1.1) company. The company is wildly popular and any service launched is going to have a lot of traffic and a lot of attacks right away. Any problems with it are going to embarass the company a lot. You're tasked with making a DNS caching recursive resolver that can operate at a large scale and will be run on thousands of servers eac…

> The company is wildly popular and any service launched is going to have a lot of traffic and a lot of attacks right away. Doesn't this also inform you that your cache will be very large, so you shouldn't use growable structures with slack space when cache entries won't grow; slop space reduces the size of your cache. And also that the query volume will be high so the cached data should require as little work as pos…

RAM is cheap. I'd find myself far far more concerned with:

* unbounded growth of the cache and properly invalidating after TTL expires (a few GBs of slop is nothing on a server with 64 or more GBs of ram, unbounded growth is a problem).

* making sure the DNS implementation works correctly on both the serving side and recursive resolution side.

* What strategy is best for deduping recursive requests across machines (if something a few miliseconds away has a live result, why do a full lookup taking hundreds or thousands of milliseconds?). This potentially improves RAM usage across the datacenter too from not having a given record on dozens (or more) machines' local cache. I don't know exactly how they do it, but naively I'd look at some sort of DHT shaped solution to look for records in peers within the datacenter. Or maybe some sort of tiered caching with the upper tier being sharded on domain name or the like.

* The biggest performance gains cloudflare can provide in Web and DNS cache come from a cache hit. This is on the order of 10s or 100s of ms due to having a big cache and short distance to the requesting machine. A suboptimal lookup algorithm that is a few microseconds slower in local compute and ram access is just not as important as the other concerns for dedup and cache sharing. That's not to say it's unimportant, just that it's not the top priority when you're trying to deliver this much larger performance gains from other aspects of the system. Thats why they are getting to it several years after release.

Cloudflare writes a lot about distributed systems solutions to various problems. They likely don't think as hard about single machine performance as much as whole datacenter performance when approaching problems.

Keep in mind that the per-server cost of the whole program pre-optimization seems to be about 10GB (from the graph in the post). IME that's not bad for a big busy caching service.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#187

Post by toast0 »

perching_aix wrote: post #98
r3trohack3r wrote: post #37

Earlier quoted context omitted.

Rob Pikes 5 Rules of Programming: Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is. Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. Rule 3. Fancy algorithms are slow when n is…

> Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. Genuine question, is software performance really linear like that, that one can and should only fight the tightest bottleneck, one workload at a time? Never really sounded right. It also sounds like the typical sleight of hand where the difficult bit is simply laundered a layer up, in this case the choic…

It can be. Sometimes you take a profile and there's a big smoking gun and nothing else matters.

Sometimes it's a lot of small things everywhere and you can pick up significant performance after a lot of small value fixes. In this case, caching wire data instead of structured data is almost one of these, because the contribution to response time for serving a cache hit is small... otoh it happens so often than a small improvement matters; but this is a pretty focused use case, you usually hit the many smalln improvement issue in a less focused application where there are many code paths.

Sometimes the whole code structure / data structures are so wrong, but it works and perf is bad and profiling will never tell you. This article is not that case; these data structures only needed refinement.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#188

Post by elcritch »

bigbuppo wrote: post #142
taneq wrote: post #135

Earlier quoted context omitted.

When I was first getting into software dev, I thought 'enterprise' meant 'industrial grade', 'powerful', 'high-performing'. Then I actually met some enterprise software, and realised that it means 'expensive', 'bespoke', 'one-off', and usually 'janky'.

Enterprise quality software is a just a DOS application, probably written for dBase III, that has been rolled forward to the present day.

Those old school systems are often much more stable than any newer systems. Autozone looks to use something like that and I’ve never seen them have issues as a customer.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#189

Post by otterley »

inigyou wrote: post #172
otterley wrote: post #71

Earlier quoted context omitted.

It's not some sort of public law with public enforcement, but it is in the RFCs that govern the protocol. I should be a bit clearer here; the TTL is an upper bound on how long it can be cached. Caches are free to consult more frequently but not less frequently. That said, out of respect for upstream cache operators and authoritative servers, most DNS caches honor TTLs as best they can.

The IETF isn't the internet police. You don't have to follow its advice.

No, but if you didn’t, the internet wouldn’t really work all that well. It was the fact that participants, despite being independent, all agreed (either explicitly or implicitly) to adhere to the standards that it became a global network. If they hadn’t, the result would have been more of the same: independent networks that only had narrow interoperability at best. It’s a textbook example of how global cooperation can yield incredible results.

Advocating to do things against agreed-upon standards without a compelling reason and without giving due consideration to the adverse consequences is one of the hallmarks of a bad engineer. Even Microsoft played nice with Internet standards for the most part (although with some notable exceptions at the application layer that got them well-deserved criticism).

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#190

Post by inigyou »

otterley wrote: post #189
inigyou wrote: post #172

Earlier quoted context omitted.

The IETF isn't the internet police. You don't have to follow its advice.

No, but if you didn’t, the internet wouldn’t really work all that well. It was the fact that participants, despite being independent, all agreed (either explicitly or implicitly) to adhere to the standards that it became a global network. If they hadn’t, the result would have been more of the same: independent networks that only had narrow interoperability at best. It’s a textbook example of how global cooperation ca…

It was the fact that adhering to the standard was in the best interest of each participant. When it isn't, they don't.
Post reply on HN