Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

Post reply on HN
Print view View original

161–170 of 306 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#161

Post by f311a »

chlorion wrote: post #145
f311a wrote: post #70

Earlier quoted context omitted.

Unfortunately, Rust is not a good choice for this kind of tricks. This is where Zig shines. In Rust, you can’t even use proper arenas, which can help a ton with allocations. Cloudflare started to pick Zig recently, for projects, that have memory constraints.

I'd like to know why I can't use arenas in rust? Especially considering that I have used them before in rust.

You can’t allocate collections without nightly or without reimplementing them in the library. Every implementation uses it’s own set of trade offs to provide safety in unsafe implementation.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#162

Post by didgetmaster »

Why do people seem to think that optimization is something you only have to deal with once the software scales so much that 100s of TB of memory or disk space (or thousands of hours of processing time) are being wasted.

It is almost like nobody even thought during the design phase about what might happen down the road.

This is why so much software is bloated and often buggy. Just gets something that half-way works out the door ASAP and worry about the rest later (too often, never).

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#163

Post by Dylan16807 »

So they optimized from Vec to Box, but they're still using Box all over and spending 16 bytes on it? The things they're boxing need 2 bytes for length, and their memory use is low enough that they could cram the pointers into 4 bytes. Trying to pack that into 6 bytes is probably too much fuss for the benefit, but I see no reason to use more than 8 bytes.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#164

Post by jacquesm »

didgetmaster wrote: post #162

Why do people seem to think that optimization is something you only have to deal with once the software scales so much that 100s of TB of memory or disk space (or thousands of hours of processing time) are being wasted. It is almost like nobody even thought during the design phase about what might happen down the road. This is why so much software is bloated and often buggy. Just gets something that half-way works ou…

It can be quite hard to predict where particular usage patterns will take a piece of software under extreme load, especially with things that have lots of internal state. Obviously when you get to spend 100 T or more the pay off of an optimization is much larger than what it is in the case of 1T or less, and your typical developer is not going to have that kind of memory even in aggregate to play with. I tend to be forgiving when it comes to watching software bloat that I did not cause myself (and yet, I'm frustrated that Ubuntu's start-up greeting message takes a whopping 500 M).

In the case of internet infrastructure I don't think there was anybody even up to the year 2000 who had any idea of how bit this was going to be. And even now we have IPV4 and lots of legacy to deal with. Cloudflare is not my favorite company, let's put it like that, but in this case they show how the sausage is made and I think that should be applauded. Much better than 'why were down again for X hours'.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#165

Post by jacquesm »

phoghed wrote: post #137
taneq wrote: post #135

Earlier quoted context omitted.

When I was first getting into software dev, I thought 'enterprise' meant 'industrial grade', 'powerful', 'high-performing'. Then I actually met some enterprise software, and realised that it means 'expensive', 'bespoke', 'one-off', and usually 'janky'.

Enterprise means it has SSO and a support contract

It means you are paying for a support contract. Whether you actually have one time will tell.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#166

Post by Dylan16807 »

vanviegen wrote: post #76
lpapez wrote: post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

It was already reasonably lean. If they had 10 bazillion systems, they now need somewhere between 6 and 8 bazillion systems.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#167

Post by NIckGeek »

vinkelhake wrote: post #30

These seem like some fairly standard approaches for reducing memory usage. I can't help to think that the approach of joining several distinct list into a single one in some way undercuts Rust's safety guarantees. If you previous had three distinct Vec objects, then Rust would guarantee that you can't index out of bounds. If you now put all those objects into a single Vec and rely on offsets, then you now open the do…

You can make a wrapper type that abstracts the offset lookup logic with a safe interface. If it's a transparent struct then rust will compile it away into nothing but you still get the abstraction in your code.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#168

Post by inigyou »

kccqzy wrote: post #110
edflsafoiewq wrote: post #67

General theme: A programming language's native in-memory object format is typically optimized for random access, uniformity, and mutability (fields at fixed offsets, etc). Serialization formats for network or disk tend to be designed explicitly to be more compact. But you can design your own in-memory representation too, with the properties you need.

That’s the old school of thought. These days, designers of newer serialization formats realize that designing a more compact format doesn’t really buy much on modern CPUs and modern networks. See for example Cap’n Proto (whose inventor, kentonv, also works at Cloudflare) and flatbuffers.

That's also the ancient school of thought, before compaction was viable and before portability was needed.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#169

Post by inigyou »

superze wrote: post #131

How much is this in euro or do we measure money in ram now?

Currently $15 per GB, he saved Cloudflare $1,500,000 and got exactly $0 bonus. He must really believe in cloudflare's vision (global enshittification). In related news, three times today Cloudflare told me that I'm a bot and shall not pass - not that it needs to check if I'm a bot before it lets me pass.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#170

Post by inigyou »

pocksuppet wrote: post #104
auspiv wrote: post #36

Earlier quoted context omitted.

I think the question is probably more along the lines of - why not do a database with 100 TB of storage/records instead of a cache? tomato / tomato.. especially with smart caching in front of database. 100TB of flash is a good bit cheaper than 100TB of memory

It's not 100TB of data. It's probably 50 GB of data on each of 2000 servers. Because it's a cache. What is the point of a central cache if it's as slow to access as the original data?

TFA gives numbers closer to 5GB.
Post reply on HN