Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

121–130 of 305 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#121
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

[dead]

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#123
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

> Produce working product first, validate the idea, stabilize the business, start generating profit,

not everybody is so lucky to be able to go in that order? The first part requires upfront capital/investment?

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#124
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

Only if you have loads of capital

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#125
post #51

Frankly weird that they were resorting to high level containers for this in the first place. Also, this line struck me as odd > Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads. jemalloc multithreaded performance is actually poor(ish) compared to other modern allocators, which makes it a weird choice. But even weirder is why they're even using an allocator in the first…

Rust supports punning via pointer casting, but you'll want to use #[repr(C)] on any data types used

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#126
The Record struct contains rtype and data where RecordData is a tagged union. Aren’t those two always in sync? Not a DNS expert, just wondering if this is redundant or there is a reason both are there. Doesn’t matter anymore if they store it already serialized but I would be interested why it was this way.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#127

Earlier quoted context omitted.

I wonder why Cloudflare didn’t think of this

This is Broadcom's business model

Quite. I was a VMware fanboi (25+ years, man and boy)

I still look after a few VMware estates and a lot of Proxmox ones (that used to run VMware).

Hilariously, VMware is described as "enterprise class", which I can only conclude means MVP and a bit wanky.

Today I repaired a Proxmox HA + Ceph node using boring old normal Linux skills and as it turns out I have 30 years of those. Part way through a remote v8 to 9 upgrade I think I lost comms due to using OpenvSwitch for networking and despite using tmux for the upgrade session. Anyway, the Proxmox ISO was useless for rescue but the classic systemrescuecd worked nicely and I could run dpkg in a chroot.

VMware "used" Linux and never really gave back. I don't miss fixing vCentres and all the other nonsense that "Enterprise" wankery has foisted on me over the years.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#129
post #115

> 56% A records, 25% AAAA, and 19% TXT And they say nobody uses IPV6.

It’s finally gaining some traction… https://www.google.com/intl/en/ipv6/statistics.html

no doubt because of scraping and the cost of IPv4

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#130

With my own MaraDNS, I aggressively optimized the memory usage of blacklist entries by having a single really big malloc() to allocate the memory for the entries, then traversing that memory block for potentially blacklisted entries. When I was using one malloc() per entry, a large blacklist took up 237 megabytes of memory. The same blacklist, once optimized to be loaded with a single malloc() call, only took up 9.5…

Why do I always find interesting new Twitter accounts just as the person is leaving :)
Post reply on HN