Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

231–240 of 304 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#231

Earlier quoted context omitted.

Not from what I've seen. The desirable neighborhoods are 500-600k. Suburbs outside major city

Desirable neighborhoods are by definition expensive. The trick is to find a neighborhood you like where your home can just be a home and not a top-heavy investment. Acknowledging this isn’t always easy or possible, but just pointing out that this is a self reinforcing problem.

Assuming you are not married.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#232

Why do people seem to think that optimization is something you only have to deal with once the software scales so much that 100s of TB of memory or disk space (or thousands of hours of processing time) are being wasted. It is almost like nobody even thought during the design phase about what might happen down the road. This is why so much software is bloated and often buggy. Just gets something that half-way works ou…

I recently heard a great analogy for this exact problem under the premise of "Make it work, make it right, make it fast".

Assume a sorting algorithm as a metaphor for your whole program.

You make it work by implementing the simplest thing you know how to write: bubble sort. Works.

At the end you notice it's way too slow and replace with something much better: quicksort.

Now, how much of your program is surviving? Almost nothing, perhaps except for the "greater than" comparison.

If you apply that idea to a real world program, we're pretty much talking about a full rewrite.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#233
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

What I find surpring here is this being about DNS. Simple optimization should have been done maybe 1997. Letting it build up to 100 TB is noteworthy, but on the other hand for IT that's common. It's surprising but also totally expected...

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#234

Earlier quoted context omitted.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

Remember that everything has an opportunity cost. Running a lot of servers might cost $10 million annually, but if the product team had to choose between a project that would recoup $5 million of that vs. an opportunity to earn $50 million ARR for the same amount of work, the logical answer would be obvious.

In my experience optimizations actually preformed tend to have ridiculously high ROI because they are so rarely prioritized.

Better performance = saving money + better user experience.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#235
post #135

Earlier quoted context omitted.

When I was first getting into software dev, I thought 'enterprise' meant 'industrial grade', 'powerful', 'high-performing'. Then I actually met some enterprise software, and realised that it means 'expensive', 'bespoke', 'one-off', and usually 'janky'.

Enterprise means it has SSO and a support contract

And in-transparent pricing and this annoying dude that asks you once in a while if everything is okay, usually before the renewal.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#237
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

>optimization is by far the easiest part of the process

Not if the whole thing is architected poorly but was a requirement of the hour so it became big. Then optimisation becomes an art, but definitely not the ‘easiest part of the process’

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#238

I've run into issues with using public wifi when I override my MacBook's DNS server to 1.1.1.1 or 8.8.8.8. I believe this is because captive portals require custom resolution of the name captive.apple.com. And external DNS servers will not resolve that correctly to the local gateway's authorization page.

Dumb captive portals, which do still exist in some places, usually do MitM attacks on the connection, so you need some http(no-s) site that you can abuse as "yeah, this can get attacked by the WiFi" to then answer the portal. The right way is that there's DHCP option for the network to signal "I have a captive portal", that's been standardized for over a decade. … or … IDK … just stop shoving ads down people's throat…

I’ve never even heard of the DHCP option for this. How widely adopted is it? It’s surprising to hear it called “the right way” when it wouldn’t work for a single-stack IPv6 (or IPv6-mostly, probably) network.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#239
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

You underestimate the cost of "optimization", sometimes it means actually rebuilding large parts of the system. I would not say it is the "easiest" part, but it is usually not what will kill your business though.
Post reply on HN