Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

Post reply on HN
Print view View original

81–90 of 305 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#81

Post by fc417fc802 »

MayeulC wrote: post #52
bhouston wrote: post #32

I've run into issues with using public wifi when I override my MacBook's DNS server to 1.1.1.1 or 8.8.8.8. I believe this is because captive portals require custom resolution of the name captive.apple.com. And external DNS servers will not resolve that correctly to the local gateway's authorization page.

AFAIK (at least it worked like that some 10 years ago) the captive portal just intercepts the HTTP page load and inserts its own content (most often a 302). So it just has to be a http web page. Firefox uses http://detectportal.firefox.com/canonical.html Relevant support page, though light in details: https://support.mozilla.org/en-US/kb/captive-portal Edit: ah, yes, DNS can be hijacked too (requires intercepting out…

Can we take a minute to appreciate how utterly broken this state of affairs is? The dogged over centralization of DNS is an endless source of problems.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#82

Post by casey2 »

lpapez wrote: post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

This reasoning assumes you have access to infinite runway. You don't.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#84

Post by win311fwg »

eviks wrote: post #63
win311fwg wrote: post #58

Earlier quoted context omitted.

The existence of 1.1.1.1 speaks to a much larger design problem. If you want to talk about what should have been done, you need to step much, much further back.

I don't want to step back and go off topic

"Should" cannot be evaluated in a vacuum. The only thing that would be off-topic is pretending that it can be.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#85

Post by [deleted] »

vanviegen wrote: post #76
lpapez wrote: post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

[deleted]

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#86

Post by bcrosby95 »

vanviegen wrote: post #76
lpapez wrote: post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

The way I usually prevent having to scale out to a bazillion systems is never getting more than 10 users.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#87

Post by mayli »

toast0 wrote: post #16
mhitza wrote: post #5

Earlier quoted context omitted.

Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.

Using obviously better data structures the first time isn't premature optimization.

Maybe it's not that obviously better when it's impltd.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#88

Post by bch »

bcrosby95 wrote: post #86
vanviegen wrote: post #76

Earlier quoted context omitted.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

The way I usually prevent having to scale out to a bazillion systems is never getting more than 10 users.

The Art of Production

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#89

Post by froh »

bcrosby95 wrote: post #86
vanviegen wrote: post #76

Earlier quoted context omitted.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

The way I usually prevent having to scale out to a bazillion systems is never getting more than 10 users.

pro move. made my evening.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#90

Post by [deleted] »

pbhjpbhj wrote: post #45
seiferteric wrote: post #29

Earlier quoted context omitted.

You have to cache, cloudflare doesn't know all the records ahead of time, they have to do recursive lookups to the authoritative servers that own the records and that is only good for the period of the TTL of the record. There is no "global" DNS record database or something like that.

>that is only good for the period of the TTL of the record. Not really, TTLs are often short, but IPs might not change for years. You can probably generate your own TTL, at scale, and avoid many DNS requests.

[deleted]
Post reply on HN