Live data from Hacker News

Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

blog.cloudflare.com

131–140 of 305 posts

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#132
post #96
post #52

Earlier quoted context omitted.

AFAIK (at least it worked like that some 10 years ago) the captive portal just intercepts the HTTP page load and inserts its own content (most often a 302). So it just has to be a http web page. Firefox uses http://detectportal.firefox.com/canonical.html Relevant support page, though light in details: https://support.mozilla.org/en-US/kb/captive-portal Edit: ah, yes, DNS can be hijacked too (requires intercepting out…

I've had reliable success by using http://neverssl.com to force a basic HTTP connection for kickstarting a public WiFi portal login, although I have to disable NextDNS (iOS) too.

The ironic part of neverssl.com is that it does indeed now support SSL.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#133
Funny thing about cloudflare. I have a dns warming script that uses their top 1k or 10k addresses. Then when my master starts up it warms the entire cache. Everything else uses memcache so the cluster is nice and toasty. As far as I can tell no one else releases domain statistics like them.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#134

Earlier quoted context omitted.

Why would anyone want to use a DNS resolver that tampered with records on a large scale? The TTL is intentionally set by the originator of the record. Or alternatively, if you don't tamper why would I want to use a service that serves stale data?

Every distributed system serves stale data. You can define away ‘stale’ by picking a consistency model, but look inside the consistency machinery and you will see fresher data you aren’t allowed to have yet.

[deleted]

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#135

Earlier quoted context omitted.

This is Broadcom's business model

Quite. I was a VMware fanboi (25+ years, man and boy) I still look after a few VMware estates and a lot of Proxmox ones (that used to run VMware). Hilariously, VMware is described as "enterprise class", which I can only conclude means MVP and a bit wanky. Today I repaired a Proxmox HA + Ceph node using boring old normal Linux skills and as it turns out I have 30 years of those. Part way through a remote v8 to 9 upgra…

When I was first getting into software dev, I thought 'enterprise' meant 'industrial grade', 'powerful', 'high-performing'.

Then I actually met some enterprise software, and realised that it means 'expensive', 'bespoke', 'one-off', and usually 'janky'.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#136

> we store the records as a single Box containing each record encoded as a 2-byte length prefix followed by its raw bytes. Interestingly this is exactly how netlink works-ish: https://manpages.ubuntu.com/manpages/focal/man3/netlink.3.ht... You start, get the type & length, and then that is how many bytes you read. Some issues with that when you deserialize, from a raw stream in to `[u8; 4096]` buffer, the alignment i…

This kind of encoding[0] is ubiquitous in networking protocols. It scales down to small silicon well and enables the receiver to estimate resource requirements or skip parts of a serial byte stream without storing it in memory first. These encodings usually aren't aligned by design.

[0] https://en.wikipedia.org/wiki/Type–length–value

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#137
post #135

Earlier quoted context omitted.

Quite. I was a VMware fanboi (25+ years, man and boy) I still look after a few VMware estates and a lot of Proxmox ones (that used to run VMware). Hilariously, VMware is described as "enterprise class", which I can only conclude means MVP and a bit wanky. Today I repaired a Proxmox HA + Ceph node using boring old normal Linux skills and as it turns out I have 30 years of those. Part way through a remote v8 to 9 upgra…

When I was first getting into software dev, I thought 'enterprise' meant 'industrial grade', 'powerful', 'high-performing'. Then I actually met some enterprise software, and realised that it means 'expensive', 'bespoke', 'one-off', and usually 'janky'.

Enterprise means it has SSO and a support contract

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#138
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

> Produce working product first, validate the idea, stabilize the business, start generating profit, not everybody is so lucky to be able to go in that order? The first part requires upfront capital/investment?

So obviously you start at optimization

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#139
post #68

This is the right way to deliver software. Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs. In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

This assumes that you have plenty of cash to burn in the process, which is approximately correct for VC-backed ventures, and for offshoots of large corporations that play a lomg game.

Re: Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

#140
post #91

Earlier quoted context omitted.

Exactly, and you need to start turning a profit before the end of that runway. Even if that means running code that is suboptimal.

i suppose you could say the same about buying a house. just make that initial 300.000 and from there on out its easy. everything looks better on paper

My house is a ~700 sqft. condominium, gov. subsidized for lower income individuals, and even my mortgage is more than 300k… maybe I’m just basing my info off of coastal city prices, but is it possible to buy a reasonably nice home located in a reasonably nice amerikkkan city… for $300k in 2026?
Post reply on HN