Live data from Hacker News

Tell HN: Cloudflare silently injects its analytics when you switch nameservers

news.ycombinator.com

191–200 of 217 posts

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#191
post #190

Earlier quoted context omitted.

I just double checked `about:preferences#privacy` is "Enhanced Tracking Protection: Strict" and clicking "Advanced Settings" confirms the radio button is indeed under "Strict". Firefox 153.0.4

Just saw the edit, all clicked now. As described this is expected behavior. The entitieslist (can't link right now because Github is down but got a local copy for some experiments) contains exceptions for owners of tracking URLs, in this case as a resource for Cloudflare.com and others owned by them only. Basically, because they are the same entity, they are considered one. Whether that could be communicated better b…

Yeah, I was trying to get hold of the list from Github myself before posting the edit. It is unfortunate strict isn't truly strict, but at least now I know. Thanks for that.

Pending Github fixing itself, could you confirm if `browser.events.data.microsoft.com` is on your local copy ?

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#192
post #165

Earlier quoted context omitted.

I don't buy that, there is no reason for 99.9999% of sites to use cloudflare. The "reason" to use cloudflare is hype.

It's the reason for its existence and the reason people are choosing it. If they are making a mistake, hoodwinked by hype, that's totally unrelated to the point being discussed. I think you've just seized the opportunity to post your disdain for them without regard to the actual topic being discussed.

Truly doubt that. Sure, but if they are making a mistake due to dark patterns then it is another thing.

This thread does indicate the latter.

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#193
post #190

Earlier quoted context omitted.

Just saw the edit, all clicked now. As described this is expected behavior. The entitieslist (can't link right now because Github is down but got a local copy for some experiments) contains exceptions for owners of tracking URLs, in this case as a resource for Cloudflare.com and others owned by them only. Basically, because they are the same entity, they are considered one. Whether that could be communicated better b…

Yeah, I was trying to get hold of the list from Github myself before posting the edit. It is unfortunate strict isn't truly strict, but at least now I know. Thanks for that. Pending Github fixing itself, could you confirm if `browser.events.data.microsoft.com` is on your local copy ?

Yeah, agree, purely from a user expectation perspective, strict sounds like it'd suppress everything, not just third-party, especially with lower compatibility turned on.

Yeah "browser.events.data.microsoft.com" is in, but mainly because it just ends up under microsoft.com anyways, subdomains are accepted inside the entitieslist.

Here the full copy (initially wanted to share via Pastebin but some links triggered a spam filter): https://cdn.jsdelivr.net/gh/mozilla-services/shavar-prod-lis...

If you or anyone else reading is interested, Mozilla has written some pleasant docs, lots to learn, easy to understand, comprehensive. Could not even consider what I am attempting without the resources they've provided: https://firefox-source-docs.mozilla.org/toolkit/components/a...

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#194
post #156

Earlier quoted context omitted.

Many years ago I started to describe how it works in an HN comment and some reply complained about the idea of terminating TLS, i.e., decrypting, and then re-encrypting. Obviously this sacrifices something, e.g, speed, in order to gain _control_ But this is what Cloudflare does and no one seems to mind Large companies also do this to protect their LANs I'm not running a CDN, only a small home LAN. I'm only procesing…

This sounds like exactly the kind of thing I'd waste a weekend setting up but what do you actually do with the decrypted traffic? What do you inject? I know you said it's personal but maybe some basic ideas. Do you find any websites or services that fail because of cert pinning or similar? Why do you restrict dns caching to periodic intervals, just for external privacy? In terms of the speed I doubt the time to decry…

"... but what do you actually do with the decrypted traffic?"

I read it

On the most basic level, consider logging

I can add response headers with additional information about the request and these are stored in the log

I can create a comprehensive log of every HTTP request, stored in a text file or SQL database

I store these logs in an unpopular compression format that allows searching tar'd and compressed files using regex without decompressing them

(This is not possible using zstd, gzip, etc.)

I use this "browsing history" with a variety of programs

Compare this with the design paradigm of storing history inside each individual program, as is the case with a so-called "modern" graphical web browser

Other basic proxy functions that I use are (a) control over HTTP headers, including cookies, (b) control over TLS settings, including SNI, and (c) rewriting URLs

The settings are site-specific and/or URL-specific. Lists of websites that have special requirements, e.g., need for SNI, specific HTTP headers, TLS version, HTTP versions, etc., are stored in text files. These are mapped into the proxy's memory. Not every website has the same requirements

I also use CDN IP address ranges to make decisions about what requirements a website might have

Compare this with trying to control these things from within each individual program. In some cases, it is not possible. Or compare with trying to control these things with Javascript. Generally I don't use a graphical browser. Nor do I use NodeJS. I use Lua. As such, Javascript is not the best solution for me

Having lists of which websites have some specific requirement(s) allows me to learn about the sites I visit in ways that a so-called "modern" web browser does not

The same goes for controlling DNS. I learn about where website operators outsource DNS, how DNS providers (mis)configure their services and when DNS data changes or remains static for years

Perhaps the most important proxy function for me is the ability to avoid DNS lookups. The IP address data is stored in an SQL database using a simple, custom format, extracted to a text file and mapped into the proxy's memory

Using a proxy allowed me to move away from storing DNS data in, and serving it from, zone files

This is only a partial list of functions that I use. I like to experiment

Proxies today can do much more than basic functions like the ones I mentioned

For example, injecting Javascript or modifying an HTML tag before it reaches a browser is trivial, albeit inefficient from a purist perspective

This is not something I need to do most of the time. I have relatively simple needs. Most times, I'm not using a graphical browser. I'm using the command line in textmode, no X11, etc.

Cloudflare has a different use case

What does Cloudflare actually do with the decrypted traffic. Only CF knows

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#195
post #71
post #64

Earlier quoted context omitted.

They have a product called Magic Transit that offers DDoS protection and such for plain IP traffic, where Cloudflare does not terminate TLS. Pricing is not public but starts in the five-digit USD per month range according to people I talk to. This may tell you something about how keen Cloudflare are to handle traffic they themselves cannot decrypt.

Magic Transit uses BGP magic to work. That only makes sense at scale - i believe you have to have your own ASN for it to work. Realistically its a totally different product, and 5 digit price is probably cheap relative to competitors in that space.

Nope, you don't need your own AS for this to work, nor do you need to use BGP. They support static routes too. BGP peering support is actually "beta" according to Cloudflare: https://developers.cloudflare.com/magic-transit/get-started/...

You do ideally want your own /24, though even that's not a hard requirement. And it can be provider-assigned space as long as your provider is willing to sign an LOA for you.

As for competitors, there are a few that start in the low 4 digits per month for similar services. That's not to say Cloudflare doesn't have anything unique to offer though, they're great at scale and standardization.

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#196

Earlier quoted context omitted.

Cool, so you went from > Cloudflare needs the data. [...] If someone is mad, maybe they should pay or use a different architecture. To "they would violate GDPR". Are you ok with that behavior then? And are you basing your ethical decision solely on the current legislation?

I never said "they would violate GDPR". I was questioning if it could be the case. Someone said it's not the case as the feature is turned off in EU. Reading their blog, it also looks like they don't collect much identifiable data. When selecting free tier of such commercial MITM, opt-in RUM data collection is really far from the top ethical concern. I'm not saying it's a good thing and I'm not deploying their soluti…

If it's turned off for EU customers, it's a pretty clear case that it would be in violation of GDPR, wouldn't it?

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#197

Earlier quoted context omitted.

Say I'm a vegetarian restaurant, and I contract out my delivery service. One day the delivery service starts adding free sausages to every delivery on an opt-out basis. By the time you realize, they'll already have delivered adulterated meals. You won't have a lot of customers left! People are often vegetarian for ethical reasons. In a similar way, many people have Ethical Opinions (tm) about Analytics as well. They…

For delivery service, it's typically done with a closed box so you get exactly what has been prepared. Here you told them to unpack the food and repack it (TLS termination), filtering order (WAF), reheating pre-made food (cache) and they offer service to even prepare food or part of the food (edge compute). When your food is not available they will serve their own (error pages), with their brand visible. At that leve…

[deleted]

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#198

Earlier quoted context omitted.

Say I'm a vegetarian restaurant, and I contract out my delivery service. One day the delivery service starts adding free sausages to every delivery on an opt-out basis. By the time you realize, they'll already have delivered adulterated meals. You won't have a lot of customers left! People are often vegetarian for ethical reasons. In a similar way, many people have Ethical Opinions (tm) about Analytics as well. They…

For delivery service, it's typically done with a closed box so you get exactly what has been prepared. Here you told them to unpack the food and repack it (TLS termination), filtering order (WAF), reheating pre-made food (cache) and they offer service to even prepare food or part of the food (edge compute). When your food is not available they will serve their own (error pages), with their brand visible. At that leve…

> If you are vegan and care about your customers (conservative on data collection and JS), maybe you should use a vegan architecture and not give your last-mile to the cheapest (free-tier) all-purpose repacking kitchen and delivery service.

Right, that's the lesson being learned here today.

To be fair, up 'till now, afaik these services actually have been trustworthy on this count, so this is a new low.

Just because people extend you a lot of trust to do A,B, and C doesn't mean you can't lose it if you randomly start doing D instead.

I don't think we can run a viable (high trust) western economy if we constantly have to beware of our own providers. Principle of least surprise applies.

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#199
post #155

Earlier quoted context omitted.

You get something for free, they announce it and you complain about not having a bigger announcement? If they had the banner and you missed it [or someone else] they would complain about the banner not being big enough.

> they announce it Forgive me for not subscribing to their blog? That's not how you announce changes to customers. Using a Level 7 CDN is based on trust. This is not a trustworthy action, period.

Maybe you're right. The important thing is to remember this moment the next time you're signing up for a free product that seems just a little too good to be true.

Re: Tell HN: Cloudflare silently injects its analytics when you switch nameservers

#200

Earlier quoted context omitted.

Many years ago I started to describe how it works in an HN comment and some reply complained about the idea of terminating TLS, i.e., decrypting, and then re-encrypting. Obviously this sacrifices something, e.g, speed, in order to gain _control_ But this is what Cloudflare does and no one seems to mind Large companies also do this to protect their LANs I'm not running a CDN, only a small home LAN. I'm only procesing…

NB. I actually do not encrypt then re-encrypt for the majority of HTTP requests I make I generate HTTP myself using own programs and connect to the localhost proxy using various TCP clients. The proxy does the encryption and remote connections not the client programs I process response bodies, using own software, into SQL, CSV, simple HTML or plain text I'm not using a graphical web browser to make HTTP requests This…

*preferences
Post reply on HN