Eric Lawrence (PM on Edge, previously on Chrome, previously author of fiddler) showing screenshots of documentation that explains all that, and saying it's been documented this way for 14 years: https://twitter.com/ericlaw/status/1152933704198758401
Edge sends full URLs of pages visited to Microsoft
121–130 of 161 posts
Re: Edge sends full URLs of pages visited to Microsoft
#122Is there even a "pure" browser anymore? By that, I mean one which by default will do this: if I enter a URL in the address bar, it will fetch that page and its associated resources. If I click a link on a page, it will fetch its destination (this is obviously a simplified view, ignoring things like JS on the page etc.) No other network activity, no "recommendations", no other attempts to be "smart" or "helping" by do…
Re: Edge sends full URLs of pages visited to Microsoft
#123Eric Lawrence (PM on Edge, previously on Chrome, previously author of fiddler) showing screenshots of documentation that explains all that, and saying it's been documented this way for 14 years: https://twitter.com/ericlaw/status/1152933704198758401
Can’t it just send a hash of the domain and of the full url for checking?
For example they could hash the domain, path and query separately.
Re: Edge sends full URLs of pages visited to Microsoft
#124Every time I see people on HN campaigning for Microsoft as some kind of reformed tech company that has seen the errors of its past and turned into a force for good, something like this comes up. Same old Microsoft. Second verse same as the first.
Not sure if it's bought or organic, but no other company gets the response Microsoft gets. The smallest triviality get praised, the biggest flaws get rationalized away. Like, people here were going wild when they added UNIX line-ending support to notepad.
Re: Edge sends full URLs of pages visited to Microsoft
#125Is there even a "pure" browser anymore? By that, I mean one which by default will do this: if I enter a URL in the address bar, it will fetch that page and its associated resources. If I click a link on a page, it will fetch its destination (this is obviously a simplified view, ignoring things like JS on the page etc.) No other network activity, no "recommendations", no other attempts to be "smart" or "helping" by do…
Re: Edge sends full URLs of pages visited to Microsoft
#126Is there even a "pure" browser anymore? By that, I mean one which by default will do this: if I enter a URL in the address bar, it will fetch that page and its associated resources. If I click a link on a page, it will fetch its destination (this is obviously a simplified view, ignoring things like JS on the page etc.) No other network activity, no "recommendations", no other attempts to be "smart" or "helping" by do…
This led me to look for a "pure" email client that would _only_ contact my server. Popular apps like Blue Mail and myMail did not work at all behind a firewall that blocked random internet access.
I am happy to report that K-9 mail (open source) was what I was looking for: it never tried to access anything other than my mail server.
I wholeheartedly recommend using NoRoot firewall (or similar) to see just how many random servers your (flashlight) apps talk to (and watch them fail miserably when you block that).
Re: Edge sends full URLs of pages visited to Microsoft
#127Earlier quoted context omitted.
Can’t it just send a hash of the domain and of the full url for checking?
I'm guessing a single hash could be problematic for detections based on the domain for example. But this could be circumvented by sending hashed parts of the URL. For example they could hash the domain, path and query separately.
The "Google Safe Browsing Update API" (used by Firefox, Chrome, Safari and others) solved this a long time ago. In that protocol, the browser hashes the URL, sends a short prefix of the hash to the server, and receives a list of hashes for the URLs that should be blocked. A huge number of valid URLs all hash to each prefix and the server does not know which one the user has visited. Also, the client caches the list of hash prefixes for which the block list is non-empty, to avoid unnecessary fetches of empty lists, which further improves privacy and reduces response time.
Also, the client doesn't send any kind of user ID token to the server.
Re: Edge sends full URLs of pages visited to Microsoft
#128Is there even a "pure" browser anymore? By that, I mean one which by default will do this: if I enter a URL in the address bar, it will fetch that page and its associated resources. If I click a link on a page, it will fetch its destination (this is obviously a simplified view, ignoring things like JS on the page etc.) No other network activity, no "recommendations", no other attempts to be "smart" or "helping" by do…
surf [0] does that. "It is able to display websites and follow links." [0]: https://surf.suckless.org/
Re: Edge sends full URLs of pages visited to Microsoft
#129Earlier quoted context omitted.
Wait, this is not happening locally, against bloom filters or something?
There is a first step which uses bloom filters, and if the filter gives a result, it is hashed and the hash is sent to the google service to double-check. That being said, a four byte / 32 bit hash is enough to almost uniquely identify a website. There number of 32 bit numbers and websites is roughly the same order of magnitude. It's a problem without a good solution because if you create many collisions then you als…
Furthermore the protocol does not declare as URL as blocked just because the 32-bit hash prefix matches. If the prefix matches, the browser downloads the list of full hashes and checks the full hash against that list locally.
It's frustrating to see people jumping to erroneous conclusions about how Safe Browsing works when the spec is publicly available and quite clear. https://developers.google.com/safe-browsing/v4/
Re: Edge sends full URLs of pages visited to Microsoft
#130Earlier quoted context omitted.
There is a first step which uses bloom filters, and if the filter gives a result, it is hashed and the hash is sent to the google service to double-check. That being said, a four byte / 32 bit hash is enough to almost uniquely identify a website. There number of 32 bit numbers and websites is roughly the same order of magnitude. It's a problem without a good solution because if you create many collisions then you als…
The safe browsing protocol computes (up to) 30 hashes of each URL, most of which include path components of the URL. So the space of URLs subject to hashing is far greater than 2^32, and the hash prefix alone tells you practically nothing. Furthermore the protocol does not declare as URL as blocked just because the 32-bit hash prefix matches. If the prefix matches, the browser downloads the list of full hashes and ch…
Computing 30 hashes of each URL means you are sending a fuzzy hash... and 2^(32*30) is fairly precise...