Downgrade User Agent Client Hints to 'harmful' • Hacker News Board

Re: Downgrade User Agent Client Hints to 'harmful'

#61

Post by 1vuio0pswjnm7 » Tue, Jul 13, 2021, 8:16 PM UTC

Ajedi32 wrote: post #2

> Moving stuff around (from User-Agent to Sec-CH-UA-*) doesn't really solve much. That is, having to request this information before getting it doesn't help if sites routinely request all of it. I think this is sort of ignoring the whole point of the proposal. By making sites request this information rather than simply always sending it like the User-Agent header currently does, browsers gain the ability to deny exce…

"By making sites request this information rather than simply sending it like the User-Agent header currently does..."

This is also true with respect to SNI which leaks the domain name in clear text on the wire. The popular browsers send it even when it is not required.

The forward proxy configuration I wrote distinguishes the sites (CDNs) that actually need SNI and the proxy only sends it when required. The majority of websites submitted to HN do not need it. I also require TLSv1.3 and strip out unecessary headers. It all works flawlessly with very few exceptions.

We could argue that sending so much unecessary information as popular browsers do when technically it is not necessary for the user is user hostile. It is one-sided. "Tech" companies and others interested in online advertising have been using this data to their advantage for decades.

Re: Downgrade User Agent Client Hints to 'harmful'

#62

Post by zenexer » Tue, Jul 13, 2021, 8:25 PM UTC

jefftk wrote: post #56

dathinab wrote: post #53

Earlier quoted context omitted.

IMHO #3 is fundamentally flawed as I just can't imagine browsers improving to a point where you couldn't cross reference such "fixed" entropy budges to clearly identify the user. The only IMHO reasonable technical solution is to reduce entropy as much as possible, even below any arbitrary set entropy limit. Through in the end I think the right way is a outright (law based) ban of micro targeting and collecting of any…

There are really only two ways this can go: A. Browsers successfully reduce available entropy to where users cannot reliably be tracked across sites. B. Browsers fail at this, and widely available JavaScript libraries allow cross-site tracking. If it's possible to extract enough bits, they will be extracted. The thing is, if you can't get all the way to (A) then in removing bits you're just removing useful functional…

Why not both (A) and a regulatory solution? I see no reason to avoid the regulatory route.

Re: Downgrade User Agent Client Hints to 'harmful'

#63

Post by billyhoffman » Tue, Jul 13, 2021, 8:32 PM UTC

1vuio0pswjnm7 wrote: post #61

Ajedi32 wrote: post #2

> Moving stuff around (from User-Agent to Sec-CH-UA-*) doesn't really solve much. That is, having to request this information before getting it doesn't help if sites routinely request all of it. I think this is sort of ignoring the whole point of the proposal. By making sites request this information rather than simply always sending it like the User-Agent header currently does, browsers gain the ability to deny exce…

"By making sites request this information rather than simply sending it like the User-Agent header currently does..." This is also true with respect to SNI which leaks the domain name in clear text on the wire. The popular browsers send it even when it is not required. The forward proxy configuration I wrote distinguishes the sites (CDNs) that actually need SNI and the proxy only sends it when required. The majority…

How would this work?

SNI is sent by the client in the initial part of the TLS handshake. If you don't send it, the server sends the wrong/bad cert. The client could retry the handshake using SNI to get the correct cert but:

- This adds an extra RTT, on the critical path of getting the base HTML, hurting performance.

- A MITM could send back an invalid cert, causing the browser to retry with SNI, leaking it anyway (since we aren't talking about TLS 1.3 and an encrypted SNI).

I suppose the client could maintain a list of sites that don't need SNI, like the HSTS preload list, but that seems like a ton of overhead to avoid sending unneeded SNI, especially when most DNS is unencrypted and would leak the hostname just like SNI anyways.

Re: Downgrade User Agent Client Hints to 'harmful'

#64

Post by billyhoffman » Tue, Jul 13, 2021, 8:39 PM UTC

hypertele-Xii wrote: post #48

admax88q wrote: post #31

Serving different content for the same URI based upon various metadata fields in the request goes completely against the spirit of a URI.

No it doesn't? Ever heard of Accept or Lang headers? Or cookies for that matter? Dynamic content?

Agreed, and thanks for bring up the Accept header. The author seems uninformed about HTTP's built in Content Negotiation. They write about servers using the User-Agent header, specifically talking about WebP. Accept: "image/webp" works just fine for the major CDNs regardless of the UA.

Re: Downgrade User Agent Client Hints to 'harmful'

#65

Post by billyhoffman » Tue, Jul 13, 2021, 8:40 PM UTC

herpderperator wrote: post #59

banana_giraffe wrote: post #52

Earlier quoted context omitted.

I don't think I've ever seen it called out, but I always assumed it's "Secure" in the sense it hasn't been modified by a script. But that's 100% a guess on my part.

Great, so now we have the HttpOnly flag for cookies which differs from the Secure flag for cookies, while the Secure in the Sec headers has the same meaning as HttpOnly.

And we have SameSite in Cookies, and Allow-Origin in headers!

Re: Downgrade User Agent Client Hints to 'harmful'

#66

Post by 1vuio0pswjnm7 » Tue, Jul 13, 2021, 9:01 PM UTC

billyhoffman wrote: post #63

1vuio0pswjnm7 wrote: post #61

Earlier quoted context omitted.

"By making sites request this information rather than simply sending it like the User-Agent header currently does..." This is also true with respect to SNI which leaks the domain name in clear text on the wire. The popular browsers send it even when it is not required. The forward proxy configuration I wrote distinguishes the sites (CDNs) that actually need SNI and the proxy only sends it when required. The majority…

How would this work? SNI is sent by the client in the initial part of the TLS handshake. If you don't send it, the server sends the wrong/bad cert. The client could retry the handshake using SNI to get the correct cert but: - This adds an extra RTT, on the critical path of getting the base HTML, hurting performance. - A MITM could send back an invalid cert, causing the browser to retry with SNI, leaking it anyway (si…

"I suppose the client could maintain a list of sites that don't need SNI."

That list would be much larger than the list of sites that do require SNI.

Generally, I can determine whether SNI is required by IP address, i.e., whether it belongs to a CDN that requires SNI. Popular CDNs like AWS publish lists of their public IPs. I use TLSv1.3 plus ESNI with Cloudflare but they are currently the only CDN that supports it. Experimental but works great, IME.

The proxy maintains the list not the browser. The proxy is designed for this and can easily hold lists of 10s of 1000s of domains in memory. That's more domains than I visit in one day, week, month or year.

Is it not a question of whether this is possible. "How would this work". I have already implemented it. It works. It is not difficult to set up.

Why this works for me and would unlikely work for others.

I am not a heavy user of popular browsers, I "live on the command line". Installing a custom root certificate with appropriate SANs to suppress browser warnings is a nusiance that would likely dissuade others since they are heavy users of those programs. However I generally do not use those browsers to retrieve content from the web.

Re: Downgrade User Agent Client Hints to 'harmful'

#67

Post by ajsnigrutin » Tue, Jul 13, 2021, 9:03 PM UTC

notriddle wrote: post #25

thaumasiotes wrote: post #21

Earlier quoted context omitted.

> Set-Cookie was supposed to have the browser ask the user to confirm whether they wanted to set a cookie. Not many clients doing that today. No worries, that's why we have laws to make the website do in the content what the browser no longer wants to do in the viewer. ;D

Having the browser explicitly prompt for cookies is neither necessary nor sufficient to do what strong, consistently-enforced privacy laws can do, because the browser can't tell a tracking cookie (which needs a prompt) apart from a settings cookie (which does not).

But browsers could disable third party cookies, and autodelete first party cookies on page/tab close by default.

There would be a "keep cookies for this site" button somewhere near the address bar, and at each login, the browser would also ask you if you want to save your password and/or save cookies for that domain.

99% of websites don't require persistant storage, and those who do, 99% of them are sites you're logged into and already prompt the user, asking if they want to save the password.

Re: Downgrade User Agent Client Hints to 'harmful'

#68

Post by [deleted] » Tue, Jul 13, 2021, 9:03 PM UTC

billyhoffman wrote: post #63

1vuio0pswjnm7 wrote: post #61

Earlier quoted context omitted.

"By making sites request this information rather than simply sending it like the User-Agent header currently does..." This is also true with respect to SNI which leaks the domain name in clear text on the wire. The popular browsers send it even when it is not required. The forward proxy configuration I wrote distinguishes the sites (CDNs) that actually need SNI and the proxy only sends it when required. The majority…

How would this work? SNI is sent by the client in the initial part of the TLS handshake. If you don't send it, the server sends the wrong/bad cert. The client could retry the handshake using SNI to get the correct cert but: - This adds an extra RTT, on the critical path of getting the base HTML, hurting performance. - A MITM could send back an invalid cert, causing the browser to retry with SNI, leaking it anyway (si…

[deleted]

Re: Downgrade User Agent Client Hints to 'harmful'

#69

Post by [deleted] » Tue, Jul 13, 2021, 9:11 PM UTC

billyhoffman wrote: post #63

1vuio0pswjnm7 wrote: post #61

Earlier quoted context omitted.

"By making sites request this information rather than simply sending it like the User-Agent header currently does..." This is also true with respect to SNI which leaks the domain name in clear text on the wire. The popular browsers send it even when it is not required. The forward proxy configuration I wrote distinguishes the sites (CDNs) that actually need SNI and the proxy only sends it when required. The majority…

How would this work? SNI is sent by the client in the initial part of the TLS handshake. If you don't send it, the server sends the wrong/bad cert. The client could retry the handshake using SNI to get the correct cert but: - This adds an extra RTT, on the critical path of getting the base HTML, hurting performance. - A MITM could send back an invalid cert, causing the browser to retry with SNI, leaking it anyway (si…

[deleted]

Re: Downgrade User Agent Client Hints to 'harmful'

#70

Post by notriddle » Tue, Jul 13, 2021, 9:16 PM UTC

lupire wrote: post #35

notriddle wrote: post #25

Earlier quoted context omitted.

Having the browser explicitly prompt for cookies is neither necessary nor sufficient to do what strong, consistently-enforced privacy laws can do, because the browser can't tell a tracking cookie (which needs a prompt) apart from a settings cookie (which does not).

That would be an extension or fork of Set-Cookie.

Of course a web server could report which cookies are for tracking, and which are for authentication or configuration, instead of doing it within the content.

But so what? The browser has no way to tell if it’s lying.