Live data from Hacker News

Gmail is opening and caching URLs within emails without user intervention (2019)

support.google.com

101–110 of 271 posts

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#101
post #14

HTTP GET requests should not be interpreted by the server as a request to change something. That's what POST, PUT, DELETE and PATCH are for.

I agree, but how do you initiate a POST request via an email message? Embedding a form sometimes raises its own security alert.

The unsubscribe link should lead to a separate HTML form describing the unsubscription and a "confirm" button.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#102

Earlier quoted context omitted.

From the very beginning of the web when HTTP was defined, there has been the rule that a GET request should never take an action on its own. Actions should be based on some other method like a POST request. Google is following the standard. People who take actions based on GET requests are not. Sure, mistakes happen out of ignorance, but they should be fixed.

From the very beginning of the web when HTTP was defined, there has been the rule that a GET request should never take an action on its own. Actions should be based on some other method like a POST request. Maybe, but then several decades passed and now billions of people don't use online systems the same ways any more. It used to be that I could send a legitimate mail to a friend or family member and not worry that…

"Things change as time goes on" is not an excuse to ignore valid standards.

All big email providers know this and know that GETs should not affect their unsubscribe/confirmation links. It's part of their job.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#103
post #6

All URLs sent to any major email provider are "clicked" because they are scanning the page to see if it is phishing or otherwise malicious (desktop antivirus and other things will also prescan URLs). It also protects privacy by defeating click tracking on marketing emails. Google will also pre-load all the images in your email too. You shouldn't take any write action to your database just based on a URL being visited…

That also include links sent on all major chat, in this day and age, if you not self-hosting, or E2E all your links will be minded by companies.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#104
post #3

I built a small Go web app to do some security testing. When a user registers for an account, I generate a 128-bit secure token and email it to the address they provided (as a URL). Token URLs look like this: /validate/email/1d00a5c2648c211befd33f5a8a7cbfab The token is cryptographically strong and disappears after access. It can't be guessed and no one but the email account holder should click it, but I am seeing th…

You need to authenticate the user before the activation.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#105
post #97

Earlier quoted context omitted.

> to open one secret link "Secret link" is an oxymoronical concept. Resource identifiers are exactly that: identifiers. They're not private names, and any design that relies on keeping them secret is inherently flawed. If it's accessible on the openly resolvable web, then the content needs to be treated as if it's public. If your use calls for authentication or authorization, then actually use an authentication or au…

They are in fact private names, because they're unknown to the public. This is in fact an authentication system. Yes, the public could guess a 128-bit random value and log in - but that's no different from the ability of the public to guess your password, or your session cookie, or your SSL session state, or whatever. Every authentication mechanism is based on "There is a high-entropy value, and nobody but the author…

I understand they are known to the public? Every MTA from any random site between the sender and receiver gets the mail, including secrets. They can all decide to scan the site, write them in a log,...

Then, when you click the link, if you don't have https, anything between receiver and site also gets a copy of the link. And there are proxys, add injecting ISPs, etc.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#106

I always wondered when single-click unsubscribe was going to be a problem because of exactly this. I mean, how do you expect to give a URL to Google and have them just never crawl it?

Oh, that's the rake we stepped on.

Bloody obvious in retrospect, but it took us an embarrassingly long time to realize that we were leaking mailing list subscribers because of these one-click unsub links.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#107

Earlier quoted context omitted.

You make just visiting the URL not everything that needs to be done. So for example, the URL you visit then also runs a small bit of javascript behind the scenes that does the actual unsub action - or the javascript just does a redirect. Or even simpler, you make it so the user has to click a button to POST the request. You've had to do this for years, now. I would assume though, that Gmail is smart enough to go, "oh…

This is what MailChimp does for its one-click unsubscribes. You visit a URL, and some JS POSTs to ` https://[youraccount].us1.list-manage.com/unsubscribe/post ` with a body containing your subscription and list IDs. I'm not sure what prevents crawlers executing JavaScript on that page and triggering the unsubscribe action anyway, though, unless it's just that email crawlers don't execute JS.

it could do some checks on the environment - useragent, screen size etc?

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#108
post #100

We have had the same problem with Microsoft where our marketing department is effectively DDoSing our service by sending out links to 50k+ users. We would get spikes of thousands of requests per second from Microsoft IP addresses, which after some googling were linked to their threat detection.

Threat detection is becoming essential because of ransomware phishing and alikes, protection from click tracking is good too.

Just send email in tranches.

If the marketing department is sending too many e-mails for the web server to handle, then the volume is probably out of proportion to the company and what they are doing is probably just spam.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#109
post #97

Earlier quoted context omitted.

They are in fact private names, because they're unknown to the public. This is in fact an authentication system. Yes, the public could guess a 128-bit random value and log in - but that's no different from the ability of the public to guess your password, or your session cookie, or your SSL session state, or whatever. Every authentication mechanism is based on "There is a high-entropy value, and nobody but the author…

I understand they are known to the public? Every MTA from any random site between the sender and receiver gets the mail, including secrets. They can all decide to scan the site, write them in a log,... Then, when you click the link, if you don't have https, anything between receiver and site also gets a copy of the link. And there are proxys, add injecting ISPs, etc.

Are you claiming that the contents of emails are public?

Which "random sites" see emails between sender and receiver?

Yes, proxies and ad-injecting ISPs can see the contents of plaintext HTTP. But that's hardly a reason to say that logging into a website with a password or presenting a cookie doesn't count as an authentication system!

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#110
post #88

You can (or used to be able to) embed an HTML form inside an email, and clicking the submit button will perform a POST if that is the method specified. This would probably solve the issue. You avoid an extra click on the resulting page, it's a POST which matches the standard, and Google (presumably) won't hit it.

Forms are not supported in all major clients.
Post reply on HN