Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gmail is opening and caching URLs within emails without user intervention (2019)

support.google.com

Post reply on HN
Print view View original

51–60 of 271 posts

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#51

Post by dragonwriter »

justinator wrote: post #26
wcoenen wrote: post #14

HTTP GET requests should not be interpreted by the server as a request to change something. That's what POST, PUT, DELETE and PATCH are for.

I agree, but how do you initiate a POST request via an email message? Embedding a form sometimes raises its own security alert.

> I agree, but how do you initiate a POST request via an email message? Embedding a form sometimes raises its own security alert.

So, your question is how to evade security alerts for actions with potentially significant side effects?

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#52

Post by mtwittman »

_wldu wrote: post #3

I built a small Go web app to do some security testing. When a user registers for an account, I generate a 128-bit secure token and email it to the address they provided (as a URL). Token URLs look like this: /validate/email/1d00a5c2648c211befd33f5a8a7cbfab The token is cryptographically strong and disappears after access. It can't be guessed and no one but the email account holder should click it, but I am seeing th…

Consider a different approach to mitigate automated URL fetching interference (this can apply to both email ownership verifications and password resets).

Make the emailed verification/reset link (GET request) idempotent (1 and >1 request has the same effect).

Have the link just present an interface for the user to take the next step. In the next step make a POST request that actually commences your verification/reset process.

In all likelihood you'll want expiry logic (let's say it's 30 minutes) - if you store the token with a created_at timestamp on the server you can have your verification/reset process check that now If expired, provide a UI for the user to request a fresh verification/reset email.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#53

Post by jsnell »

contravariant wrote: post #28
etaioinshrdlu wrote: post #2

I always wondered when single-click unsubscribe was going to be a problem because of exactly this. I mean, how do you expect to give a URL to Google and have them just never crawl it?

Run some javascript on the page that sends a POST? Not ideal, but not sure how else to fix it, you can't send a POST with a single link as far as I can tell. Google could of course send the POST as well but then at least they're violating the HTTP standard.

Any security scanner product should also run the JS just like the real browser does. Otherwise cloaking the phishing page will be completely trivial.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#54

Post by jhanschoo »

dheera wrote: post #12
mike_d wrote: post #6

All URLs sent to any major email provider are "clicked" because they are scanning the page to see if it is phishing or otherwise malicious (desktop antivirus and other things will also prescan URLs). It also protects privacy by defeating click tracking on marketing emails. Google will also pre-load all the images in your email too. You shouldn't take any write action to your database just based on a URL being visited…

> Google will also pre-load all the images in your email too PLEASE disable automatic loading in Gmail settings. Don't let the idiots use unethical, stalkerish e-mail read receipts.

For those confused by this comment, they were referring to automatic loading of images when you open mail.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#55

Post by justinator »

dragonwriter wrote: post #51
justinator wrote: post #26

Earlier quoted context omitted.

I agree, but how do you initiate a POST request via an email message? Embedding a form sometimes raises its own security alert.

> I agree, but how do you initiate a POST request via an email message? Embedding a form sometimes raises its own security alert. So, your question is how to evade security alerts for actions with potentially significant side effects?

> So, your question is how to evade security alerts for actions with potentially significant side effects?

Well there's already a big ol' button in an email that says, "click me to register". The end user doesn't really care about the implementation. If one pops up a security alert (the POST form) and one doesn't (the simple link), how do you think everyone implements that big ol' button, 100% of the time, for 100% of everything?

I wish email didn't work this way, but as far as I understand, this is the lay of the land. If there's a better way, I'll be happy to implement it in the system(s) that I have control over.

I'm really asking for engagement within the community with help solving this sticky problem (if it wasn't clear). If link caching is this prevalent, what to do about it, for things like registering via email?

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#56

Post by Jach »

elif wrote: post #33
etaioinshrdlu wrote: post #2

I always wondered when single-click unsubscribe was going to be a problem because of exactly this. I mean, how do you expect to give a URL to Google and have them just never crawl it?

one-click unsub is implemented by ignoring scrapers. Google is especially good about using a unique UA string

Also from my anecdotal experience it's possible google is a bit smart about unsubscribe links. My tiny blog has "unsubscribe" as part of its URL, it's truly one-click unsub and doesn't do anything clever like user agent checking or JS actions. Nevertheless, I see 8 gmails that still have email notifications on, and 3 that have unsubbed at some point.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#57

Post by _wldu »

elif wrote: post #33
etaioinshrdlu wrote: post #2

I always wondered when single-click unsubscribe was going to be a problem because of exactly this. I mean, how do you expect to give a URL to Google and have them just never crawl it?

one-click unsub is implemented by ignoring scrapers. Google is especially good about using a unique UA string

The access I saw to the registration URL was from cache.google.com and it looked like a client browser

74.51.221.37 - - [19/Aug/2021:22:05:16 +0000] "GET /validate/email/1d00a5c2648c211befd33f5a8a7cbfab HTTP/1.1" 404 0 "" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36"

$ dig -x 74.51.221.37 +short

cache.google.com.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#58

Post by kyleee »

Silhouette wrote: post #32
ShakataGaNai wrote: post #15

Earlier quoted context omitted.

But to be fair, 1-click unsubscribe is a very user friendly thing to do. As a user, if I have to jump through a bunch of hoops to unsubscribe, I'm just going to mark your message as spam and move on with my life.

Exactly. And it's not just one-click unsubscribe. Using a secret link sent to an account's email address as a way to implicitly log in instead of having to remember a password is increasingly common and also an interesting idea in terms of user experience and security. If it's OK for your mail service to open one secret link, where does it stop? Is it also OK for them to spider the content they can reach from that li…

Gross never thought of that. Lots of valuable data so no doubt someone will try it (if they aren't already)

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#59

Post by anamexis »

sp332 wrote: post #42
rgbrgb wrote: post #18

Earlier quoted context omitted.

Doesn't gmail's preloading defeat the read receipts? It makes it so every tracking pixel sent to gmail gets loaded (and not by your IP), thereby making it meaningless.

This was true briefly in 2013. https://arstechnica.com/information-technology/2013/12/gmail... But they got so much pushback they effectively disabled it. https://arstechnica.com/information-technology/2013/12/dear-... (It's still cached, but not in a privacy-preserving way.)

It's still more privacy-preserving than not preloading them at all, right? Whoever is serving the images doesn't get your IP addresses, cookies, etc.

Not saying Google is virtuous here -- it only serves to enforce their advertising monopoly -- but I don't see how the image caching in itself is a bad thing.

Re: Gmail is opening and caching URLs within emails without user intervention (2019)

#60

Post by sp332 »

anamexis wrote: post #59
sp332 wrote: post #42

Earlier quoted context omitted.

This was true briefly in 2013. https://arstechnica.com/information-technology/2013/12/gmail... But they got so much pushback they effectively disabled it. https://arstechnica.com/information-technology/2013/12/dear-... (It's still cached, but not in a privacy-preserving way.)

It's still more privacy-preserving than not preloading them at all, right? Whoever is serving the images doesn't get your IP addresses, cookies, etc. Not saying Google is virtuous here -- it only serves to enforce their advertising monopoly -- but I don't see how the image caching in itself is a bad thing.

Ok, that's probably true. Still works as a read receipt though.
Post reply on HN