Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Magic links” can end up in Bing search results, rendering them useless

medium.com

Post reply on HN
Print view View original

101–110 of 243 posts

Re: “Magic links” can end up in Bing search results, rendering them useless

#101

Post by heipei »

sandermvanvliet wrote: post #97

I've had to deal with this with e-mail verification links and Auth0. The user clicked the link after getting it in their mailbox but then Auth0 throws up an error page because the e-mail address has already been verified (by Outlook scanning). The problem becomes worse if for some reason the mail ends up in the junk mail folder so the user thinks they've never received the mail but when you check it looks like the e-…

Honestly that's why I built the email verification page so the user still has to click a button on that page.

Re: “Magic links” can end up in Bing search results, rendering them useless

#102

Post by dmw_ng »

sandermvanvliet wrote: post #97

I've had to deal with this with e-mail verification links and Auth0. The user clicked the link after getting it in their mailbox but then Auth0 throws up an error page because the e-mail address has already been verified (by Outlook scanning). The problem becomes worse if for some reason the mail ends up in the junk mail folder so the user thinks they've never received the mail but when you check it looks like the e-…

Links like this are stupid regardless of Outlook's behaviour because they require a perfectly reliable client and network and user in a perfectly undisturbed flow. If I can't F5, if I double-click, if my mouse is wonky, my wifi is bad, my power goes out, my computer hangs, my DSL dies just after a click, if I accidentally close the tab.. there are any of a thousand reasons why abusing GET for a one-time-use page or redirect is horribly wrong.

It takes incredible arrogance to continue using them in order to "improve usability" given all the obvious and common cases where they completely destroy usability. The difficulty for a provider to verify they aren't sending you to a phishing or browser 0day page barely scratches the surface.

Re: “Magic links” can end up in Bing search results, rendering them useless

#103

Post by heipei »

To all the folks that suggest preventing opening of single-use links by robots.txt or user-agent detection etc: Just don't. There are dozens of tools at use throughout the various stages of an email with URLs being delivered that will go out and fetch websites. You have to design any confirmation dialog so the user still has to click a button to confirm, otherwise any one of these tools might inadvertently trigger your confirmation.

Re: “Magic links” can end up in Bing search results, rendering them useless

#104

Post by _8j50 »

bigtones wrote: post #14

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

It is common for corporate email security appliances as well. URLs should not be used for authentication neither should email. I really want to pick brains of people that work on these types of systems to see why they don't think so.

Re: “Magic links” can end up in Bing search results, rendering them useless

#105

Post by rsbadger »

noisem4ker wrote: post #100
usrn wrote: post #70

Earlier quoted context omitted.

I just love it when they "scan" password reset links.

The HTTP GET method is idempotent by specification. Visiting a webpage should not trigger password resets or any other actions by itself. If that's a problem then it's the site's fault for being defective.

You’re right that it was a bit of an oversight on my behalf, as the links were only generated after a verified human user action (signup) I had assumed the 1 time links to their email would be safe. But regardless of the link action, it shouldn’t be passing that data to Bingbot to crawl and (possibly) index in search engine results. Private email data should not be shared with search engine crawlers IMO.

Re: “Magic links” can end up in Bing search results, rendering them useless

#106

Post by dTal »

sandermvanvliet wrote: post #97

I've had to deal with this with e-mail verification links and Auth0. The user clicked the link after getting it in their mailbox but then Auth0 throws up an error page because the e-mail address has already been verified (by Outlook scanning). The problem becomes worse if for some reason the mail ends up in the junk mail folder so the user thinks they've never received the mail but when you check it looks like the e-…

>We ended up adding a custom page to handle e-mail validation so we could handle the situation where the user lands on the page and the address has already been verified.

That's a yikes from me! So I can sign up on your service as anyone with an Outlook account, without verification?

Re: “Magic links” can end up in Bing search results, rendering them useless

#107

Post by rsbadger »

_8j50 wrote: post #104
bigtones wrote: post #14

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

It is common for corporate email security appliances as well. URLs should not be used for authentication neither should email. I really want to pick brains of people that work on these types of systems to see why they don't think so.

Many people (most?) prefer to signup to services by email address. To do so, those email addresses must be verified. How would you verify it without sending them an email link?

Re: “Magic links” can end up in Bing search results, rendering them useless

#108

Post by rsbadger »

dmw_ng wrote: post #102
sandermvanvliet wrote: post #97

I've had to deal with this with e-mail verification links and Auth0. The user clicked the link after getting it in their mailbox but then Auth0 throws up an error page because the e-mail address has already been verified (by Outlook scanning). The problem becomes worse if for some reason the mail ends up in the junk mail folder so the user thinks they've never received the mail but when you check it looks like the e-…

Links like this are stupid regardless of Outlook's behaviour because they require a perfectly reliable client and network and user in a perfectly undisturbed flow. If I can't F5, if I double-click, if my mouse is wonky, my wifi is bad, my power goes out, my computer hangs, my DSL dies just after a click, if I accidentally close the tab.. there are any of a thousand reasons why abusing GET for a one-time-use page or r…

The only purpose of this link was to verify that the email address is valid. Once it’s verified, you can login.

Re: “Magic links” can end up in Bing search results, rendering them useless

#109

Post by TremendousJudge »

rsbadger wrote: post #108
dmw_ng wrote: post #102

Earlier quoted context omitted.

Links like this are stupid regardless of Outlook's behaviour because they require a perfectly reliable client and network and user in a perfectly undisturbed flow. If I can't F5, if I double-click, if my mouse is wonky, my wifi is bad, my power goes out, my computer hangs, my DSL dies just after a click, if I accidentally close the tab.. there are any of a thousand reasons why abusing GET for a one-time-use page or r…

The only purpose of this link was to verify that the email address is valid. Once it’s verified, you can login.

I have seen services where you have to click a link every time you want to log in

Re: “Magic links” can end up in Bing search results, rendering them useless

#110

Post by mikro2nd »

Just wondering how this might be an attack vector for fucking with Bing... I can think of a couple of avenues; the most elegant would be if the URL itself triggered something within the scanner/URL processor; next up would be the content of the target page attacking the Bing infrastructure. I'd guess the backend processing is sandboxed, but it seems like an interesting avenue that a malicious actor might explore.

Don't try this at home, kids. :)

Post reply on HN