Live data from Hacker News

“Magic links” can end up in Bing search results, rendering them useless

medium.com

91–100 of 243 posts

Re: “Magic links” can end up in Bing search results, rendering them useless

#91
post #79

Earlier quoted context omitted.

They are publishing them - it has bitten us (e.g. expired one click links for customers ending up on Bing from their emails)

If you can say for certain that the links being published are coming from the malware scanning, and not being taken from users' browser sessions that are using Microsoft Edge you should elaborate on this.

That would be even worse.

Re: “Magic links” can end up in Bing search results, rendering them useless

#92
post #58

Earlier quoted context omitted.

Yes, but there is no indication they are publishing it in the search results. The original post is just complaining that the malware scanning is visiting the links. They come to the following conclusion > This effectively makes all one-time use links like login/pass-reset/etc useless. Which we all know is not true because sites like onetimesecret.com allow for entering a separate password to prevent this sort of thin…

They are publishing them - it has bitten us (e.g. expired one click links for customers ending up on Bing from their emails)

I think this is where we use meta tags.

All pages with one click links should have no index follow or no index no follow. Your seo consultant (if you have one) should have advised you on this.

I am not saying this excuses the privacy violation but just suggesting there are things we can do...

Re: “Magic links” can end up in Bing search results, rendering them useless

#93
post #66
post #43

Earlier quoted context omitted.

The HTTP GET method is idempotent: it should behave the same way on multiple accesses. A single use link, e.g. for resetting a password or confirming a subscription, will usually show a webpage with a form that does a POST. Once that POST has been performed, the single use link is used up. Single use links will mostly have a one-time secret that should not be leaked. Mails that contain such links or any sensitive inf…

How do you send mail to an Outlook user and encrypt it so Microsoft can't snoop on it?

[deleted]

Re: “Magic links” can end up in Bing search results, rendering them useless

#94
post #79

Earlier quoted context omitted.

If you can say for certain that the links being published are coming from the malware scanning, and not being taken from users' browser sessions that are using Microsoft Edge you should elaborate on this.

That would be even worse.

Nobody is saying it isn't.

It's about trying to get to the core of the issue, not just the random speculation going on in the article and in this comment thread.

Re: “Magic links” can end up in Bing search results, rendering them useless

#95
post #76
post #64

Earlier quoted context omitted.

I have observed that twitter's bot hits links within seconds of being tweeted. The traffic comes from several locations, not all twitter ASNs. One interesting source is Apple. Their bot/scanner hits soon after.

Anyone paying for the firehose access can do this.

That's true of course. What's interesting to me is that they've decided to pay for this access and visit the links so quickly. It must be pretty expensive or hard to get if only around two-dozen companies pay for access to the data[0].

[0] https://www.washingtonpost.com/technology/2022/06/08/elon-mu...

Re: “Magic links” can end up in Bing search results, rendering them useless

#96
post #79

Earlier quoted context omitted.

They are publishing them - it has bitten us (e.g. expired one click links for customers ending up on Bing from their emails)

If you can say for certain that the links being published are coming from the malware scanning, and not being taken from users' browser sessions that are using Microsoft Edge you should elaborate on this.

I would be pretty mortified if browsers were using user browser sessions to scan content and pass it to bingbot…? What about if you’re browsing something local? Or your bank account?

Re: “Magic links” can end up in Bing search results, rendering them useless

#97
I've had to deal with this with e-mail verification links and Auth0. The user clicked the link after getting it in their mailbox but then Auth0 throws up an error page because the e-mail address has already been verified (by Outlook scanning). The problem becomes worse if for some reason the mail ends up in the junk mail folder so the user thinks they've never received the mail but when you check it looks like the e-mail address was verified successfully. That has caused a lot of annoying back and forth trying to figure out what the hell is going on. We ended up adding a custom page to handle e-mail validation so we could handle the situation where the user lands on the page and the address has already been verified. Super annoying.

Re: “Magic links” can end up in Bing search results, rendering them useless

#98

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

scanning with bing useragent? That's not a good idea.

Re: “Magic links” can end up in Bing search results, rendering them useless

#99

Even if Microsoft claim this is about security scanning, isn't it fairly trivial to configure your webserver to serve up different content depending on the User-Agent request header? BingBot scans the link, gets a dummy page with 'clean' content, Microsoft delivers the email message to the user, user clicks through the link with actual browser, gets phishing / malware content...

Sure, or even just ignore user agents if you know your target has this scanning in place, just send the malware to the 2nd click.

It's not just MS. Lots of enterprise email security stuff works like this.

Re: “Magic links” can end up in Bing search results, rendering them useless

#100
post #70

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

I just love it when they "scan" password reset links.

The HTTP GET method is idempotent by specification. Visiting a webpage should not trigger password resets or any other actions by itself. If that's a problem then it's the site's fault for being defective.
Post reply on HN