Live data from Hacker News

“Magic links” can end up in Bing search results, rendering them useless

medium.com

191–200 of 243 posts

Re: “Magic links” can end up in Bing search results, rendering them useless

#193

Earlier quoted context omitted.

I believe the recommended practice is to hover over the URL before clicking the link. If you do so, in Outlook, there will be a pop that shows "Original URL: XXX". This allows users to make a determination for themselves whether the link is safe or not.

We got some security courses about that too. Unfortunately, outlook replaces all of them with some safelink url rewriting, so the only way left to find out if a link is scammy is clicking it.

Scammy Microsoft.

Re: “Magic links” can end up in Bing search results, rendering them useless

#194
post #149

Earlier quoted context omitted.

It sounds like you’re advocating for no client at all

Just as a trivial example, how confident would you be in this auth scheme? 1. User opens Outlook and types in their email and password. 2. The app requests the user's password hash from the server and checks it. 3. Outlook tells the server auth was successful and gets a session token.

In this example you're right. For something like scanning a site for malicious content, on-device is not a bad approach. It decreases the amount of data sent to the server.

The client has a much bigger issue to worry about if the client-side malware scanning has been compromised. Malware could modify the UI/network calls such that your server-side scanning displays a positive result anyway.

You have to trust the client to display information to the user at some point. Link malware scanning that job can safely be delegated to the client. Authentication cannot.

Re: “Magic links” can end up in Bing search results, rendering them useless

#195
post #176

Earlier quoted context omitted.

HTTP GET requests are supposed to be idempotent, meaning that when you call an URL twice it should not lead to any different result compared to calling it once. This is part of the HTTP standard. So while I think what Outlook does here is wrong, what these webpages do is simply a bug that should be fixed and shows a lack of understanding of HTTP.

It's idempotent. Your account won't get un-verified if you open the verification link twice. Idempotent != side-effect free

It's not though. Loading the page twice creates two different outcomes. Idempotent endpoints can't.

!Idempotent != Reversal of changes

Re: “Magic links” can end up in Bing search results, rendering them useless

#196

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

Do they guarantee anywhere they’re not collecting this data to build profiles or do other analysis?

Re: “Magic links” can end up in Bing search results, rendering them useless

#197

Earlier quoted context omitted.

It's idempotent. Your account won't get un-verified if you open the verification link twice. Idempotent != side-effect free

It's not though. Loading the page twice creates two different outcomes. Idempotent endpoints can't. !Idempotent != Reversal of changes

What are the two different outcomes?

1. You open the link once = your account is verified

2. You open the link twice = your account is still verified

???

Re: “Magic links” can end up in Bing search results, rendering them useless

#198
post #176

I've had to deal with this with e-mail verification links and Auth0. The user clicked the link after getting it in their mailbox but then Auth0 throws up an error page because the e-mail address has already been verified (by Outlook scanning). The problem becomes worse if for some reason the mail ends up in the junk mail folder so the user thinks they've never received the mail but when you check it looks like the e-…

HTTP GET requests are supposed to be idempotent, meaning that when you call an URL twice it should not lead to any different result compared to calling it once. This is part of the HTTP standard. So while I think what Outlook does here is wrong, what these webpages do is simply a bug that should be fixed and shows a lack of understanding of HTTP.

I wouldn't go around saying others don't understand http. Puts you in a very awkward position when you are wrong. Which, you are. Idempotency and safety are separate concepts related to http. But, you do you. Take the advice however you want.

Re: “Magic links” can end up in Bing search results, rendering them useless

#199
post #173
post #134

Earlier quoted context omitted.

Holy shit thats bad. Do unlisted youtube and gdrive share links get indexed through this?

You’d assume those have proper robots.txt configuration?

I have a disallow all robots.txt for a production system. Have had from the beginning.

Bing indexes it. This is my first major security incident and I have no idea how to fix this without making everything totally shitty for the users.

Re: “Magic links” can end up in Bing search results, rendering them useless

#200
post #174
post #164

Earlier quoted context omitted.

Worse would be links that are private to the people who posses the url. Like a private video on YouTube or a private document in google docs. The security depends on the URL being secret. This would silently publish secret information.

If those pages have no proper meta tags or robots.txt, there’s absolutely nothing wrong with this. Security by obscurity was never a good approach; from Proxies to security scanners, there has always been software that crawls unassuming URLs and published the results somewhere, if only a report to the admin.

robots.txt disallow is ignored for my production site at least. This is super bad.
Post reply on HN