Live data from Hacker News

“Magic links” can end up in Bing search results, rendering them useless

medium.com

131–140 of 243 posts

Re: “Magic links” can end up in Bing search results, rendering them useless

#131
post #125

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

> Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks The problem with that is that the logic is broken. Microsoft cannot possibly know all phishing sites, especially for smaller things. By obfuscating the link the user can no longer verify it by themselves without clicking, but Microsoft will say it's safe. So the user is left with…

Maybe you should implement a "feature" that serves a simple static HTML page

This webpage is safe.

to "bingbot" and serve the real page to everyone else.

Re: “Magic links” can end up in Bing search results, rendering them useless

#132

Just wondering how this might be an attack vector for fucking with Bing... I can think of a couple of avenues; the most elegant would be if the URL itself triggered something within the scanner/URL processor; next up would be the content of the target page attacking the Bing infrastructure. I'd guess the backend processing is sandboxed, but it seems like an interesting avenue that a malicious actor might explore. Don…

Sure, we'll do this right after cracking Google through Googlebot.

Re: “Magic links” can end up in Bing search results, rendering them useless

#133
post #116

Earlier quoted context omitted.

I have seen services where you have to click a link every time you want to log in

They are called magic links... only thing magic about them is their ability to annoy me

I think they exist to simplify the flow for the subset of users who end up using the Reset Password link each time their session expires.

And I think that subset is much larger than some would expect.

Re: “Magic links” can end up in Bing search results, rendering them useless

#134

Earlier quoted context omitted.

But there is nothing to indicate either in the post or in the referenced SO thread that the URLs are published to the search results. They are visited by bingbot, that much seems confirmed, but there’s no example where one of these results shows up in the public search results.

They were indexed in Bing results, I’ve shared the URL of that in this thread

Holy shit thats bad. Do unlisted youtube and gdrive share links get indexed through this?

Re: “Magic links” can end up in Bing search results, rendering them useless

#135
post #80

Microsoft does this because they're security scanning / checking all links in every Outlook email for known phishing and malware attacks. If Bing has not seen the web page before and it's not in the Bing dangerous web page index it first needs to check it to make a determination of if it's a phishing/malware page by scanning/indexing it before returning that outcome back to Outlook to flag the email as dangerous.

Wouldn't it be trivial to keep the list of malicious pages locally and not send any data?

You mean, push bing's entire list of malicious websites to every client? I doubt they want to or can do that.

And also use the local client to scan unknown links? They probably dont want outsiders to have access to this code.

Re: “Magic links” can end up in Bing search results, rendering them useless

#136
post #126
post #122

Earlier quoted context omitted.

What? But you're the one writing the client.

Doesn't matter. Never trust the client - it's outside of your control, it can be patched, it can be hacked, it can be spoofed, etc.

Little understanding: Undying trust of the client

Dunning-Kruger level of understanding: Never trust the client for anything ever, it's unreliable, everything must be off client.

Never mind the client is literally the interface into your system, so it being compromised is already game over for an application where the user is most vulnerable party you wanted to protect...

Deep understanding: Trusting the client requires a well thought out security model.

If the client is hacked in this case, they already have full control over what the user sees, they can cut out your remote check.

Maybe a good balance would be to hash the root of the URLs and compare those, or use fuzzy hashing on page contents, just so that the backend isn't getting a bunch of private urls that might accidentally get logged somewhere.

Trades detecting stuff hidden behind redirects for less liability on your backend, something to possibly consider depending on functional requirements.

Re: “Magic links” can end up in Bing search results, rendering them useless

#137
post #39

Earlier quoted context omitted.

Office 365 just seems to make links useless for security now. Our 365 instance now turns every link into this massive monolith of safelink checking URLs through Microsoft, making literally every email undeterminable if it is a phishing attempt or otherwise without turning to pasting it into one of many online 'decoders'...

At work they enabled safelinks whilst all the mandatory training stated best practice was to check the links before clicking. Its a shame those links can't have an alttext to show the real link.

You don't want to train users to disambiguate phishing with alt texts which can be spoofed in other contexts.

Re: “Magic links” can end up in Bing search results, rendering them useless

#138
post #104

Earlier quoted context omitted.

It is common for corporate email security appliances as well. URLs should not be used for authentication neither should email. I really want to pick brains of people that work on these types of systems to see why they don't think so.

Many people (most?) prefer to signup to services by email address. To do so, those email addresses must be verified. How would you verify it without sending them an email link?

You can verify validity of an email like that, no issue there. Just don't use that as a factor authentication. Control over an email account should not trump passwords (what you know) or proper 2fa (what you have, typically, email can be 2fa like sms and like sms it is not a good choice). If a person proves they control an email account then you ask them for additional info like secret questions or other information configured during registration.

I should not be able to take over your life because I compromised your phone which has sms, TOTP app and email.

Re: “Magic links” can end up in Bing search results, rendering them useless

#139
post #126
post #122

Earlier quoted context omitted.

What? But you're the one writing the client.

Doesn't matter. Never trust the client - it's outside of your control, it can be patched, it can be hacked, it can be spoofed, etc.

It sounds like you’re advocating for no client at all

Re: “Magic links” can end up in Bing search results, rendering them useless

#140
post #66
post #43

Earlier quoted context omitted.

The HTTP GET method is idempotent: it should behave the same way on multiple accesses. A single use link, e.g. for resetting a password or confirming a subscription, will usually show a webpage with a form that does a POST. Once that POST has been performed, the single use link is used up. Single use links will mostly have a one-time secret that should not be leaked. Mails that contain such links or any sensitive inf…

How do you send mail to an Outlook user and encrypt it so Microsoft can't snoop on it?

You re-ask the password on the visited page before presenting the form responsible for the one time POST call.
Post reply on HN