Live data from Hacker News

“Magic links” can end up in Bing search results, rendering them useless

medium.com

221–230 of 243 posts

Re: “Magic links” can end up in Bing search results, rendering them useless

#221

Earlier quoted context omitted.

Many people (most?) prefer to signup to services by email address. To do so, those email addresses must be verified. How would you verify it without sending them an email link?

a confirmation code? Also, mail might not live on the same computer.

It doesn't matter if it's on the same computer. Sometimes all you need to do is click the link, not do anything on the page.

Re: “Magic links” can end up in Bing search results, rendering them useless

#222
post #83
post #59

Okay, I thought M$ was just a little bit better than $G. It turned to be as bad...

Why would you think that? If M$ had the same position as google even more things would be closed source and more connected with law enforcement and less private.

I was naive to think so as M$ did not have as many obvious malicious moves as $G recently. But I forgot all those companies are there for money and for sure they will do whatever they can.

Re: “Magic links” can end up in Bing search results, rendering them useless

#223

Earlier quoted context omitted.

> It tells bots what they should do while directly crawling your site. But if a search engine gets to a URL some other way—for example if it follows a link from somewhere outside your site—it will still index that page. I must confess I've been sceptial of robots.txt for a very long time (if I want to stop bots I serve them HTTP 403 Forbidden using .htaccess or similar). Be that as it may, it appears I'm also confuse…

Your step 1 is enough for URLs to be indexed. Even a well-behaved search engine does not need to visit your site to index a URL, including whatever anchor text pointed at it. If the crawler does then visit your site, it will see your robots.txt and (if well-behaved) obey it and not crawl the contents of the page at that URL. But this does not mean it will remove the URL itself from its index. Again: robots.txt is int…

> Again: robots.txt is intended to control crawler behavior, not search index visibility.

I'm obviously not asking the question clearly, I'm wanting to stop bots from crawling (it's scraping that annoys me), not search engines from listing URIs.

If I want to completely stop a bot from crawling my site (in the sense of "retrieving my content"), won't robots.txt prevent that? Even in the case of the bot having obtained a valid list of my URIs but not the pages contents from a 3rd party source?

Lets say I email you a list of URIs on my site. My robots.txt forbids all crawlers. Are you allowed to give the list of URIs to your bot and retrieve the content?

Re: “Magic links” can end up in Bing search results, rendering them useless

#224
While I think a Turing check can easily solve the problem without much friction, this only increases my hatred for Outlook scanning. The worse part - to turn it off, you also have to turn off junk mail protection (well, used to, it's been a while since I tried).

Now, having my private links indexed by Bing is a bit too much!? I sincerely hope OP is mistaken and Bingbot is actually the outlook scanner.

Re: “Magic links” can end up in Bing search results, rendering them useless

#225

While I think a Turing check can easily solve the problem without much friction, this only increases my hatred for Outlook scanning. The worse part - to turn it off, you also have to turn off junk mail protection (well, used to, it's been a while since I tried). Now, having my private links indexed by Bing is a bit too much!? I sincerely hope OP is mistaken and Bingbot is actually the outlook scanner.

Unfortunately not - the links were indexed and shown in Bing search results

Re: “Magic links” can end up in Bing search results, rendering them useless

#226

Earlier quoted context omitted.

I think they exist to simplify the flow for the subset of users who end up using the Reset Password link each time their session expires. And I think that subset is much larger than some would expect.

This. You'd be amazed how many users just do a password reset each time to login instead of remembering their login info.

My father has insisted on doing this for over 20 years, but he doesn't know how to do it himself. I expect a password-reset phone call from him every 2 or 3 days and have done since 1998. Just recently he had someone from his bank's IT department call him directly about resetting his password over 500 times.

Re: “Magic links” can end up in Bing search results, rendering them useless

#227

Earlier quoted context omitted.

Your step 1 is enough for URLs to be indexed. Even a well-behaved search engine does not need to visit your site to index a URL, including whatever anchor text pointed at it. If the crawler does then visit your site, it will see your robots.txt and (if well-behaved) obey it and not crawl the contents of the page at that URL. But this does not mean it will remove the URL itself from its index. Again: robots.txt is int…

> Again: robots.txt is intended to control crawler behavior, not search index visibility. I'm obviously not asking the question clearly, I'm wanting to stop bots from crawling (it's scraping that annoys me), not search engines from listing URIs. If I want to completely stop a bot from crawling my site (in the sense of "retrieving my content"), won't robots.txt prevent that? Even in the case of the bot having obtained…

You are correct: a bot that is well-behaved (follows robots.txt directions) will not crawl your site if your robots.txt forbids crawling.

Re: “Magic links” can end up in Bing search results, rendering them useless

#228

Earlier quoted context omitted.

We got some security courses about that too. Unfortunately, outlook replaces all of them with some safelink url rewriting, so the only way left to find out if a link is scammy is clicking it.

It is in fact possible to extract a destination URL from a Safelink one without clicking it. For the full link this can be tedious, but identifying the domain can still be done quickly.

For normal URLs, I agree. But in this case you have adversarial urls. Suppose the scammer puts some http and www.google.com in the url parameters, after some randomly generated 8 characters dot someobscuretld site.

I don't trust myself enough to be 100% sure I can decode an URLencoded misleading mess perfectly all the time.

They already hid urls in the username of the url, like www.google.com.unholymessherethatscrollsoutoftheurlbar @ malignantdomainnotgoogle.blah

Re: “Magic links” can end up in Bing search results, rendering them useless

#229
post #214

Earlier quoted context omitted.

Is it ok to do a password reset through email? Because once you can do that you basically have email based authentication. The password only makes this autentication less secure and it's not needed.

It is not. You can initiate password reser via email but additional recovery controls like security questions should still be required. In an ideal world you have 2fa as well, if you reset that via email as well the it isn't actually 2fa, it is email based 1fa with extra steps. If your 2fa has a separate mechanism for recovery as well,that would be ideal. If it was my webapp, I would use hashes of 3 answers to user c…

Hashing an answer to many questions isn't prevent someone from guessing the input until the hashes match. So why bother hashing?

Recovery codes exist and are created at the time before recovery is necessary. But most people are going to lose their codes.

Who is going to remember what 3 things they picked out of 30 years ago?

Re: “Magic links” can end up in Bing search results, rendering them useless

#230

Earlier quoted context omitted.

This. You'd be amazed how many users just do a password reset each time to login instead of remembering their login info.

My father has insisted on doing this for over 20 years, but he doesn't know how to do it himself. I expect a password-reset phone call from him every 2 or 3 days and have done since 1998. Just recently he had someone from his bank's IT department call him directly about resetting his password over 500 times.

I'm not sure if he's still doing it but someone put together https://theuserisdrunk.com/ and https://theuserismymom.com/ a few years back... I wonder if you could do something similar here, given the level of absolute predictability that seems to be involved.

I sadly can't put my finger on what's so compelling about this, just that my "oh that person should talk to a UX team lead!" meter just went plink

Post reply on HN