Bing search results showing up in Google
jacquesmattheij.com
Bing search results showing up in Google
1–10 of 131 posts
Re: Bing search results showing up in Google
#2robots.txt applies to the source of the links, not the target.
So if, say, http://www.paulgraham.com/ links to http://m.bing.com/search, then http://m.bing.com/robots.txt does not apply to that.
EDIT: If you think this is wrong, please explain it instead of just downvoting me, because I think it is pretty unfair that I lose karma for explaining my interpretation.
Re: Bing search results showing up in Google
#3I agree with the author here. I think that Google will come away from this looking combative and childish.
Re: Bing search results showing up in Google
#4robots.txt applies to the source of the links, not the target. So if, say, http://www.paulgraham.com/ links to http://m.bing.com/search , then http://m.bing.com/robots.txt does not apply to that. EDIT: If you think this is wrong, please explain it instead of just downvoting me, because I think it is pretty unfair that I lose karma for explaining my interpretation.
Why would someone with no authority over your site linking to your site override your robots.txt?
Edit: I didn't downvote you, but I think you're wrong because it makes no sense - if I link to your site that shouldn't give search engines a free pass to ignore your wishes and do whatever they want with your content.
Re: Bing search results showing up in Google
#5Bit different.
Re: Bing search results showing up in Google
#6robots.txt applies to the source of the links, not the target. So if, say, http://www.paulgraham.com/ links to http://m.bing.com/search , then http://m.bing.com/robots.txt does not apply to that. EDIT: If you think this is wrong, please explain it instead of just downvoting me, because I think it is pretty unfair that I lose karma for explaining my interpretation.
That's not true.
Re: Bing search results showing up in Google
#7This looks like microsoft is assuming robots.txt is case insensitive?
Re: Bing search results showing up in Google
#8Dear Microsoft, case-sensitivity is important. [1]
m.bing.com/robots.txt says "/search", not "/Search". All of the crawled [2] urls are "/Search" or "/~/search".
Also,
wap.bing.com/robots.txt explicitly "Allow:"s several search pages, which are indexed by google.
[1] EDIT: Case insensitivity is often important. Above comment notes that some robots are case-insensitive. I suspect Google is not, based on the results.
[2] EDIT: I said indexed, a reply corrects to crawled. Good point, thanks.
Re: Bing search results showing up in Google
#9robots.txt applies to the source of the links, not the target. So if, say, http://www.paulgraham.com/ links to http://m.bing.com/search , then http://m.bing.com/robots.txt does not apply to that. EDIT: If you think this is wrong, please explain it instead of just downvoting me, because I think it is pretty unfair that I lose karma for explaining my interpretation.
Why would someone with no authority over your site linking to your site override your robots.txt? Edit: I didn't downvote you, but I think you're wrong because it makes no sense - if I link to your site that shouldn't give search engines a free pass to ignore your wishes and do whatever they want with your content.
I understand a robots.txt "Disallow: /foo" to mean that it must not crawl that page, i.e. look at the links _inside_ that page.
Re: Bing search results showing up in Google
#10[deleted]