>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions. So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.
Serious question, because I can't imagine your use case - under what circumstances would you wan't to block all bots except one?
Applebot, the web crawler for Apple
21–30 of 94 posts
Re: Applebot, the web crawler for Apple
#22>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions. So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.
Serious question, because I can't imagine your use case - under what circumstances would you wan't to block all bots except one?
Re: Applebot, the web crawler for Apple
#23>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions. So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.
Serious question, because I can't imagine your use case - under what circumstances would you wan't to block all bots except one?
Re: Applebot, the web crawler for Apple
#24Re: Applebot, the web crawler for Apple
#25Please let this be the beginning of an Apple search engine. We really need some better alternatives to Google.
My question is: why do we need an alternative really? What is better when even more companies index everything and perhaps screw with your privacy? My intuition agrees with you though. More companies == more competition. At some point privacy might be a selling point for them. On the other hand: it is kind of crazy that when I read about "indexing the web" it starts ringing all kind of privacy bells, while apple's in…
Re: Applebot, the web crawler for Apple
#26This is interesting. A while back, I think either Cook or Jobs mentioned that Apples makes PRODUCTS and doesn't sell ADS. If that's true (and stays true) AND this is the beginning of a search engine for them, it's going to be VERY interesting to see what it looks like.
Apple already sells ads, and in fact restricts some technology to advertising partners. WebGL, full screen ads etc. They also have a patent on unskippable ads. I don't know why people don't treat statements like that with enough cynicism.
Re: Applebot, the web crawler for Apple
#27This is interesting. A while back, I think either Cook or Jobs mentioned that Apples makes PRODUCTS and doesn't sell ADS. If that's true (and stays true) AND this is the beginning of a search engine for them, it's going to be VERY interesting to see what it looks like.
"Our business model is very straightforward: We sell great products. We don’t build a profile based on your email content or web browsing habits to sell to advertisers."
This obviously is not saying Apple doesn't sell ads. It's saying their core business model is selling products. The users are not the product.
Re: Applebot, the web crawler for Apple
#28Please let this be the beginning of an Apple search engine. We really need some better alternatives to Google.
I'd rather have a decentralized alternative ala bitcoin then another mega corp search engine. EDIT: on that point. It would be very neat but probably very difficult to build a search index based on a agreed upon algorithm like bitcoin. There would be a need for some sort of voting system to update the algorithm and moderate people gaming it.
Re: Applebot, the web crawler for Apple
#29>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions. So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.
Serious question, because I can't imagine your use case - under what circumstances would you wan't to block all bots except one?
Re: Applebot, the web crawler for Apple
#30Earlier quoted context omitted.
My question is: why do we need an alternative really? What is better when even more companies index everything and perhaps screw with your privacy? My intuition agrees with you though. More companies == more competition. At some point privacy might be a selling point for them. On the other hand: it is kind of crazy that when I read about "indexing the web" it starts ringing all kind of privacy bells, while apple's in…
When one company have more than 90% market share then: Yes, we do need an alternative. (Google market in search has been reported to be as high as 94% in countries like Denmark). If a company manages to upset Google then that company ceases to exists online, regardless of the validity of Google reason to blacklist that company. Ideally no search engine would be above 20% market share (20% being a random low number I…