Live data from Hacker News

Applebot, the web crawler for Apple

support.apple.com

11–20 of 94 posts

Re: Applebot, the web crawler for Apple

#11
>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions.

So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.

Re: Applebot, the web crawler for Apple

#12

>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions. So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.

Serious question, because I can't imagine your use case - under what circumstances would you wan't to block all bots except one?

Re: Applebot, the web crawler for Apple

#13
post #9
post #3

Please let this be the beginning of an Apple search engine. We really need some better alternatives to Google.

My question is: why do we need an alternative really? What is better when even more companies index everything and perhaps screw with your privacy? My intuition agrees with you though. More companies == more competition. At some point privacy might be a selling point for them. On the other hand: it is kind of crazy that when I read about "indexing the web" it starts ringing all kind of privacy bells, while apple's in…

When one company have more than 90% market share then: Yes, we do need an alternative. (Google market in search has been reported to be as high as 94% in countries like Denmark).

If a company manages to upset Google then that company ceases to exists online, regardless of the validity of Google reason to blacklist that company. Ideally no search engine would be above 20% market share (20% being a random low number I just made up).

Right now websites and marketing material/money is directed at Google exclusively, making it continually harder for new search companies to succeed.

Re: Applebot, the web crawler for Apple

#14

This is interesting. A while back, I think either Cook or Jobs mentioned that Apples makes PRODUCTS and doesn't sell ADS. If that's true (and stays true) AND this is the beginning of a search engine for them, it's going to be VERY interesting to see what it looks like.

Apple already sells ads, and in fact restricts some technology to advertising partners. WebGL, full screen ads etc. They also have a patent on unskippable ads.

I don't know why people don't treat statements like that with enough cynicism.

Re: Applebot, the web crawler for Apple

#15

This is interesting. A while back, I think either Cook or Jobs mentioned that Apples makes PRODUCTS and doesn't sell ADS. If that's true (and stays true) AND this is the beginning of a search engine for them, it's going to be VERY interesting to see what it looks like.

But Apple does sell ads...

And while I would love a privacy focused search engine, I just don't think you can build a good one without private data.

Re: Applebot, the web crawler for Apple

#16
post #3

Please let this be the beginning of an Apple search engine. We really need some better alternatives to Google.

I'd rather have a decentralized alternative ala bitcoin then another mega corp search engine.

EDIT: on that point. It would be very neat but probably very difficult to build a search index based on a agreed upon algorithm like bitcoin. There would be a need for some sort of voting system to update the algorithm and moderate people gaming it.

Re: Applebot, the web crawler for Apple

#18

This is interesting. A while back, I think either Cook or Jobs mentioned that Apples makes PRODUCTS and doesn't sell ADS. If that's true (and stays true) AND this is the beginning of a search engine for them, it's going to be VERY interesting to see what it looks like.

iAd Advertising with Apple http://advertising.apple.com/

Re: Applebot, the web crawler for Apple

#19
post #9
post #3

Please let this be the beginning of an Apple search engine. We really need some better alternatives to Google.

My question is: why do we need an alternative really? What is better when even more companies index everything and perhaps screw with your privacy? My intuition agrees with you though. More companies == more competition. At some point privacy might be a selling point for them. On the other hand: it is kind of crazy that when I read about "indexing the web" it starts ringing all kind of privacy bells, while apple's in…

It's nice to have alternatives for something that's so fundamental to how we use the internet.

Google works well today, but what about 5 years from now, or 10 years from now? What if you need to be signed into Google+ to Google search? What if your Google+ account needs to be tied to a mobile phone? What if Google has limited search, and you need to own a Chromebook or Android device for the full search experience and viewing more than 10 results?

Competition keeps them in check from doing anything outrageous, and then if they decide to anyway, we have other choices we can migrate towards.

Re: Applebot, the web crawler for Apple

#20
post #12

>If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions. So if I set in my robots.txt to disallow all bots except Googlebot, Applebot will index anyway? I don't think I like that precedent.

Serious question, because I can't imagine your use case - under what circumstances would you wan't to block all bots except one?

Maybe for ethical reasons or some kind of exclusive agreement with one particular search vendor?
Post reply on HN