'Search Neutrality' is neccessary to promote growth and innovation

New search engines pop up often. Off the top of my head: Cuil and Powerset. How often do new telecommunications companies pop up? The key word was relatively. Search engines aren't lemonade stands, but they aren't telecom companies either.

I guess this is just asking for negative karma, but I too thought that the conventional wisdom was that the barrier to entry for search engines was relatively high.

No, not telecom high, but it does take a pretty significant amount of hardware to index and deliver that much data. No?

Crawling a few million pages, making some sort of searchable index, and building a simple frontend is a nice exercise for anyone, and building a new search engine as such is nothing special.

Getting something up and running that has a good enough coverage of the web, constantly has fresh enough content, is able to catch up on sudden events (ie. Michael Jackson's death) and show relevant content within a maximum of a couple of minutes, is scalable and can handle the load if it becomes popular, has a low enough latency for the end user (a single server in a basement halfway around the world does not do the trick), AND offers something of significance that makes users actually want to change their habits and switch from their current search engine... Now, that's a whole different story.

Especially the last one is tricky. Even though I agree that in theory the cost of switching search engines is pretty close to zero, these habits die hard unless you have something to offer that is obviously much better.

So while Cuil and Powerset got a little bit of IT media attention, I don't think it is fair to say that they actually grabbed that much of Google's market share? And I'm not really sure it is that easy to make a new search engine that will be able to get the market share of ie. Yahoo!, Bing or Ask. Unless you can do most of the things mentioned above AND you have that extra special feature on top that makes everyone stop and stare. (Which then would have to be something the other search engines could not easily implement too and roll out within months :)

Disclaimer: I work for Yahoo!, but these opinions are mine alone.

> it does take a pretty significant amount of hardware to index and deliver that much data. No?

That's a barrier due to the intrinsic nature of the technology in question. It's not a barrier due to legal, political, social, etc difficulties.

My comment from a dead link to a blog entry on this NYT article:

>Because of its domination of the global search market and ability to penalize competitors while placing its own services at the top of its search results, Google has a virtually unassailable competitive advantage...The preferential placement of Google Maps helped it unseat MapQuest from its position as America’s leading online mapping service virtually overnight.

Alternately, it was a better product. Remember the first time you were able to drag a map? It wasn't in Mapquest. Let's see which is better today. I'm looking for New York City. I go to maps.google.com, and see a single search box. I type in NYC, and I see, within a second, the five boroughs I was looking for. Well done you, Google.

Next, I go to mapquest. They've got multiple search boxes, so I have to break up my search into address/city/state/zip code boxes? Ugh. Is it still 1998? Ok, I put in NYC into the "City" field, and the whole screen dims, with a box saying "processing". Even after the processing box goes away, the screen is still dimmed. How ugly. I scroll down to the map, and see that nothing's changing, even after waiting two minutes. I click, and the dim screen lightens, but I haven't gone anywhere. I go back to the search boxes, and again, search NYC. Oh, it's popping up a "which did you mean" alert next to the search box where I wasn't looking. 2 results: 1. Nyce Lake, AZ 2. Nyctea Hills, AK.

If this is their example of "unfair competitive advantage", I want more of it.