Live data from Hacker News

Ask HN: Is there room for another search engine?

news.ycombinator.com

21–30 of 200 posts

Re: Ask HN: Is there room for another search engine?

#22
Yes. In today's search engines, I cannot give you a blacklist and say filter out these results. If I am looking for tutorials, I cannot say no video results. If I am looking for market research, I cannot filter out news websites from the links. For personalization, I cannot give google any suggestions on what I absolutely do not want to be included etc.

Re: Ask HN: Is there room for another search engine?

#23

Google has the best search results cause it has the most people using its service. Its models learn every time you click on a result. There's no way to take that on directly. What you need is to find an angle that Google can't easily follow, as with DuckDuckGo and privacy. Are there areas where Google can't go?

Apparently Google's search results have plenty to improve:

https://news.ycombinator.com/item?id=13885631

Re: Ask HN: Is there room for another search engine?

#24
That's a good question, and something I've spent much time on. Cuil (2008-2010) tried. I knew some of those people. It cost them about $30 million to launch a full scale search engine. They had no revenue model. In retrospect, they were hoping to be acquired by somebody. It was some ex-Google people, trying to replicate older Google technology. They had a great launch, but the system wasn't very good and traffic rapidly fell off. Their technology wasn't that great. Their big selling point was that they could do the job on less hardware than Google used.

Yahoo had a search engine from 1995 to 2009. Yahoo is now a Bing reseller. There was a period around 2007 when Yahoo search was better than Google search. They pioneered integrated vertical search: special cases for weather, celebrities, and such. But Google copied that.

Blekko (2010-2015) had a scheme with "slashtags" which attracted a small following but never caught on. They were trying to crowdsource part of the problem. Eventually, Blekko was acquired by IBM's Watson unit, and ceased offering public search.

Bing, Microsoft's entry, remains active. Microsoft seems to have given up on trying to raise Bing's market share. Bing no longer has a CEO of its own; it's just a miscellaneous online service Microsoft provides. It's still #2 in search, but only has 7% market share.

There remain a few little search engines. Ask, formerly Ask Jeeves, continues to operate, but has only 0.17% market share. Ask is from IAC, in Oakland, a spinoff of Barry Diller's Home Shopping Network. Excite, formerly Excite@Home, with 0.02% market share, continues to operate. Excite, in its day, was a hot startup powered by too much venture capital.

Outside the US, there's Baidu (China) and Yandex (Russia). Neither has much traction outside their home countries.

It's possible to do a better search engine than Google from the user perspective. It's not clear how to get it to profitability. There are two things Google does badly - business legitimacy and provenance. Google doesn't background-check businesses online. (I do that with Sitetruth; it's not only possible, it could be done better with a tie-in to costly business background services such as Dun and Bradstreet.) This allows bogus and marginal businesses to reach the top of search via the usual SEO techniques. Google is also bad at provenance - figuring out that site A is using text derived from site B, and thus B should be ranked higher. This is what allows scraper sites to rank highly in Google.

Fix those two problems, and a new search engine could be better than Google. Whether anyone would notice is questionable. Profitability would be tough. The reward for success is high. Search ads are more relevant and more profitable than any other form of advertising. When someone sees a search ad, they're actively looking for the item of interest and may be ready to buy. Almost all other ads are interruptions or annoyances. That's the basic reason for Google's success.

Re: Ask HN: Is there room for another search engine?

#25
For a general search engine, no, there isn't.

The upfront capital investment, in terms of the data center capacity necessary to make a modern scraping and search infrastructure, is immense. And since the ad-word business model does not scale linearly with market share – e.g. the market leader collects a disproportionate share of the available profit – you will be losing additional money for a long time.

Since the market leader is good enough that it isn't possible to disrupt the market purely through result quality (as Google did), you will need to rely on bigger and more effective marketing spend. Not only will you have to outspend and outperform Google, but also Microsoft/Bing, who have tried to do the same thing for years, with only limited success.

Even if you have the funding necessary to do all of this, then you would be better off either buying shares in an existing search engine company, or starting a business in a different market, one with lower upfront costs and less dominant incumbents.

Re: Ask HN: Is there room for another search engine?

#27
post #24

That's a good question, and something I've spent much time on. Cuil (2008-2010) tried. I knew some of those people. It cost them about $30 million to launch a full scale search engine. They had no revenue model. In retrospect, they were hoping to be acquired by somebody. It was some ex-Google people, trying to replicate older Google technology. They had a great launch, but the system wasn't very good and traffic rapi…

There are way more than two things that Google does wrong. Remapping my search terms into oblivion so it can pretend it's fast is the worst one. Especially when this happens to a query I've modified to quote "every" "single" "flipping" "term." I think Google is cheating, and that their usable index is much shallower than they'd have you believe.

What's needed is a search engine with functional queries (as opposed to Google, which now only operates in "the user is drunk" mode), that doesn't give a damn about your robots.txt, and that can capture content in a way that is more akin to archive.org than Google's shoddy and increasingly absent cache.

Another issue is spam/false matches. Why does Google return illegitimate results? Because, let me tell you, any search for "some nifty computer book pdf" returns pages upon pages of bogus links leading to ad link mazes. A crawler should be able to trivially crawl such a page, determine that no PDF is linked, and blacklist the result, but this doesn't happen.

Google is slow and preoccupied. Their business is ripe for disruption.

Post reply on HN