Live data from Hacker News

We Need Viable Search Engine Competition

peebs.org

91–100 of 146 posts

Re: We Need Viable Search Engine Competition

#91
Is there any new information regarding the future plans for Bing? This article from a couple of months ago had it looking ominous:

http://www.bloomberg.com/news/2013-11-08/microsoft-ceo-candi...

"Besides emphasizing Office, Elop would be prepared to sell or shut down major businesses to sharpen the company’s focus, the people said. He would consider ending Microsoft’s costly effort to take on Google with its Bing search engine, and would also consider selling healthy businesses such as the Xbox game console if he determined they weren’t critical to the company’s strategy, the people said."

Re: We Need Viable Search Engine Competition

#92
post #44

Is PageRank really the indomitable tech of our generation? Nobody can do better algorithmically, or integrate some kind of crowd sourced feedback, or measure browsing time and habits, or simply hand tune some of the most competitive key phrases? I’m sure I’m oversimplifying, but I wonder if we haven’t all been hypnotised by the complexity, much of which is marketing hype... Actually, the tech is extremely complex, an…

> is DDG viable competition for Google? Will it ever be?

For search? Absolutely, it is today.

However, Google figured out a long time ago that they could lock people in by offering a hundred other services too. DDG has no maps, no mail, no image search, no video search, no drive, no docs, no news, no book search, no academic paper search, no patent search, no stock graphs, no language translation, and so on. For some of those, it can link off to other services, none of which are viable competitors to the Google equivalents.

Re: We Need Viable Search Engine Competition

#93
post #44

Is PageRank really the indomitable tech of our generation? Nobody can do better algorithmically, or integrate some kind of crowd sourced feedback, or measure browsing time and habits, or simply hand tune some of the most competitive key phrases? I’m sure I’m oversimplifying, but I wonder if we haven’t all been hypnotised by the complexity, much of which is marketing hype... Actually, the tech is extremely complex, an…

Vertical search engine might be the answer, much easier to implement and much more useful from users point of view.

I think vertical search engines are part of the answer too. For general search engines Google have such an advantage in data and R&D that only Microsoft can afford to compete.

On the data side I think http://commoncrawl.org/ can help with creating vertical search engines. Their crawl is much smaller than Google or Bing but it is web scale (2 billion pages of 2013 data). Data recency is still a problem but it can help with finding which sites belong to a niche. Some smaller scale crawling of these sites would then be a much more achievable task.

Re: We Need Viable Search Engine Competition

#94
post #44

Is PageRank really the indomitable tech of our generation? Nobody can do better algorithmically, or integrate some kind of crowd sourced feedback, or measure browsing time and habits, or simply hand tune some of the most competitive key phrases? I’m sure I’m oversimplifying, but I wonder if we haven’t all been hypnotised by the complexity, much of which is marketing hype... Actually, the tech is extremely complex, an…

> is DDG viable competition for Google? Will it ever be? For search? Absolutely, it is today. However, Google figured out a long time ago that they could lock people in by offering a hundred other services too. DDG has no maps, no mail, no image search, no video search, no drive, no docs, no news, no book search, no academic paper search, no patent search, no stock graphs, no language translation, and so on. For some…

What do you mean by "locking people in"? I don't have gmail or youtube account, or picassa or any of that crap, but just use their search because it is orders of magnitude more relevant than DDG for my queries. And I consider maps, scholar, patents to be part of the search.

Re: We Need Viable Search Engine Competition

#95
post #87

The comments here about building search engines that viably compete with Google in terms of quality are hilariously wrong. Hilariously wrong. The dominating cost is not hiring smart people to work on the problem, it's hiring enough smart people to work on the problem. Consider. Here at Microsoft, we employ a search relevance staff of -- what, 20% the size of Google's? And Google's engineers are rock solid. MS might b…

You know what? I really can't get behind the idea that MS is somehow the stirling example of engineering prowess and discipline. This is the organisation who fails their way through every other operating system. One of the poster children for missing the boat on the internet, tablets, operating systems, cloud services, and more.

I also reject this idea that Google's got a lock on every smart person ever. That they don't have any politics or wasted effort. That they're the snowflake they think they are. This kind of thinking is pathetically misguided, and a huge part of their marketing. Google is not perfect. They are staggeringly weak in the face of real competition (witness Facebook vs. G+ and Apple vs. Android).

Your argument is a replay of the prevailing attitude in the mid nineties regarding operating systems (from the same company, surprise!). And then some college student named Torvalds pushed out the most influential operating system ever, and did it for free, with the help of the world.

Nobody and nothing is invincible, and people who say that a situation is unassailable are always, categorically wrong, given enough time.

Re: We Need Viable Search Engine Competition

#96
post #87

The comments here about building search engines that viably compete with Google in terms of quality are hilariously wrong. Hilariously wrong. The dominating cost is not hiring smart people to work on the problem, it's hiring enough smart people to work on the problem. Consider. Here at Microsoft, we employ a search relevance staff of -- what, 20% the size of Google's? And Google's engineers are rock solid. MS might b…

You know what? I really can't get behind the idea that MS is somehow the stirling example of engineering prowess and discipline. This is the organisation who fails their way through every other operating system. One of the poster children for missing the boat on the internet, tablets, operating systems, cloud services, and more. I also reject this idea that Google's got a lock on every smart person ever. That they do…

Hi OP. Your points, as I understand them, are the following. Correct me if I'm wrong.

* MS is not a good example of a strong engineering org. Further, it is not a good example because Windows sucks.

* Google doesn't employ every smart person ever.

* Google can't actually compete. (ed note: in any field, or just against iOS and Fb?)

* My argument is like the argument that no one would supplant Windows.

* Given enough time, everything dies.

Points 2 and 3 are about Google. Let's start there. You're right that Google doesn't employ every smart person ever, but then, who said they did? :) But there are a limited number of search relevance engineers, and finding enough to keep up with Google is a monumental, and maybe insurmountable feat. If you want to compete with Google, you will need to have a serious advantage that supercedes this. That is just a fact.

Point 3 is about competition. I'll grant you that G+ is no Facebook, but Android is the most widely adopted mobile OS on the planet, and by a huge margin -- iOS is basically not even competitive, except for the top 5% of the market. Further, a billion smartphones will be bought this year, the most of which will be Internet enabled Android devices, and most of which will be bought in developing countries by people coming on the Internet for the first time. You tell me who's forward thinking there, because when that market comes online, it will be huge. The fact that you mention this as being non-competitive indicates to me that you might not know what you're talking about. :(

Point 1 is about the MS org. What can I say, OP, I work here, so maybe I'm not the best person to have this discussion with. But FWIW I chose this place over some much sexier jobs because the team I work with is arguably the best of its type in the world. There are bad neighborhoods, but the disparity between a good team in MS and a good team at Google is basically negligible. Also I think Windows is one of the great engineering feats of CS, so ... :| (Note that I still use UNIX at home.)

Point 4 is probably the result of confusion. I don't think no one can compete with Google. Bing has 20% market share! Clearly we can. But I do think it will be hard to compete with Google on search quality. I don't see how you can argue that.

And point 5 is obviously true but not relevant.

EDIT: Actually I now see your point 1 as saying "MS couldn't pull this off because they're not a good engineering org, but someone else could". Maybe someone else can build a better search engine, but I think what MS has pulled off with Bing is a monumental feat.

For starters, we built the entire Bing stack from scratch. No OSS. No common platforms like the JVM. Nothing like that. We started from nothing, and invented the server infrastructure, the data pipeline, the runtime that would support the site, the ML tools, everything. The fact that the site runs at all is a small miracle, but the site does not "just" run: the most remarkable thing by far is that the quality of our tooling is quite incredible, generally an order of magnitude better than the OSS equivalents. For example, the largest deployment of an OSS NoSQL datastore seems to be a few thousand nodes. The small NoSQL cluster backing our MapReduce implementation is stably deployed on a cluster an order of magnitude larger than this. This is something you only really see at companies like Amazon, Google, or MS.

I understand that the consumer market is not something MS is strong at, but I am hoping this gives you a taste of the scale and quality of what's happening behind the scenes. Happy to talk more about this if you drop me a line or skype me at `mrclemmer` :)

Re: We Need Viable Search Engine Competition

#97

I agree with the article. I wrote last week ( http://markwatson.com/blog/2013-12/practical-internet-securi... ) about using two browsers. Chrome with default security settings: used only for gmail, google search, facebook, and twitter; Firefox with ghostery: all normal web browsing, search using duckduckgo and bing. My setup encourages me to at least use duckduckgo and/or bing when I am using firefox.

FYI: I would keep tabs on Ghostery, there's a for-profit company behind it that works with Ad Networks - They don't do anything bad now but experience shows that a massive user-base that you make zero revenue off can be a tempting fruit when profits start falling. http://en.wikipedia.org/wiki/Ghostery#Criticism

Thanks Andre, I will watch out for that.

Re: We Need Viable Search Engine Competition

#98
post #67

One of the viable venues to compete with Google is not by developing "new, better search engine", but rather "better vertical platform". What's the best search engine where buyers go? Not google.com and never really been google, it's amazon.com and ebay.com. What's the best search engine to find answers on technical questions? Stackoverflow.com is approaching google very fast. So the answer would be for sharp minds t…

That's exactly what my startup is trying to address - how do you help users navigate 100, 1000 or even a million search engines? Ultimately, you need a way to browse search engines by topic and bundle them into custom lists for your specific needs. It's early going, but check Nuggety out if you're interested: http://nuggety.com/

Re: We Need Viable Search Engine Competition

#99
post #48

The complaint seems to boil down to how Google deals with spammers. The question is, how is competition going to make it better? If we instead had five search engines that each had 20% market share, would people stop trying to game them? Would false positives in efforts to thwart spammers be eliminated? How? The problem isn't even one that responds to market forces -- the "victims" (sites that should rank highly in o…

You falsely assume a new search engine can only be "as good as Google". Google's search has become highly commercialized over the years, it used to be about finding the "best" answer to your search query (the "original" PageRank), but it has increasingly become the "best consumerist" answer to your query (where to buy something). This make it more susceptible to spammers, as their basic incentives are similar. New, d…

A search engine that didn't bury good quality results under vast amounts of recent ("fresh") blog spam would be a useful start...

Re: We Need Viable Search Engine Competition

#100
post #33

Perfectly valid expectation with near impossible odds of coming true. Google has gotten so rich, entrenched and popular that IMHO no competitor can dislodge it. I say this as a thoroughly-disappointed user who's tried nearly all the alternatives to Google in the various segments that it operates in. I've managed to stop using nearly all Google services except Android (running CM) and Search. As others have pointed ou…

If you think Google is impervious to competition, maybe you're thinking about the challenge in the wrong way. If someone beats Google it won't by being exactly the same as Google but better, but by solving what Google solves in a better way. A relevant Quora answer: http://www.quora.com/Search-Engines/What-should-a-new-search...
Post reply on HN