Live data from Hacker News

Google should open source what actually matters: their search ranking algorithm

cdixon.org

31–40 of 70 posts

Re: Google should open source what actually matters: their search ranking algorithm

#31
Welcome to: http://www.wowowbiz.com The website wholesale for many kinds of fashion shoes, like the nike,jordan,prada,adidas, also including the eans,shirts,bags,hat and the decorations. All the products are free shipping, and the the price is competitive, and also can accept online payment.,after the payment, can ship within short time. http://www.wowowbiz.com our price: jordan air max oakland raiders $34--39; Ed Hardy AF JUICY POLO Bikini $25; Christan Audigier BIKINI JACKET $25; gstar coogi evisu true jeans $35; coach chanel gucci LV handbags $36; coogi DG edhardy gucci t-shirts $18; CA edhardy vests.paul smith shoes $32; jordan dunk af1 max gucci shoes $37; EDhardy gucci ny New Era cap $16; coach okely Adidas CHANEL DG Sunglass $18; http://www.wowowbiz.com thank you for your visit Wish you a merry Christmas

Re: Google should open source what actually matters: their search ranking algorithm

#32
post #20
post #2

The notion that being 'committed' to open source means that a company should give away everything of value is ludicrous. Google came up with great software that makes millions of peoples' lives easier, and they have every right to keep it secret and profit off of it. The zealous idea that open source and trade secrets cannot coexist is detrimental to the open source movement as a whole, in my opinion.

Additionally it may be the case that a large part of the value of the algorithm is created from its secrecy. If the algorithm was open it would be trivial to create sites with artificially high rankings which would reduce the value of the search results.

> If the algorithm was open it would be trivial to create sites with artificially high rankings

I don't necessarily think this is true. Sites that have high page rank, do so because other sites of high value link to them.

Even if Google open sourced their search engine, spam sites would still have a hard time getting high quality inbound links. The reputable and high ranking sites wouldn't have any increased incentive to link to low quality spam sites.

Re: Google should open source what actually matters: their search ranking algorithm

#33
I upvoted the story not because I agree with it, but because it explains a very successful open sourcing strategy that I wish more people understood.

In general people are willing to spend a certain amount on a solution. Therefore if you can things that make the complement to what you make money on cheaper, that increases how much you can charge for your piece of the solution. Open sourcing is a way to make those pieces cheaper.

Google does this. They give away tools and knowledge about how to make websites. They give away analytics and an A/B testing platform and educate people about how that makes it easier to produce good websites. This makes building good websites cheaper, and therefore increases how much people are willing to spend to advertise those websites.

This isn't a new strategy. In the late 90s when Oracle and IBM got into open source, this was their strategy. Oracle wanted to get people to stop paying for Sun hardware so Oracle could negotiate higher prices. (I know more than a few people who were surprised to find that licenses for Oracle on Linux are higher than they are on Sun.) IBM was looking to reduce the price for complements to IBM websphere, and was looking for opportunities to avoid having customers pay the Microsoft tax.

If you run a company and wish to strategically open source something, you should think in the same terms. What is the complement to what you are charging for, and how can you make that cheaper?

Re: Google should open source what actually matters: their search ranking algorithm

#34
post #20

Earlier quoted context omitted.

Additionally it may be the case that a large part of the value of the algorithm is created from its secrecy. If the algorithm was open it would be trivial to create sites with artificially high rankings which would reduce the value of the search results.

> If the algorithm was open it would be trivial to create sites with artificially high rankings I don't necessarily think this is true. Sites that have high page rank, do so because other sites of high value link to them. Even if Google open sourced their search engine, spam sites would still have a hard time getting high quality inbound links. The reputable and high ranking sites wouldn't have any increased incentiv…

It's not the ranking algorithm specifically that's a secret; it's the code that used to detect sites that work together to artificially raise page rank of another site by linking to it. The whole Page Rank algorithm is predicated on the idea that human website owners and contributors do a better job at assessing the value of a website and they express this by linking to the website. When links are created for other reasons they reduce the effectiveness of Page Rank, and when they are created specifically to bias Page Rank they do the most damage to the algorithm's results.

Keeping the algorithm secret, and in particular the tweaks that eliminate links which aren't expressing "I judge this to be a good site", keep Page Rank useful.

Re: Google should open source what actually matters: their search ranking algorithm

#35
post #9

I would love if they open-sourced their infrastructure tools such as GFS, BigTable and Map-Reduce tools. Most of these tools are built using open-source technology such as Linux.

Apache's Hadoop ( http://hadoop.apache.org/ ) has open-sourced implementations of those tools.

Yahoo was the company that opensourced hadoop, not google.

Re: Google should open source what actually matters: their search ranking algorithm

#36
post #20

Earlier quoted context omitted.

Additionally it may be the case that a large part of the value of the algorithm is created from its secrecy. If the algorithm was open it would be trivial to create sites with artificially high rankings which would reduce the value of the search results.

> If the algorithm was open it would be trivial to create sites with artificially high rankings I don't necessarily think this is true. Sites that have high page rank, do so because other sites of high value link to them. Even if Google open sourced their search engine, spam sites would still have a hard time getting high quality inbound links. The reputable and high ranking sites wouldn't have any increased incentiv…

Absolutely. I've worked in several contracted roles within online marketing and I've heard some awful advice from self-proclaimed SEO gurus about how to achieve a high rank for keywords on Google. A lot of what achiving high ranks is about is common sense and as Google improves their algorithm fewer tricks work. Anything past a simple competitive analysis, keyphrase analysis and a backlink analysis on you and your competitors is overkill.

The quality of SEO analysts out there is so poor that I believe you could release the source code to Google's entire search engine and people would still get things wrong or believe in ways to get around what I assume is an extremely complex set of mathematically-oriented algorithms. It's a classic example of the blind following the blind, and those that typically have followers are those that are popular for performing SEO work on huge companies that would never actually need it (e.g. Coca Cola).

Re: Google should open source what actually matters: their search ranking algorithm

#37
And what is wrong with commoditizing the complement? Google's working hard to make operating systems free and we'll probably see Microsoft working hard to open up search engines. End result: both operating systems and search engines will be open and the consumer benefits.

Re: Google should open source what actually matters: their search ranking algorithm

#38

I don't get it, is the author talking about PageRank? http://en.wikipedia.org/wiki/PageRank What more does he want?

The algorithm used by Google today is light-years away from that described in the PageRank paper. It uses a form of PageRank, but it also uses many other parameters.

If it's secret, then how can you be so sure?

My impression is that PageRank, while initially extremely successful at filtering signal-from-noise, is very susceptible to manipulation. For example links farms which make unimportant pages seem important. As a result of such manipulation, Google started modifying their crawling, in order to reduce the amount of garbage going into PageRank (less garbage in, less garbage out). For example, they might look at the age and ownership in DNS records, and exclude domains that look fishy based on some heuristics. The spammers fought back with more tricks, and the resulting arms race has come to have an acronym: SEO. But the underlying idea of using an iterative approximation of the likelihood of encountering a page via a random walk of the web is preserved. (Some details here: http://www.miislita.com/information-retrieval-tutorial/matri...)

If that's right (and I could well be wrong... would like to be convinced...), then I don't see it as a "light-years away" kind of issue, but rather a series of hacks to stay one step ahead of the spammers. I don't see any particular value in open sourcing those.

Re: Google should open source what actually matters: their search ranking algorithm

#39
Well, I can't understand this article, there is not "secret sauce". The google pagerank algorithm is open for all to see and understand. I myself made my own simple implementation.

Everybody could do it. This man is not asking for code, is asking for DATA only google has. It's like an atheist saying to a catholic what he must do because he believes. When this man gives something of value to the world,like google did(I remember the Altavista-Ad-flickering in your eyes days) then he could demand to be given by others.

Re: Google should open source what actually matters: their search ranking algorithm

#40
post #13

Google is a business. Their job, legally defined, is to create value (aka profit) for its shareholders. A more open web has helped Google, but opening up its search algorithms, a cornerstone to Google's growth, is just ludicrous and bad business. Compared to a lot of other companies (Apple, Microsoft, nearly every other tech giant), they are an open company. It's already great that they realize there's value in an Op…

Their job, legally defined, is to create value (aka profit) for its shareholders. This is a meme I see a lot, but never with any evidence. That the board has a fiduciary duty to the shareholders I can understand. That it would have a legal obligation to create value, well, I think that would be hard to define, and I have never seen any evidence for the existence of this obligation.

It may be true that it is a company's legal responsibility to serve its shareholders first, but that's not the way it should be. With this mentality, corporations will minimally follow the law (if the fine is less than the profits, the corporation may be legally required to break the law!) and do whatever they can in their own interests. Society as a whole would benefit massively if caring for others and society in general was regarded as a good thing.
Post reply on HN