Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google should open source what actually matters: their search ranking algorithm

cdixon.org

Post reply on HN
Print view View original

61–70 of 70 posts

Re: Google should open source what actually matters: their search ranking algorithm

#61

Post by iamelgringo »

showerst wrote: post #2

The notion that being 'committed' to open source means that a company should give away everything of value is ludicrous. Google came up with great software that makes millions of peoples' lives easier, and they have every right to keep it secret and profit off of it. The zealous idea that open source and trade secrets cannot coexist is detrimental to the open source movement as a whole, in my opinion.

The notion that being 'committed' to open source means that a company should give away everything of value is ludicrous.

I disagree with the statement that Google would be giving away everything by opening up page rank. Google's "secret sauce" is hardly page rank alone. Google is really secretive about this stuff, but they arguably have the world's larges corpus of multilingual text in the form of their html index, coupled with the world's largest fault tolerant compute cluster, the largest repository of scanned text,the largest repository of digital video, and probably the largest repository of digital audio in the world. I'm sure that they also have the world's largest repository of people's surfing and searching habits, along with the one of the largest sets of analytics data.

Even if page rank, map reduce, GFS, etc.. were open sourced I'm sure they'd still find a way to make billions off of the data that they have.

But, I think the larger point that the author makes, is that unlike many giant technology companies (Microsoft, HP, Apple, etc..), Google has made billions off of using open source software. And, while they are a very large supporter of open source, what total percentage of their profits go back to OSS? What percentage of their code gets re-released as open source? These are good questions to ask, and I'd love to hear an open discussion about it.

Re: Google should open source what actually matters: their search ranking algorithm

#63

Post by ez77 »

ez77 wrote: post #54

As a matter of fact, I think Google represents the ideal business model: one which should work even in a patents-free, copyright-free world.

Let me elaborate. Not even Stallman would call for this... What can of totalitarian regime is desired, where businesses (even individuals?) are forced to publish all of their code? Wouldn't it be good enough to do away with patents and copyright? As long as I manage to keep my stuff private, let me (and Google) face the advantages and disadvantages.

Some balance in the force, for Yoda's sake.

Re: Google should open source what actually matters: their search ranking algorithm

#64

Post by eli »

xcombinator wrote: post #39

Well, I can't understand this article, there is not "secret sauce". The google pagerank algorithm is open for all to see and understand. I myself made my own simple implementation. Everybody could do it. This man is not asking for code, is asking for DATA only google has. It's like an atheist saying to a catholic what he must do because he believes. When this man gives something of value to the world,like google did(…

The documented PageRank algorithm doesn't have very much to do with how results are actually returned from Google today.

Re: Google should open source what actually matters: their search ranking algorithm

#65

Post by eli »

mbreese wrote: post #53
jacquesm wrote: post #45

Earlier quoted context omitted.

> Perhaps transparency is what Google needs the most at the moment. Absolutely, and not only on the search engine part of it, Adwords/Adsense are extremely opaque and as a publisher continuously give you the feeling you're being screwed without any chance to prove that for lack of information.

As a publisher, couldn't you do some sort of A/B testing with another ad provider? Something like 10% of page views using another set of ads? Or, maybe setup some kind of onclick tracking to verify ad clicks? I don't know what could be done as verification for the advertiser, but the publisher should be able to do something, right?

No one else gets enough volume :/

Re: Google should open source what actually matters: their search ranking algorithm

#66

Post by anigbrowl »

idlewords wrote: post #21
mattmanser wrote: post #15

Earlier quoted context omitted.

I think you're missing the point, the article is pretty much a direct response to: http://googleblog.blogspot.com/2009/12/meaning-of-open.html (submitted here: http://news.ycombinator.com/item?id=1008908 ) In it a Google open source zealot rants wildly about openess being ultimate, apart from search algorithms which should be closed. It also fails to mention all the times open has failed (communism being one of the m…

How on earth was communism an example of "open"?

Well, superficially it sounds like pure economic democracy - anyone who has a good idea that would benefit people proposes it, and resources are allocated to them by consensus. That's the seductive part, but it takes no account of the variations in intelligence, foresight and altruism, which is why it only works in Star Trek.

I don't think communism is evil so much as fundamentally naive, its proponents assume the mass of people think alike and that selfishness is a learned behavior imposed from above by wicked capitalists. It lasted as long as it did in the USSR because both military and scientific folk are into the idea of setting aside individual gain for a greater purpose, such as patriotism or the advancement of knowledge.

Communism is bad if you look at it from the perspective of a democracy with a functioning market. Then again, that perspective is a luxury if you have the misfortune to be stuck in a feudal society - most places where it took root were extremely dysfunctional monarchies prior to turning communist. Aristcracies aren't what we'd call ideal societies either; even here in the west, the replacement of aristocracy by parliamentary government was viewed by traditionalists as a form of godless mob rule.

And it's not that long ago that people were complaining that GPL represented a kind of license tyranny which would undermine proprietary innovation.

Re: Google should open source what actually matters: their search ranking algorithm

#67

Post by Semiapies »

RyanMcGreal wrote: post #60
jacquesm wrote: post #49

Earlier quoted context omitted.

> Communism is a spectacular failure of centralized planning over the entire economy. Wow, and here I was thinking it was a form of government. Communism in principle has absolutely nothing to do with the flow of information and/or decision making, and its implementations have only very little to do with its basic ideas. A true communistic government has never even been tried, we've only seen a 'new boss same as the…

>Wow, and here I was thinking it was a form of government. Like most ideologies, the term "communism" is gooshy enough to mean whatever you want it to mean - but at heart, communism is concerned with the matter of who owns and controls the means of production. It's a form of government insofar as the decision on who gets to control the means of production is ultimately a political decision. Communists believe the peo…

"It's no different from "real" conservatives distancing themselves from the Bush administration because the implementation of their political system ended up violating most of its ostensible core values."

To define American-style conservative and liberal philosophy as what the Republican and Democratic parties and their politicians do is ridiculously simplistic. The parties are just coalitions of interests and voter demographics, not philosophies - you might as well talk about what direction a plate of spaghetti points at.

And as for conservatives' "political system" - whose? Do you mean evangelicals and other theocrats, PNAC neocons, populist teabaggers, or Lew Rockwell/Ron Paul fans? We're not talking remotely the degree of unity imposed by the Politburo and Comintern upon the global communist movement when it comes to either US party. In a bipolar system, both parties must be big tents or be crushed by the bigger tent. Elected officials are not (and Bush was not) advancing a single "conservative" or "liberal" ideology; they're trying to please various power blocs who agree or disagree with them on different issues.

Bush's failures highlight the failure of particular policies and ideas, but they don't say much about the ideas of self-described "real conservatives" who thought Bush was a ridiculously free-spending, anti-federal, protectionist, power-consolidating, Big Government guy long before Iraq. (No more than Obama's decisions indict the ideas of supporters who disagree with him on those decisions, certainly.)

Re: Google should open source what actually matters: their search ranking algorithm

#68

Post by vicaya »

I'm an open source advocate (even for search algorithms in general), but ranking algo/model is inherently adversarial and gameable until true search AI (always correctly identify good and original content without false positives) is feasible.

I think it would be a good revenue strategy, if they charge $dollars to display top 10 positive features and top 10 negative features of any given query-document pair without any real weights/scores (maybe re-squashed with some logistic function). The $dollars amount is computed dynamically to offset the negative impact of the partial reveal of the ranking algo/model. This can be a tremendously useful tool for troubleshooting search results by site owners.

Re: Google should open source what actually matters: their search ranking algorithm

#70

Post by idlewords »

anigbrowl wrote: post #66
idlewords wrote: post #21

Earlier quoted context omitted.

How on earth was communism an example of "open"?

Well, superficially it sounds like pure economic democracy - anyone who has a good idea that would benefit people proposes it, and resources are allocated to them by consensus. That's the seductive part, but it takes no account of the variations in intelligence, foresight and altruism, which is why it only works in Star Trek . I don't think communism is evil so much as fundamentally naive, its proponents assume the m…

I don't think your answer makes much sense. Historically, communism had to do with allocation of resources by the state rather than a market, in the service of certain ideological beliefs. 'Openness' was not one of them.
Post reply on HN