Google Needs Sex

One could, in theory, manually create a database of a few million (or way more) websites, and rate the content in each of them (including advertisements - just how the page is showed to the end user). This database could then be used to train algorithms, with genetic fitness based on how close the ratings of the algorithm are to the human ratings of value (note that the algorithms would not be aware of the human value ratings).

A second - perhaps slightly smaller - database could then be used to test the performance of the best algorithms in the "real world," or at least on data which they weren't trained on. This would select against algorithms which are adapted solely for the first database. Content ratings generated algorithmically could then be used to modify the ranking of websites in the results, penalizing websites that seem to have bad content.

I'm not sure how I got here from thinking about sex between search engines; I suppose it's because one way to deal with the evolution would be taking the best few algorithms each time and combining them (which I'm sure that there are issues with). Of course, people far more intelligent than I have certainly had this idea before, and probably figured out why it wouldn't work (or, alternatively, that it would work, which would suggest that someone is now busy implementing it).

Re: Google Needs Sex

#6

Post by gregpilling » Tue, Jan 11, 2011, 3:33 AM UTC

Great headline, crappy article. People have been trying to game search engines since a few minutes after they were invented. And other search engines don't appear to be better, or the market share would have shifted by now. It costs nothing to the consumer to switch, and people talk about which websites they like - so if something that is more appealing to Joe Public comes along, it will be noticed. Bing's ascent in the number of searches has not been huge yet. Personally I hope for more than one search superpower in the future.

Re: Google Needs Sex

#7

Post by melvinram » Tue, Jan 11, 2011, 3:37 AM UTC

What the author is saying, as I understand it, is that Google needs a way to mix up their algo's so they aren't easily "gamed" by scammers and spammers.

The question is not whether Google is doing this actively but how fast and decisively they do this as well as what specific issue they deem as priority to address.

Google's algo, as I understand it, is an array of knobs that are turned up and down to increase or decrease impact of various factors such as domain age, incoming links, quality of content, etc. Google is always adjusting these knobs and adding/removing knobs. The goals of adjusting these knobs are obviously only known by Googlers but what is clear to many is that the sole object of adjusting these knobs in not to make the results pages more relevant. Don't get me wrong... they definitely care about relevancy (or they wouldn't have the trust of millions) but they are a public company with obligations to meet so they must take many factors into account when making adjustments.

The point? The sex (aka variation over time) that the author refers to will likely come from adjusting existing knobs and introducing new knobs such as social authority, locality, etc... but it doesn't necessarily mean that problems of relevance as we see it will get fixed through these variations. Google will take into account a number of factors when deciding what problems they want to solve and to what extend.

Re: Google Needs Sex

#8

Post by mikeklaas » Tue, Jan 11, 2011, 3:43 AM UTC

This is quite possibly the dumbest article about technology I've seen in a mainstream publication.

What Google really needs is a willingness to accept a way higher threshold of false negatives in weeding out content. I'd love to have a "known good" version of Google that risked leaving out some content. Let's start by banning all .info domains along with any that include a hyphen.

Re: Google Needs Sex

#9

Post by beoba » Tue, Jan 11, 2011, 3:46 AM UTC

sorbus wrote: post #5

One could, in theory, manually create a database of a few million (or way more) websites, and rate the content in each of them (including advertisements - just how the page is showed to the end user). This database could then be used to train algorithms, with genetic fitness based on how close the ratings of the algorithm are to the human ratings of value (note that the algorithms would not be aware of the human valu…

The catch is this database would need to be kept away from spammers, lest they be able to test their site designs against it directly.

Re: Google Needs Sex

#10

Post by michaelcgorman » Tue, Jan 11, 2011, 3:53 AM UTC

I think he's suggesting a genetic algorithm... Google-style organisms, competing for fewest retried searches, most-different front-page results, etc., that over time merge and split in various ways. Seems pretty similar to some of the tactics Google's already trying to me.