Google Web spam - Gabriel Weinberg's Blog

Post by gojomo » Tue, Jul 27, 2010, 10:31 PM UTC

epi0Bauqu wrote: post #10

moultano wrote: post #9

Being in the index isn't very meaningful. Very little of our spam-fighting/ranking prevents sites from showing up for "site:" queries, because in general we think that if someone is intending on going to a domain directly, the only reasonable thing to do is to show that domain. Unfortunately I don't have a better way of assessing it to offer. Internally, we often look at impression-weighted precision as a metric, but…

I think this method would violate (at least the spirit of) my privacy policy: http://duckduckgo.com/privacy.html That said, if anyone has a meaningful sample query set, I'm certainly interested in running it against my spam index. I see a lot of hits on it via other search APIs.

Post by pierrefar » Tue, Jul 27, 2010, 11:03 PM UTC

moultano wrote: post #9

Being in the index isn't very meaningful. Very little of our spam-fighting/ranking prevents sites from showing up for "site:" queries, because in general we think that if someone is intending on going to a domain directly, the only reasonable thing to do is to show that domain. Unfortunately I don't have a better way of assessing it to offer. Internally, we often look at impression-weighted precision as a metric, but…

Post by pierrefar » Tue, Jul 27, 2010, 11:17 PM UTC

moultano wrote: post #14

pierrefar wrote: post #13

Earlier quoted context omitted.

Can you please explain "impression-weighted precision" and how it's used? Sounds very interesting.

In general we want to measuring what matters to users, so removing a lot of spam that nobody sees doesn't really make anything better. By the same token, if you are trying to launch a new spam classifier that has some false positives, if one of the false positives is yahoo or facebook, it doesn't really matter how good it is, it will never be worth the collateral damage. As a result, rather than measuring precision/r…

Post by moultano » Tue, Jul 27, 2010, 11:46 PM UTC

apollo wrote: post #5

sadiq wrote: post #2

On an tangential note, it's a shame that Google don't offer a service like BOSS (though BOSS could be enhanced by offering revenue sharing, as an alternative to charging per query). It seems the Google search APIs have actually gone backwards over the last few years.

Yahoo's goal with BOSS is to fragment the search market; Google is so far ahead that Yahoo knows they can't compete head-on. Google doesn't want the search market to be fragmented; they want to dominate the market. I think that's why Google doesn't have a good search API offering.

Post by omarchowdhury » Tue, Jul 27, 2010, 11:57 PM UTC

pierrefar wrote: post #15

moultano wrote: post #14

Earlier quoted context omitted.

In general we want to measuring what matters to users, so removing a lot of spam that nobody sees doesn't really make anything better. By the same token, if you are trying to launch a new spam classifier that has some false positives, if one of the false positives is yahoo or facebook, it doesn't really matter how good it is, it will never be worth the collateral damage. As a result, rather than measuring precision/r…

Thanks. Am I right in understanding that if a SERP result for a given keyword doesn't get clicked by users enough, it will be removed? By the same token: does the result's bounce rate matter? I imagine spammy sites have a very high bounce rate.