Blekko already has a list of domains to block from results • Hacker News Board

Re: Blekko already has a list of domains to block from results

#21

Post by aj700 » Fri, Jan 07, 2011, 10:42 AM UTC

Omitting these domains from results is "automated".

Managing the list isn't. It's based partly on how many users report a domain as being spam. At least that's one of the reasons for inclusion. And don't bayesian filters, with little data to work with and if newly implemented always have false positives?

Maybe some are labelling valid stuff as spam out of spite.

When blekko has millions of users labelling stuff as spam instead of very few, the system will be harder to abuse and the list much better.

Re: Blekko already has a list of domains to block from results

#22

Post by sdfghrjtr » Fri, Jan 07, 2011, 10:43 AM UTC

●---------------------------------------● ●------ http://goo.gl/l5v0b ------●

●------ http://goo.gl/l5v0b ------●

●------ http://goo.gl/l5v0b ------● ●---------------------------------------●

Re: Blekko already has a list of domains to block from results

#23

Post by nervechannel » Fri, Jan 07, 2011, 10:56 AM UTC

greglindahl wrote: post #8

cubicle67 wrote: post #4

can anyone explain what bayes and mfa mean? I picked the site http://www.basemetals.com/ (bayes (spam 8.6 > 5.3)) at random, and although it won't win any design awards I can't see what the problem with it is. Am I missing something?

mfa = Made for Adsense. That means we believe that the domain seems designed more to show ads than to provide content. bayes = our Baysian analysis gizmo thinks bad things about this site's content. Too much Viagra, not enough content. Like all artificially-intelligent things, it can sometimes be hard to see why it's upset.

Maybe try using something like a decision tree where the classification steps are much more obvious.

There are parallelizable implementations out there...

Re: Blekko already has a list of domains to block from results

#24

Post by robinduckett » Fri, Jan 07, 2011, 11:25 AM UTC

I'm seeing:

http://www.deadmau5.com -> Really, spam?

http://www.comparethemarket.com -> Price comparison site with no known biases in the UK and not owned by British Telecommunications, as it is listed.

Even ONE false positive is enough to make me think this listing is a load of bullshit. How is geocities' "closing down" page spam?

Re: Blekko already has a list of domains to block from results

#25

Post by viraptor » Fri, Jan 07, 2011, 11:35 AM UTC

Doesn't include swik.net which is a crap link aggregation / search tag spam - I'd expect that one to be removed...

Re: Blekko already has a list of domains to block from results

#26

Post by JamesDB » Fri, Jan 07, 2011, 11:42 AM UTC

robinduckett wrote: post #24

I'm seeing: http://www.deadmau5.com -> Really, spam? http://www.comparethemarket.com -> Price comparison site with no known biases in the UK and not owned by British Telecommunications, as it is listed. Even ONE false positive is enough to make me think this listing is a load of bullshit. How is geocities' "closing down" page spam?

A site in the UK that gets a huge amount of traffic and is backed by a large off-line national advertising campaign. Why has this been banned? Never seen spam on there, nor have seen them spamming.

Re: Blekko already has a list of domains to block from results

#27

Post by kilian » Fri, Jan 07, 2011, 1:00 PM UTC

I wonder why they have both the www and the non-www version in there for domains?

Re: Blekko already has a list of domains to block from results

#28

Post by vaksel » Fri, Jan 07, 2011, 1:06 PM UTC

aaronbrethorst wrote: post #13

Wow, I'm seeing tons of false positives. Why on earth is the co-written blog of a nobel laureate and a 7th circuit appeals court judge making blekko's bayesian classifier freak out? #88 - http://www.becker-posner-blog.com - bayes (spam 31.6 > 5.3)

my guess is the fact that it has two dashes, which is almost always indicative of spammers.

Re: Blekko already has a list of domains to block from results

#29

Post by eli » Fri, Jan 07, 2011, 1:37 PM UTC

Misha_B wrote: post #19

In spite of the (justified) complains you get about the false positives, I think that's a great way to go. Unlike with email where missing a message might be critical, in search I'd rather have even as much as 10-20% false positives than deal with the spam sites Google delivers. More in general, concerning the front page search examples: "cure for headaches" works very well indeed compared to google. However, "global…

Really? I'd rather have spam sites than 20% of my legitimate results missing.

I guess it depends on what you're searching for. "cure for headaches" will probably be just fine with some missing sites, but "Deadmau5 tour dates" will definitely be affected by the false positive block on deadmau5.com

Re: Blekko already has a list of domains to block from results

#30

Post by shiftpgdn » Fri, Jan 07, 2011, 1:45 PM UTC

kilian wrote: post #27

I wonder why they have both the www and the non-www version in there for domains?

Google counts them as different websites. It is typically a wise SEO move to use your .htaccess file to setup a 301 redirect to/from the www. version of your site.