Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How My Popular Site was Banned by Google

kbrower.posterous.com

Post reply on HN
Print view View original

91–100 of 132 posts

Re: How My Popular Site was Banned by Google

#91

Post by option1138 »

Matt_Cutts wrote: post #85
eurohacker wrote: post #77

Earlier quoted context omitted.

As we have heard lately how Google has manipulated the Yelp search results and others for their benefit, one has to wonder - since google has aquired some big websites like Zagat, theoretically - if one day they will just start to screw Zagat competitors , what can be actually done about it , i mean how can one prove that google is just attacking you and they just dont like you as their competitor ... its relative -…

It's a fair question, but our quality guidelines have been quite consistent for the last decade or so. Our team takes action without regard to whether a website is an advertiser, partner, or competitor. The long-term loyalty/trust of our users is worth much more than any sort of short-term revenue that a sneaky deal would provide. And Google's culture is such that anyone inside or outside of the company can claim tha…

That's the ultimate check on Google: if we start to act too abusive or "evil" we know that people can desert us. So it's in our enlightened self-interest to try to act in our users' long-term interests.

I hear this line thrown about quite a bit. And while it's true with regular users, it's certainly not true for webmasters or advertisers. Google controls around 67% of all US search share. If an advertiser doesn't play by your rules, they forfeit a significant amount of natural search traffic.

All good for most of your policies but there are some real gray areas. I had a site years ago that got hit with a javascript hack on an obscure page. StopBadware found it within a day or two and suddenly we were blacklisted... virtually all organic traffic disappeared overnight. It took weeks to get the warning lifted and that was only as a result of a significant viral PR campaign (such as this).

Maybe this has been addressed, but there are other areas. Affiliate sites are also penalized quite heavily by Google. It's one thing to take a stand due to the supposed quality of many of these sites (which frankly has little correlation with the presence of affiliate links... most sites suck). It's quite another when Google has a large affiliate advertising practice in house and a significant investment in an affiliate link tracking/cloaking company.

I mention this with all due respect and I hope you take it as constructive criticism. I think the organic side of the house does a great job overall. The paid side is another story IMHO. Part of this is organizational stupidity... I struggle with this every day and I have a much smaller organization.

Re: How My Popular Site was Banned by Google

#92

Post by option1138 »

SteveOllington wrote: post #89

lol, poor Matt, always under fire from everybody who hasn't got the rankings they want. It's simple really, Google try to make it fair for everybody (and granted it doesn't always work perfectly, which is why the continue to make changes, it's evolution) and if some sites give themselves an unfair advantage and are caught, then they're punished. How else could a search engine work? If it were any different then al of…

This is a fair comment but I believe the issues stem more from the process Google follows.

It is obvious that Google cannot communicate exact reasons why a site was penalized as that would help spammers. However, there is nothing that prevents them from adding a step to warn the offending website and give them a heads up before the ban/penalty takes place, along with an explanation of the policy that is/was being violated.

Most of these heads up would go ignored, some would not and yes, it would incur a support cost. However, the number of websites which are significantly penalized isn't onerous... I believe fewer than 1,000 each year?

When a company has become the defacto gateway to the internet, I believe they have a responsibility to webmasters. Google has lost a lot of goodwill over the years because of these seemingly arbitrary penalties... Instituting such a practice would be a worthwhile investment.

Re: How My Popular Site was Banned by Google

#93

Post by tripzilch »

Matt_Cutts wrote: post #15

You have an autogenerated web site that consists of practically nothing other than affiliate links to Amazon. You can make an infinite number of autogenerated pages on your site, e.g. http://www.filleritem.com/index.html?q=hacker+news http://www.filleritem.com/index.html?q=31.69 http://www.filleritem.com/index.html?q=teen+sex and each autogenerated page consists of literally hundreds of affiliate links stuffed with k…

Well, since this is a site people have a need for, look for and use, that must mean the process involving your "quality guidelines" is wrong, isn't it?

However, you seem to imply there's something wrong with the site.

Let me ask you a question, who decides whether a site is "right" or not? Google or the people that use the web? What was the original purpose of PageRank?

You can make excuses all you want, but fact of the matter is that Google's Search Results quality has been plummeting over the past couple of months and it all stems from Google pretending to better know what people want than the people themselves.

Good luck with your website.

Re: How My Popular Site was Banned by Google

#94

Post by tripzilch »

Matt_Cutts wrote: post #36
gojomo wrote: post #28

Earlier quoted context omitted.

Still, when used as intended (with the query a dollar amount, like your 31.69 example), isn't his site useful ? (I've not noticed a similarly straightforward tool for this same purpose at Amazon.com itself.) The non-dollar-amount queries seem like a bug -- triggering a default of listing up from 0.00, unrelated to the query. If this were an intentional manipulative tactic, wouldn't the outlinks be related to the quer…

I'm not saying that the issues aren't fixable. But when the site came up for a review, we saw things that violate our quality guidelines: autogenerated pages with hundreds of affiliate links that consist of lots of keywords, and the links/keywords are duplicate content. If he fixes the issues by e.g. blocking out the autogenerated pages, then I expect his homepage will be found in Google. The autogenerated pages are…

Why would he "fix" anything, it was your mistake and it's your Search Results that are becoming less and less useful as a consequence.

Re: How My Popular Site was Banned by Google

#95

Post by cheae »

Iv wrote: post #84
mapgrep wrote: post #74

Earlier quoted context omitted.

No one should be banned from Google for failing to fill out robots.txt in the manner Google prefers. This is a horrible abuse of power -- and a competitor -- by Google. "Quite simple really."

Even if I generally like Google, we do need a competitor. I occasionally use duckduckgo

"We do need a competitor" I think that's the core issue here. I run a portfolio of sites, did a series of experiments on Google's quality guidelines. Frustratingly, they are nowhere closer to finding the quality content as humans do, but they think they know it and implementing it.

My experimental sites(the contents that I made sure they are crappy in the eyes of humans) with cheap content ($3/100 words) without any editorial control marked as quality sites and perform well in searches. While the sites that we spent thousands of dollars with strict editorial control were punished for some reason.

At one point a scrapper site that picks up one of our sites partial feed with 150 words excerpt out ranked us in the search results. What kind of quality guideline is that, a 750 words post is bad quality while a 150 words excerpt of that is good.

Google's motive may be good, observing it for the past 7 months, I found their approach is wrong.

As you do I use BIng/DuckDuckGO these days. Google has so much market share (around 85% in my observation ), if it goes down to 30% then we don't need to worry about pleasing Google, but focus on pleasing our customers/visitors like normal business do.

Re: How My Popular Site was Banned by Google

#96

Post by Matt_Cutts »

relix wrote: post #86
Matt_Cutts wrote: post #80

Earlier quoted context omitted.

I take your points, and in my group we've worked hard in the last year to improve our transparency and our appeals process, e.g. see http://googlewebmastercentral.blogspot.com/2011/09/reconside... from a few weeks ago. Likewise, AdWords now has phone support for all AdWords advertisers. I can't speak for the AdSense (for Domains) team other than to say that when they shut down an account, they think that they have go…

Support for all Google users would obviously be painful. I think it would be fair if there was some personal support for people who either spend a lot of money on Adwords (I think there already is?) or make a lot of money on Adsense. These people probably have the highest levels of stress due to "insta-evaporation probability", and no doubt the amount of support people would be several magnitudes less if you used a t…

There is phone support for everyone who uses AdWords now. I don't know the support situation for AdSense though.

Re: How My Popular Site was Banned by Google

#97

Post by Matt_Cutts »

option1138 wrote: post #92
SteveOllington wrote: post #89

lol, poor Matt, always under fire from everybody who hasn't got the rankings they want. It's simple really, Google try to make it fair for everybody (and granted it doesn't always work perfectly, which is why the continue to make changes, it's evolution) and if some sites give themselves an unfair advantage and are caught, then they're punished. How else could a search engine work? If it were any different then al of…

This is a fair comment but I believe the issues stem more from the process Google follows. It is obvious that Google cannot communicate exact reasons why a site was penalized as that would help spammers. However, there is nothing that prevents them from adding a step to warn the offending website and give them a heads up before the ban/penalty takes place, along with an explanation of the policy that is/was being vio…

option1138, I'm afraid you need to recalibrate your expectations of spam on the web. Blekko made a site called Spam Clock that estimates 1 million spam pages are created every hour: http://www.spamclock.com/ .

There's 200+ million websites out there. 1,000 spam sites would be a spam rate of 5.0 × 10^-6. If you remember the days of Altavista before Google, the actual rate of spam on the web is much higher. Here's one stat: I once heard a search engine rep (not from Google) say that they had to crawl 20 billion pages to find 1 billion non-spam pages.

So yes, we do tackle more than 1,000 websites a year. There's a ton of spam on the web, and Google has to operate on the scale of the web (e.g. in 40 different languages) to tackle all that spam.

Re: How My Popular Site was Banned by Google

#98

Post by OpenAlgorithm »

option1138 wrote: post #72
OpenAlgorithm wrote: post #41

Earlier quoted context omitted.

Would it be likely that if the site was to use another monetization plan that it would be less likely to be penalized e.g. advertising instead of affiliate links?

Sadly, quite true. "True" - because my current understanding (which Matt_Cutts can elucidate on if he chooses to) is that Google has looked into - but does not currently incorporate - the presence of advertising as a spam signal. "Sadly" - because my independent research has shown that advertising - most notably the presence of Google AdSense - is a reliable predictive variable of a page being spam. All things being…

I agree but don't you think that from an algorithmic point of view Google would be better looking at what the user wants and what monetization models they prefer to see versus the averages in terms of monetization models on spam sites.

That way their focussing less on removing spammers and more on user quality and thus removing spam.

Re: How My Popular Site was Banned by Google

#99

Post by mapgrep »

ErrantX wrote: post #88
mapgrep wrote: post #74

Earlier quoted context omitted.

No one should be banned from Google for failing to fill out robots.txt in the manner Google prefers. This is a horrible abuse of power -- and a competitor -- by Google. "Quite simple really."

I disagree; Google should have strong rules to filter out crap and give me great search results. In this case we have something of an edge case; the site is useful, but the auto-generated pages cause potential problems for search results. If you were talking about how Google communicates these issues, or how it goes about resolving these edge cases, then I'd be agreeing. Much to be improved. But getting rid of the po…

Of course it should. But a rule that says you're banned because you're too open with search engines is insane. robots.txt should have nothing to do with it.

Re: How My Popular Site was Banned by Google

#100

Post by xpose2000 »

http://www.filleritem.com/index.html?q=teen+sex This should have no results. Other queries should have relevant results, but they don't. How can anyone complain over his site being axed?

The guy needs to code a better site and I sure as hell would never use that so he can get affiliate money.

Sorry, but your fortunate this site has survived for so long.

Post reply on HN