Live data from Hacker News

How My Popular Site was Banned by Google

kbrower.posterous.com

71–80 of 132 posts

Re: How My Popular Site was Banned by Google

#71
post #53

You have an autogenerated web site that consists of practically nothing other than affiliate links to Amazon. You can make an infinite number of autogenerated pages on your site, e.g. http://www.filleritem.com/index.html?q=hacker+news http://www.filleritem.com/index.html?q=31.69 http://www.filleritem.com/index.html?q=teen+sex and each autogenerated page consists of literally hundreds of affiliate links stuffed with k…

The same could be said of Google's search engine itself: an autogenerated web site consisting of essentially nothing other than links to other places, many of them paid. But no one can deny that it's a useful site. I think that is the issue here -- probably the vast majority of such sites are undesirable, but you can't discount them all.

But Google had to spend resources to crawl those other sites, collect the data those sites created, evaluate and order the results excerpting those sites' data based on a user's query, clearly adding value for the user performing a search... Oh, wait.

Re: How My Popular Site was Banned by Google

#72

Earlier quoted context omitted.

When I do the search [filler item], one of the top results I see is http://www.makeuseof.com/tag/top-5-amazon-filler-item-finder... which shows five different services to fill this information need, and that also has pictures. I do think that's a pretty helpful result. I mentioned filleritemfinder.com as a random example (there are many of these services), but filleritemfinder.com appears to use AJAX to keep results…

Would it be likely that if the site was to use another monetization plan that it would be less likely to be penalized e.g. advertising instead of affiliate links?

Sadly, quite true.

"True" - because my current understanding (which Matt_Cutts can elucidate on if he chooses to) is that Google has looked into - but does not currently incorporate - the presence of advertising as a spam signal.

"Sadly" - because my independent research has shown that advertising - most notably the presence of Google AdSense - is a reliable predictive variable of a page being spam.

All things being equal, a page with AdSense blocks on it is far more likely to be spam. Yet as of a few months ago, that does not appear to weigh very heavily into the equation.

Re: How My Popular Site was Banned by Google

#73
post #70

Earlier quoted context omitted.

I'm not saying that the issues aren't fixable. But when the site came up for a review, we saw things that violate our quality guidelines: autogenerated pages with hundreds of affiliate links that consist of lots of keywords, and the links/keywords are duplicate content. If he fixes the issues by e.g. blocking out the autogenerated pages, then I expect his homepage will be found in Google. The autogenerated pages are…

This is moronic. The pages are "autogenerated" because it's a web app. They have affiliate links because that MAKES TOTAL SENSE given the nature of the app ("show me as many products as possible that cost $X"). You know who else generates infinite autogenerated pages with affiliate codes attached? GOOGLE. http://www.google.com/search?q=teen+sex&tbm=shop&hl=... It's scary that Google uses "logic" like yours to make th…

The difference is Google uses robots.txt to exclude these autogenerated pages from search engine crawlers. filleritem.com didn't; now they do[1]. Quite simple really.

[1]: http://news.ycombinator.com/item?id=3100488

Re: How My Popular Site was Banned by Google

#74
post #70

Earlier quoted context omitted.

This is moronic. The pages are "autogenerated" because it's a web app. They have affiliate links because that MAKES TOTAL SENSE given the nature of the app ("show me as many products as possible that cost $X"). You know who else generates infinite autogenerated pages with affiliate codes attached? GOOGLE. http://www.google.com/search?q=teen+sex&tbm=shop&hl=... It's scary that Google uses "logic" like yours to make th…

The difference is Google uses robots.txt to exclude these autogenerated pages from search engine crawlers. filleritem.com didn't; now they do[1]. Quite simple really. [1]: http://news.ycombinator.com/item?id=3100488

No one should be banned from Google for failing to fill out robots.txt in the manner Google prefers. This is a horrible abuse of power -- and a competitor -- by Google. "Quite simple really."

Re: How My Popular Site was Banned by Google

#76
post #44

Earlier quoted context omitted.

Fair enough! Was there a way Brower could have received this clear explanation without appearing on the front page of HN?

The primary ways to get feedback are: - looking for messages in our webmaster console at google.com/webmasters - asking in our webmaster forum, also linked to from google.com/webmasters - doing a reconsideration request (also helps with confirmation of manual action) - you can talk to search engine reps at various search conferences, e.g. I'll be at PubCon next month. - calling AdWords support (only for AdWords custo…

Matt, I think the main problem (having seen what it looks and feels like on the receiving end) is that Google's solution and treatment of the site owner is, for lack of a better term, rather violent.

Put yourself in the shoes of someone who is not a scammer and is trying to do a good job. Perhaps someone how is learning the ropes. Getting hit with an unilateral, violent and --for all intents and purposes-- permanent action such as seems common in these cases can be devastating.

I'd like to relate a case that I witnessed that was truly perplexing. It involved about 200 domains that were registered with GoDaddy and placed in their "cash parking" program.

They were there for months with no issues whatsoever. One day, the domain owner realized that this cash parking service was Google AdSense with GoDaddy taking a bite of the minimal action. So...he moved all the domains to a product called "AdSense for Sites" (I don't remember the exact name). This services was marketed by Google as a place to park your domains and earn some money through advertising that Google would automagically place on these domains. Same service that had been on these domains for ages through GoDaddy.

The transfer went well and all domains were accepted. No problems. The domains go "live", if you will, and ads start showing as predicted.

Two days later the account is cancelled and the domains are taken off the program citing "suspicious activity". The irony is that the sites --and their content-- were handled entirely by Google through this "AdSense for Domains" product. No self-clicking activity took place whatsoever. No nefarious activity of any kind. This person was far too busy with real business to go around clicking on ads across 200 domains to make $0.25 at the end of the day.

This was simply a transfer to Google for the same service that Google had been providing through GoDaddy for many months, years in some cases.

There was no recourse. No way to speak to anyone. No way to even try to understand what this "suspicious activity" was all about. The account was banned, closed, done...forever.

Now, here's a person who had plans for legitimate and valuable real sites to be launched on some of these domains later on. The whole experience scared him to a point of simply rejecting the idea of doing anything with Google if he could avoid it.

We had no way to provide any kind of an argument to the contrary because of the violent and totalitarian nature of the cutoff. Who would want to do business under those conditions?

The fact of the matter is that AdWords/AdSense generated revenue could evaporate overnight and with no recourse whatsoever. That's a tough pill to swallow for anyone who is a legitimate entrepreneur looking to build value and make some money or earn a living through their efforts.

Scammers are a different matter. However, you seem to treat both groups with the same hammer which, in my humble opinion, is not right.

I now have to advise anyone we work with that any income that relies on Google for either lead generation or direct income (AdSense) has to be treated as though it could evaporate at any time and for any reason without any real opportunity given to restore it in a timely fashion. Without that caveat on the table I couldn't personally advise anyone to use your products.

In a normal business one would engage with ones vendors in order to resolve issues in mutually beneficial ways. In this case it is a one-way street with violent and severe consequences for your customers and partners. That's what you have to fix.

Re: How My Popular Site was Banned by Google

#77

You have an autogenerated web site that consists of practically nothing other than affiliate links to Amazon. You can make an infinite number of autogenerated pages on your site, e.g. http://www.filleritem.com/index.html?q=hacker+news http://www.filleritem.com/index.html?q=31.69 http://www.filleritem.com/index.html?q=teen+sex and each autogenerated page consists of literally hundreds of affiliate links stuffed with k…

As we have heard lately how Google has manipulated the Yelp search results and others for their benefit, one has to wonder -

since google has aquired some big websites like Zagat, theoretically - if one day they will just start to screw Zagat competitors , what can be actually done about it ,

i mean how can one prove that google is just attacking you and they just dont like you as their competitor ...

its relative - you can always say that your site is not original enough, or you have bad links etc.

Re: How My Popular Site was Banned by Google

#78
post #70

Earlier quoted context omitted.

I'm not saying that the issues aren't fixable. But when the site came up for a review, we saw things that violate our quality guidelines: autogenerated pages with hundreds of affiliate links that consist of lots of keywords, and the links/keywords are duplicate content. If he fixes the issues by e.g. blocking out the autogenerated pages, then I expect his homepage will be found in Google. The autogenerated pages are…

This is moronic. The pages are "autogenerated" because it's a web app. They have affiliate links because that MAKES TOTAL SENSE given the nature of the app ("show me as many products as possible that cost $X"). You know who else generates infinite autogenerated pages with affiliate codes attached? GOOGLE. http://www.google.com/search?q=teen+sex&tbm=shop&hl=... It's scary that Google uses "logic" like yours to make th…

Totally agree, Google is basically dictating how websites have to be designed now, or else.

Re: How My Popular Site was Banned by Google

#79
post #52

Earlier quoted context omitted.

Search results are also not supposed to be indexed. I believe this is actually in the guidelines.

Yup. Search for [quality guidelines] and they're at http://www.google.com/support/webmasters/bin/answer.py?answe... . The relevant part is "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines."

Matt, I tried that recently with one of my own sites, and after a few days I got a warning in my Google Webmaster Tools dashboard that the bot could not access my /search URL because it was blocked in my robots.txt, and I should take action to correct it. So I then unlocked my /search URL, which is in violation of the above guideline, but made the error in Google Webmaster Tools go away.

These conflicting messages from Google are very confusing!

Re: How My Popular Site was Banned by Google

#80

Earlier quoted context omitted.

The primary ways to get feedback are: - looking for messages in our webmaster console at google.com/webmasters - asking in our webmaster forum, also linked to from google.com/webmasters - doing a reconsideration request (also helps with confirmation of manual action) - you can talk to search engine reps at various search conferences, e.g. I'll be at PubCon next month. - calling AdWords support (only for AdWords custo…

Matt, I think the main problem (having seen what it looks and feels like on the receiving end) is that Google's solution and treatment of the site owner is, for lack of a better term, rather violent. Put yourself in the shoes of someone who is not a scammer and is trying to do a good job. Perhaps someone how is learning the ropes. Getting hit with an unilateral, violent and --for all intents and purposes-- permanent…

I take your points, and in my group we've worked hard in the last year to improve our transparency and our appeals process, e.g. see http://googlewebmastercentral.blogspot.com/2011/09/reconside... from a few weeks ago. Likewise, AdWords now has phone support for all AdWords advertisers.

I can't speak for the AdSense (for Domains) team other than to say that when they shut down an account, they think that they have good reason for it. And unfortunately, that's typically a situation where they can't give many details--if the team sees abuse, providing information about how the abuse was flagged would help spammers quite a bit.

I know that Google can seem abrupt sometimes, and I dislike that, but part of the issue is also scale. See https://plus.google.com/117377434815709898403/posts/1hRWj489... that notes that if each Google user had a single 10 minute issue every three years, that would need 20,000+ support people to handle that load. Or consider that there's 200M domain names, and all those webmasters want to talk to Google and ask questions.

Even this link is discouraging: http://www.theatlantic.com/magazine/archive/2011/11/hacked/8... It mentions that several thousand people get their Gmail account hijacked every day. Trying to support all the people who want to interact with Google in a scalable way is a really hard problem.

Post reply on HN