Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How My Popular Site was Banned by Google

kbrower.posterous.com

Post reply on HN
Print view View original

31–40 of 132 posts

Re: How My Popular Site was Banned by Google

#31

Post by kevindication »

mc32 wrote: post #26
dangrossman wrote: post #23

Earlier quoted context omitted.

A filler item is something with a specific (low) price used to make an order near the free shipping minimum reach that minimum. A search for the words "filler item" is not at all related and not at all helpful.

Sorry, try this instead: http://www.amazon.com/Shipping-Filler-items-under-shipping/l...

That's an Amazon user maintained list of low price items. The purpose of the filler item site is to answer the question: "What can I buy for $0.59?" That list doesn't answer that question.

Re: How My Popular Site was Banned by Google

#32

Post by amitparikh »

mc32 wrote: post #26
dangrossman wrote: post #23

Earlier quoted context omitted.

A filler item is something with a specific (low) price used to make an order near the free shipping minimum reach that minimum. A search for the words "filler item" is not at all related and not at all helpful.

Sorry, try this instead: http://www.amazon.com/Shipping-Filler-items-under-shipping/l...

Oh c'mon, now you're just submitting links blindly -- that's definitely not an official Amazon offering. That list is generated & curated by Amazon user "fargowifeandmom" and was last updated at the beginning of this year.

Re: How My Popular Site was Banned by Google

#33

Post by nirvana »

Matt_Cutts wrote: post #22
kbrower wrote: post #17

Earlier quoted context omitted.

Hi Matt, Thanks for responding. The point of the site is to find items of a particular price that qualify for free shipping on Amazon.com. If you want I will give you access to my google analytics to show you that this is a site people want and use. I guess I should not let people link to the search results page? http://www.filleritem.com/index.html?q=hacker+news and http://www.filleritem.com/index.html?q=anything+th…

My advice would be to use robots.txt to block out the autogenerated pages; when users search for e.g. a long-tail phrase and then land on a page that's nothing but affiliate links with a lot of keywords, they tend to complain to us. Users would be happier if they landed on the root page of your site or the root page of http://www.superfillers.com/ or http://www.filleritemfinder.com/ than if they landed on a deep page…

http://www.filleritemfinder.com/ is the first hit for "filler item" on google right now.

A search for hacker news results in a page full of affiliate links, just as the example you gave above. Only difference is that they didn't re-write the URL.

filleritemfinder.com has no robots.txt that I was able to pull up.

So, filleritem.com, a google customer, was blocked, but filleritemfinder.com, doing the same thing, is the number one result.

Further, shouldn't this kind of advice be given to people who appeal being excluded from the index? Or should we all post to Hacker News when it happens to us so that you can come explain directly?

I think %50 of the problem is the arbitrary picking of sites to block (and it's not working, btw[1]) and %50 of it is that google seems uninterested in explaining or advising people when it happens to them.

[1] Been buying gear for a project lately, and so doing a lot of google searches in the form of product-model-number review or product-name review. Overwhelmed with spam sites, and mindless human generated spam sites like dpreview.com, etc.

Re: How My Popular Site was Banned by Google

#34

Post by infinity »

coffee wrote: post #25

Yea that sucks, it's happened to me MANY times. The fact of the matter is, it's Googles world and we're just living in it...

I don't think that it is as simple as that. There is some form of ecosystem on the web: there are the content creators and website providers, there are the searchers, and there are the search engine providers. Each of these has its own interests and guidelines.

Searchers have quality guidelines too, though they are not explicitly published by them on the web. If a search engine delivers low-quality results, the searchers move on to the next search engine. A search engine without searchers is hardly profitable.

Most of the quality guidelines of Google are just common sense, following them will in many cases increase the usability and findability of a site.

If this has happend to you many times, you are likely doing something wrong. I have never had one of my sites banned from Google.

Re: How My Popular Site was Banned by Google

#35

Post by chime »

droithomme wrote: post #29
Matt_Cutts wrote: post #15

You have an autogenerated web site that consists of practically nothing other than affiliate links to Amazon. You can make an infinite number of autogenerated pages on your site, e.g. http://www.filleritem.com/index.html?q=hacker+news http://www.filleritem.com/index.html?q=31.69 http://www.filleritem.com/index.html?q=teen+sex and each autogenerated page consists of literally hundreds of affiliate links stuffed with k…

I have no association with this person and just heard about this. But obviously those are search term result pages and not "autogenerated pages" in the usual sense. If Google is going to apply such a standard uniformly, they'd have to ban all sites that have search functions. Autogenerated in the bad sense is link farms like fakesite.com/buy_drugwiththisname_now.html, with drugwiththisname replaced with 500,000 diffe…

> Autogenerated in the bad sense is link farms like fakesite.com/buy_drugwiththisname_now.html, with drugwiththisname replaced with 500,000 different possibilities

Actually there is no difference between search page result sites like the OP and what you said. Having a URL that ends in .html does not mean there is a static html file on the server. For examples, nextag and pricegrabber have URLs like blah.com/digital/Canon-EOS-7D-SLR-Digital-Body/m739295014.html. Scroll around these sites and you'll see they're anything but static.

Each page is simply a result of a query. Whether the database is local (like nextag) or remote (like Amazon API query like OP) is inconsequential. Personally, I would be VERY happy if Google hid/ignored these kind of search-query sites. I have not once found any of these sites useful. Problem is the large gray area in which these sites operate and what I find useless, someone else might find useful.

Re: How My Popular Site was Banned by Google

#36

Post by Matt_Cutts »

gojomo wrote: post #28
Matt_Cutts wrote: post #15

You have an autogenerated web site that consists of practically nothing other than affiliate links to Amazon. You can make an infinite number of autogenerated pages on your site, e.g. http://www.filleritem.com/index.html?q=hacker+news http://www.filleritem.com/index.html?q=31.69 http://www.filleritem.com/index.html?q=teen+sex and each autogenerated page consists of literally hundreds of affiliate links stuffed with k…

Still, when used as intended (with the query a dollar amount, like your 31.69 example), isn't his site useful ? (I've not noticed a similarly straightforward tool for this same purpose at Amazon.com itself.) The non-dollar-amount queries seem like a bug -- triggering a default of listing up from 0.00, unrelated to the query. If this were an intentional manipulative tactic, wouldn't the outlinks be related to the quer…

I'm not saying that the issues aren't fixable. But when the site came up for a review, we saw things that violate our quality guidelines: autogenerated pages with hundreds of affiliate links that consist of lots of keywords, and the links/keywords are duplicate content. If he fixes the issues by e.g. blocking out the autogenerated pages, then I expect his homepage will be found in Google. The autogenerated pages are also less useful to someone landing on a page because the pages quickly get stale: prices change, things go in/out of stock, etc.

Re: How My Popular Site was Banned by Google

#37

Post by StrawberryFrog »

chime wrote: post #35
droithomme wrote: post #29

Earlier quoted context omitted.

I have no association with this person and just heard about this. But obviously those are search term result pages and not "autogenerated pages" in the usual sense. If Google is going to apply such a standard uniformly, they'd have to ban all sites that have search functions. Autogenerated in the bad sense is link farms like fakesite.com/buy_drugwiththisname_now.html, with drugwiththisname replaced with 500,000 diffe…

> Autogenerated in the bad sense is link farms like fakesite.com/buy_drugwiththisname_now.html, with drugwiththisname replaced with 500,000 different possibilities Actually there is no difference between search page result sites like the OP and what you said. Having a URL that ends in .html does not mean there is a static html file on the server. For examples, nextag and pricegrabber have URLs like blah.com/digital/C…

> Actually there is no difference between search page result sites like the OP and what you said

Uh, yes there is. http://www.somesite.com/index.html?q=foo is obviously a search (right down to the choice if q for query), but http://www.somesite.com/buy_foo_now.html does not look like one.

You might say that technically they could have the same back-end, but the web is flexible - just about any URL scheme could resolve to the same back-end. The difference is that the first is honest and the second one goes out of its way to hide what it does.

Re: How My Popular Site was Banned by Google

#38

Post by josefresco »

Amazon probably complained to Google and Google despite already having the rules on the books decided to enforce it on this one site, while also keeping many other low traffic clone sites offering the same service alone. Call me paranoid, or cynical but I've seen much worse from Google on the AdWords side of their offerings.

Re: How My Popular Site was Banned by Google

#39

Post by Matt_Cutts »

nirvana wrote: post #33
Matt_Cutts wrote: post #22

Earlier quoted context omitted.

My advice would be to use robots.txt to block out the autogenerated pages; when users search for e.g. a long-tail phrase and then land on a page that's nothing but affiliate links with a lot of keywords, they tend to complain to us. Users would be happier if they landed on the root page of your site or the root page of http://www.superfillers.com/ or http://www.filleritemfinder.com/ than if they landed on a deep page…

http://www.filleritemfinder.com/ is the first hit for "filler item" on google right now. A search for hacker news results in a page full of affiliate links, just as the example you gave above. Only difference is that they didn't re-write the URL. filleritemfinder.com has no robots.txt that I was able to pull up. So, filleritem.com, a google customer, was blocked, but filleritemfinder.com, doing the same thing, is the…

When I do the search [filler item], one of the top results I see is http://www.makeuseof.com/tag/top-5-amazon-filler-item-finder... which shows five different services to fill this information need, and that also has pictures. I do think that's a pretty helpful result.

I mentioned filleritemfinder.com as a random example (there are many of these services), but filleritemfinder.com appears to use AJAX to keep results on the same page rather than making a new url.

"filleritem.com, a google customer, was blocked, but filleritemfinder.com, doing the same thing, is the number one result."

The filleritemfinder.com site is not doing the same thing, because it's not generating fresh urls for every possible search. But you're not really suggesting that we should treat advertising customers somehow differently in our search results, are you? The webspam team takes action regardless of whether someone is an advertising customer or not.

"shouldn't this kind of advice be given to people who appeal being excluded from the index?"

This advice is available to everyone in our quality guidelines. It sounds like the site owner reached out to the AdWords team, which gave him clear guidance that the site violated the ads policy on bridge pages. It sounds like the site owner also filed a reconsideration request, and we replied to let the site owner know that the reconsideration request was rejected because it was still in violation of our policies. It doesn't look like the site owner stopped by our webmaster support forum, at least that I could see in a quick look. At that point, the site owner did a blog post and submitted the post to Hacker News, where I was happy to reply.

Re: How My Popular Site was Banned by Google

#40

Post by kbrower »

Matt_Cutts wrote: post #22
kbrower wrote: post #17

Earlier quoted context omitted.

Hi Matt, Thanks for responding. The point of the site is to find items of a particular price that qualify for free shipping on Amazon.com. If you want I will give you access to my google analytics to show you that this is a site people want and use. I guess I should not let people link to the search results page? http://www.filleritem.com/index.html?q=hacker+news and http://www.filleritem.com/index.html?q=anything+th…

My advice would be to use robots.txt to block out the autogenerated pages; when users search for e.g. a long-tail phrase and then land on a page that's nothing but affiliate links with a lot of keywords, they tend to complain to us. Users would be happier if they landed on the root page of your site or the root page of http://www.superfillers.com/ or http://www.filleritemfinder.com/ than if they landed on a deep page…

Ok, well I am willing to give that a try. Just implemented. Thanks Matt!

http://filleritem.com/robots.txt

Post reply on HN