Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How My Popular Site was Banned by Google

kbrower.posterous.com

Post reply on HN
Print view View original

61–70 of 132 posts

Re: How My Popular Site was Banned by Google

#61

Post by OpenAlgorithm »

Matt_Cutts wrote: post #55
OpenAlgorithm wrote: post #41

Earlier quoted context omitted.

Would it be likely that if the site was to use another monetization plan that it would be less likely to be penalized e.g. advertising instead of affiliate links?

I recently talked for about a minute about the topic of "too much advertising" that sometimes drowns out the content of a page. It was in this question and answer session that we streamed live on YouTube: http://www.youtube.com/watch?v=R7Yv6DzHBvE#t=19m25s . The short answer is that we have been looking more at this. We care when a page creates a low-quality or frustrating user experience, regardless of whether it's…

Thanks Matt, took a look at the video, that answers my original question.

Also is there a preferred monetization model, e.g. do Google think advertisements are more or less harmful to the user experience than affiliate links, sponsored posts, etc?

Obviously across different models you can't just track space taken up, so is there some kind of metric that tracks the rate of content diluting via monetization?

Re: How My Popular Site was Banned by Google

#63

Post by dquigley »

Matt_Cutts wrote: post #51
gojomo wrote: post #44

Earlier quoted context omitted.

Fair enough! Was there a way Brower could have received this clear explanation without appearing on the front page of HN?

The primary ways to get feedback are: - looking for messages in our webmaster console at google.com/webmasters - asking in our webmaster forum, also linked to from google.com/webmasters - doing a reconsideration request (also helps with confirmation of manual action) - you can talk to search engine reps at various search conferences, e.g. I'll be at PubCon next month. - calling AdWords support (only for AdWords custo…

Matt, Would it be possible/financially practically for Google to offer a premium support option that would pay for itself?

Figure out how much it costs to respond to someone, offer that as a flat fee with a clear indication, that paying only gets you a consult on what's wrong with your site, and not any promise that it will get you reinstated. Basically what you did in your comments above.

I would personally make it expensive to make sure people don't try continually working the system by paying for the consult to get inside information about how to rank higher on Google.

I work in a support center, so I definitely recognize that it's expensive and difficult to do well at scale, so I appreciate that you're looking into how to do it!

Re: How My Popular Site was Banned by Google

#65

Post by option1138 »

Matt_Cutts wrote: post #51
gojomo wrote: post #44

Earlier quoted context omitted.

Fair enough! Was there a way Brower could have received this clear explanation without appearing on the front page of HN?

The primary ways to get feedback are: - looking for messages in our webmaster console at google.com/webmasters - asking in our webmaster forum, also linked to from google.com/webmasters - doing a reconsideration request (also helps with confirmation of manual action) - you can talk to search engine reps at various search conferences, e.g. I'll be at PubCon next month. - calling AdWords support (only for AdWords custo…

Hi Matt,

We don't know each other but I think we know of each other. I'm rather immersed in webspam detection and found this incredibly interesting.

You imply that the challenge is finding a solution that scales. Yet it sounds to me from your response that this site was flagged via manual review. Did I misunderstand?

If I heard you correctly, then is manual review a significant equation in the webspam detection methodology? You guys are boiling the ocean so I find that rather hard to swallow.

The more likely conclusion I can draw is that he had a significant number of (auto-generated) pages on his site flagged as spam and that in turn raised some eyebrows.

BTW, you and your team are doing some amazing work. I wish the paid side was up to the standards you set.

Re: How My Popular Site was Banned by Google

#66

Post by kposehn »

Matt_Cutts wrote: post #51
gojomo wrote: post #44

Earlier quoted context omitted.

Fair enough! Was there a way Brower could have received this clear explanation without appearing on the front page of HN?

The primary ways to get feedback are: - looking for messages in our webmaster console at google.com/webmasters - asking in our webmaster forum, also linked to from google.com/webmasters - doing a reconsideration request (also helps with confirmation of manual action) - you can talk to search engine reps at various search conferences, e.g. I'll be at PubCon next month. - calling AdWords support (only for AdWords custo…

Hi Matt, thanks for chiming on on this thread.

From your reply, I understand that your primary issue is indeed with scaling any manner of support issues. As I understand it, the current thinking at Google is to seek automated solutions over technical to reduce costs and improve issue volume handling. With that in mind, is there a system of triggers that can flag issues for manual review or intervention instead of being handled completely automatically?

I think the majority of issues that have people up in arms aren't really that big, but there are the exceptions. Instead of monitoring G+/Twitter/etc., there could instead be a series of internal checks that look for particular criteria:

1. Age of campaign/ad - was it super old and not really applicable or not even running much budget? It may be worth a lighter hand than a complete ban because it isn't indicative of a trend or pattern on the part of the advertiser.

2. Amount of impressions/clicks/spend over time - what is the overall severity based on actual impact? Are users actively clicking on the ad, returning to the site and continuing to use it? As each click and therefore organic link is logged, it would be good to cross reference them and go "well, this site raises flags but has high return-user rates and decent sentiment in organic results"

3. Traffic vs. Content ratio - does the site have fairly thin content that still gets dramatic numbers of users? Tying in with #2, it basically can help tell if a site is offering something thin on content but highly unique and/or valuable. If people are using it, they may be on to something that the QS algorithm misses.

The one thing I'd love to see is just a little bit more of a verbose notification or warning for the people that get hit. While many only have themselves to blame, many want to make sure they stay on the right side and any chance they have to do so is of great benefit. Then again, it is also important to not give to much information to the black-hatters that just make use of it to get even better at gaming the system.

These are just thoughts, but it has long been on my mind considering the metric ass-ton I've run through AdWords for myself and clients over the years. Hope this helps spark a few ideas :)

Re: How My Popular Site was Banned by Google

#67

Post by option1138 »

nirvana wrote: post #33
Matt_Cutts wrote: post #22

Earlier quoted context omitted.

My advice would be to use robots.txt to block out the autogenerated pages; when users search for e.g. a long-tail phrase and then land on a page that's nothing but affiliate links with a lot of keywords, they tend to complain to us. Users would be happier if they landed on the root page of your site or the root page of http://www.superfillers.com/ or http://www.filleritemfinder.com/ than if they landed on a deep page…

http://www.filleritemfinder.com/ is the first hit for "filler item" on google right now. A search for hacker news results in a page full of affiliate links, just as the example you gave above. Only difference is that they didn't re-write the URL. filleritemfinder.com has no robots.txt that I was able to pull up. So, filleritem.com, a google customer, was blocked, but filleritemfinder.com, doing the same thing, is the…

I think %50 of the problem is the arbitrary picking of sites to block (and it's not working, btw[1]) and %50 of it is that google seems uninterested in explaining or advising people when it happens to them.

This. Assume you work on the webspam team and you have a 92% spam detection rate but a 99.9% accuracy rate on what you do detect.

There are around 40,000,000 active domains in a given month listed on Google. That means 40,000 sites on average are being penalized without reason.

Re: How My Popular Site was Banned by Google

#68

Post by learc83 »

For better or worse Google is the gateway to the internet. If your site isn't on Google it is virtually invisible to most of the world.

Since Google has such an enormous position of power, they are really going to have to do better with customer service (and by customer service, I mean providing a way to talk to a human on their end).

I understand it's expensive, but if it doesn't get any better, eventually some government somewhere (that has jurisdiction) is going to regulate them.

Re: How My Popular Site was Banned by Google

#69

Post by dlitz »

thenextcorner wrote: post #56
joshu wrote: post #52

Earlier quoted context omitted.

Search results are also not supposed to be indexed. I believe this is actually in the guidelines.

Literally Google Webmaster Guidelines say: "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines." So a couple of questions: - What is of value for a user? - Who is determining the value for the users in these cases? As always, it's not always quite clear on what treatment you should use for search pages!

I'm pretty sure that having different search engines indexing each other would be bad. Don't cross the streams.

Re: How My Popular Site was Banned by Google

#70

Post by mapgrep »

Matt_Cutts wrote: post #36
gojomo wrote: post #28

Earlier quoted context omitted.

Still, when used as intended (with the query a dollar amount, like your 31.69 example), isn't his site useful ? (I've not noticed a similarly straightforward tool for this same purpose at Amazon.com itself.) The non-dollar-amount queries seem like a bug -- triggering a default of listing up from 0.00, unrelated to the query. If this were an intentional manipulative tactic, wouldn't the outlinks be related to the quer…

I'm not saying that the issues aren't fixable. But when the site came up for a review, we saw things that violate our quality guidelines: autogenerated pages with hundreds of affiliate links that consist of lots of keywords, and the links/keywords are duplicate content. If he fixes the issues by e.g. blocking out the autogenerated pages, then I expect his homepage will be found in Google. The autogenerated pages are…

This is moronic. The pages are "autogenerated" because it's a web app. They have affiliate links because that MAKES TOTAL SENSE given the nature of the app ("show me as many products as possible that cost $X").

You know who else generates infinite autogenerated pages with affiliate codes attached? GOOGLE. http://www.google.com/search?q=teen+sex&tbm=shop&hl=...

It's scary that Google uses "logic" like yours to make these decisions. No: It's terrifying.

Post reply on HN