How to beat comment spam • Hacker News Board

Re: How to beat comment spam

#81

Post by dkokelley » Fri, Oct 12, 2012, 5:54 PM UTC

ryanwanger wrote: post #51

TomGullen wrote: post #29

We got hit with a huge wave recently, that sent over 40,000 visits a day to our site and nearly ground it to a halt. The number 1 effective thing we have found to do is to not allow hyperlinks to be posted if they are not trusted (not enough rep/point/score whatever) Overnight it basically stopped the spam wave. Your removing the one thing of value for them, a hyperlink. I'm a big fan of accessibility and this works…

Blocking any spam that contains a link is helpful, for sure, but doesn't get everything. Every few months I see waves of comments like: "Really graet article. We need more people like you in the world." Each comment has exactly one pair of transposed letters. There is no product being pitched, and no url (we don't display or link to email address either). It's baffling.

I would guess that the transposed letters are used to keep tabs on where their comments are live. It could look innocent, as if it were a human typo, but later the spammers could run a search and see which sites are trusting their comments in order to either edit them later or use their trusted account to post spam links.

Re: How to beat comment spam

#82

Post by coderdude » Fri, Oct 12, 2012, 5:56 PM UTC

ryanwanger wrote: post #51

TomGullen wrote: post #29

We got hit with a huge wave recently, that sent over 40,000 visits a day to our site and nearly ground it to a halt. The number 1 effective thing we have found to do is to not allow hyperlinks to be posted if they are not trusted (not enough rep/point/score whatever) Overnight it basically stopped the spam wave. Your removing the one thing of value for them, a hyperlink. I'm a big fan of accessibility and this works…

Blocking any spam that contains a link is helpful, for sure, but doesn't get everything. Every few months I see waves of comments like: "Really graet article. We need more people like you in the world." Each comment has exactly one pair of transposed letters. There is no product being pitched, and no url (we don't display or link to email address either). It's baffling.

Those comments could be used to make automated spam filters less effective. Spammers could post comments that would normally be labeled as spam, but do not contain any URLs. Over time a spam filter would have a harder time distinguishing between cut-and-dry spam and real comments (assuming the admin is marking those spam comments as ham).

Also, I'm a fan of not allowing brand new accounts to post URLs in their comments. It's a no-brainer.

Re: How to beat comment spam

#83

Post by sedachv » Fri, Oct 12, 2012, 6:10 PM UTC

ryanwanger wrote: post #51

TomGullen wrote: post #29

We got hit with a huge wave recently, that sent over 40,000 visits a day to our site and nearly ground it to a halt. The number 1 effective thing we have found to do is to not allow hyperlinks to be posted if they are not trusted (not enough rep/point/score whatever) Overnight it basically stopped the spam wave. Your removing the one thing of value for them, a hyperlink. I'm a big fan of accessibility and this works…

Blocking any spam that contains a link is helpful, for sure, but doesn't get everything. Every few months I see waves of comments like: "Really graet article. We need more people like you in the world." Each comment has exactly one pair of transposed letters. There is no product being pitched, and no url (we don't display or link to email address either). It's baffling.

> Each comment has exactly one pair of transposed letters. There is no product being pitched, and no url (we don't display or link to email address either). It's baffling.

Sounds like they're doing what's known as "Bayesian poisoning" (http://en.wikipedia.org/wiki/Bayesian_poisoning) ahead of time to open the door for later link spamming.

Re: How to beat comment spam

#84

Post by theoj » Fri, Oct 12, 2012, 6:16 PM UTC

reasonattlm wrote: post #15

I've used this sort of Javascript-based approach for years, with great success: http://www.exratione.com/2010/12/how-to-block-999-of-all-mov... It works very well unless you're big enough to merit individual attention from a spammer. It's not rocket science - it just raises the bar a little above the level of effort that people who spam everything, everywhere are willing to put in. That might change. The real merit o…

There is a Wordpress plugin called Spam Free Wordpress that implements a variation of this and has effectively cut spam on my sites to zero.

The plugin improves on the method described by randomly generating the value of the additional token parameter, and keeping a list of all generated tokens. If the server receives a comment post request which does not contain one of the generated tokens, then that comment is guaranteed to be automated spam.

Re: How to beat comment spam

#85

Post by infinity » Fri, Oct 12, 2012, 6:50 PM UTC

eli wrote: post #57

gabemart wrote: post #19

Earlier quoted context omitted.

I wonder if eventually people will just stop allowing hyperlinks in comments altogether. It would, at a stroke, eliminate the biggest incentive for spam. Yes, it's nice (I guess) when someone's name is a link to their personal website or they can post the URL of a relevant article in the comments, but it's not like commenting ceases to be valuable without those features.

I thought that would happen as more people added rel=nofollow to links in comments... hasn't happened yet though.

A real spammer will take any link, it doesn't matter if the link won't be considered as some form of endorsement by search engines due to the use of the rel=nofollow attribute. A spammer will happily post a million links, there will be some poor souls out there and click on some of them. Quantity over quality has always been one characteristic trait of spam.

Spam and link spam were already there before Google existed and the PageRank was invented. The index of the AltaVista search engine was huge and full of spam.

When the nofollow value for the rel attribute was introduced there were many claims that this would reduce the amount of link and comment spam. Critical remarks came often from people who were offering link building and SEO as a service.

Re: How to beat comment spam

#86

Post by Isofarro » Fri, Oct 12, 2012, 7:37 PM UTC

vidarh wrote: post #39

whyhellothere wrote: post #17

Unfortunately (at least in the UK) this technique cannot be used on consumer facing sites as it breaks the accessibility of the form for some disabled users. For personal sites it really comes down to your preferences. Personally I would prefer that everyone was able to comment, however if it stops you having to wade through thousands of spam messages every day I can see the point of using it.

Why would this be an accessibility problem? I don't see why screen readers would have a problem dealing with it - for them the form in the users browser will appear just the same as it otherwise would.

1.) Screenreaders have different modes of operation for different aspects of web content. For dealing with Forms they have Forms mode, in which only form elements are announced. A link isn't a form element, so they wont see the submit button.

2.) Screenreader users have a shortcut key to submit the form - typically when under-qualified web developers create forms without submit buttons. This fires the form submit event, which without a JavaScript preventDefault will get the form contents sent to the URL mentioned in the action attribute on the form. So the screen reader user's comment is treated as spam.

Re: How to beat comment spam

#87

Post by jasondc » Fri, Oct 12, 2012, 7:40 PM UTC

Wait until you get the SPAM bots targeting your payment forms to validate stolen credit cards...whole different set of challenges. We had 800 payments in one day from this type of attack.

Re: How to beat comment spam

#88

Post by kidh0 » Fri, Oct 12, 2012, 7:46 PM UTC

daveid wrote: post #3

Another technique I find to be working really well is the "honeypot" technique. I create a CSS-hidden input field with a delicious, attractive name "url" and then validate it to be empty.

I use this technique a long time and it seems to beat all bots (a medium website, about 100k unique visitors per day). It's easy, unobtrusive and just works :)

Re: How to beat comment spam

#89

Post by hashtree » Fri, Oct 12, 2012, 8:11 PM UTC

One thing that really helps:

Server-side, encrypt a token which, including representing the unique form instance, contains a tick count and set a hidden input's value to it. Now, ensure that each form instance cannot be submitted more than once AND that the delta between the current tick count and the form's tick count is greater than or equal to the amount of time that would be need for a human to fill out the form.

You MUST ensure client-side error detection is superb (as you want to catch all errors prior to submitting), handle for back button usage properly (browser caching directives, http status codes, etc), and ensure you handle for browsers which may auto fill information in for the user.

You would be surprised just how many bots come in and either used a cached form or immediately submit it. Assuming they are smart enough to bypass both of these, you just reduced the number of times they could potentially spam you dramatically.

The tick count figure needs to be done on a form by form basis, as each one likely has a different minimum.

Re: How to beat comment spam

#90

Post by seanalltogether » Fri, Oct 12, 2012, 8:14 PM UTC

tomwalsham wrote: post #53

My personal favourite quick-fix (which doesn't stand up to targeted attacks, but is a very effective band-aid), is to put the following : Then disallow any form submissions server-side which contain a value for 'website'. Automated bots can't resist filling out that field.

Mine is the reverse of this idea. I have an hidden field that when you click submit, I fill in with a token via javascript. If the correct token isn't present when submitting, i reject the comment.