Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 4 down vote favorite

I've been thinking a lot lately about the need for better form security, and good ways to accomplish that.

We currently use captcha codes to screen for bots, but that's annoying to users and may not work forever.

I think that we need a more intuitive, organic system for screening bad comments/contact form submissions.

One option that has come to mind would be trying to screen comments for things that are obviously not words, in addition to screening for duplicate comments. E.g., when a spammer on Facebook, Twitter, or a comments section is stopped from just posting the same thing a lot of times, they add garbled letters and or numbers somewhere in the post to make it "unique".

If it was possible to screen out obvious not-text, this could be overcome. If you could go a step further, and screen out posts that obviously have words placed in for no reason except to make the post "unique", you could force the spammer/scam artist down to only use repetitive post options which actually make gramatical sense.

At the very least, you could have posts be flagged for moderator attention if they looked similar but which just had random garbage added in for no reason. This would significantly reduce a spammer's ability to keep spamming, even across multiple accounts.

Could it be possible to screen form field results for random word and number combinations, and words thrown in just to make a post "unique"?

share|improve this question
This is already is use for email and is already circumvented by having the body of the email be a lengthy quote from a public domain document. Your Viagara spam ad includes a lengthy quote from the Bible, for example, to fool the filters. – S.Lott Feb 20 '12 at 19:53

2 Answers

up vote 3 down vote accepted

You can borrow various Bayesian filters from email apps. There are some open source ones around that you can learn from and build your own implementation. The problem is that asute spammers use the same code to build their text generation apps to avoid the filters.

The best way I've found to prevent most automated spamming and to avoid the user experience problems with CAPTCHA is to have an AJAX validation method that will thwart bots and degrade gracefully to use server side code for non-Javascript users. This won't help with paid manual spamming where somebody in a third world Internet cafe gets paid $5 to put a link on X number of sites though.

What I've found is that combining the script on the front end with automated moderation on the backend (flagging certains words and phrases, posts with links and so forth) stops about 99.9% of spam attempts. The only ones that get through are ones that are well done manual ones and those are quite rare.

share|improve this answer
3  
The 'spammers use the same code' argument is only partially valid - the value of a Bayesian spam filter lies in both the algorithm and the training data. Spammers can use the algorithm, but they (hopefully) don't have access to your training data (i.e., hand-categorized text samples from the actual production site). As long as your training data is significantly better than the spammers', you'll still get decent results. – tdammers Feb 21 '12 at 6:49
@tdammers - Correct. I should have better emphasized the need to customize and to not use just the default training data. Having good training data is key to a good Bayesian system. – jfrankcarr Feb 21 '12 at 11:59
up vote 0 down vote

A problem with this: Acronyms. Sometimes people post non-words without them being gibberish. I asked the IRS if I could roll my 401k over to a Roth IRA. Three non-words in one sentence without it being filter-bait.

share|improve this answer
Bayesian spam filters are quite immune to such problems. If you train them well, they will not fire on such sentences, provided that your training data also contains passages with lots of acronyms in them. – tdammers Feb 21 '12 at 6:47
@tdammers: Note that he asked about checking for non-words. That's what I was replying to. – Loren Pechtel Feb 22 '12 at 0:37
Adaptive Bayesian spam filters can be trained to distinguish between words and non-words just as easily. The filter itself doesn't care what meaning you assign to its categories, it just sees training data and extracts common statistical properties. – tdammers Feb 22 '12 at 10:16
@tdammers: jfrankcarr mentioned Bayesian filters, the question did not. I was addressing the question. – Loren Pechtel Feb 22 '12 at 21:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.