Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've been reading about Bayesian spam filtering and I think I understand the theory, but I just don't see why this approach is needed in order to calculate the likelihood of a message being spam, given that it contains a certain word.

If we have a set of messages already classified by the user as either 'spam' or 'ham' and we receive a new message (containing the chosen word) which we want to classify, then surely all we need to do is divide the number of spam messages that contain the word, by the total number of messages that contain the word... Why all the equations?

share|improve this question
Post this on CrossValidated - your probability of getting answers is more. You could also know how Bayesian filters work – Ubermensch Jan 17 '12 at 9:38
“all the equations” … there is exactly one equation involved in a Bayesian classifier, and it formalises almost precisely what you’ve said in your second paragraph (only doing it correctly to account for prior probability). – Konrad Rudolph Jan 17 '12 at 11:56
@Ubermensch - Do you mean CrossValidated? *8') – Mark Booth Jan 17 '12 at 15:37
up vote 15 down vote accepted

Alright, first off, there is not only positive evidence but also negative evidence. Some words make an email message very likely to be spam, some make it very likely to be real. Other words make a message very likely to be spam by their absence, while yet others have the opposite effect. For instance, if you research drosophila for a living and frequently correspond with colleagues about them, the presence of that term is almost like a password, because no mass-mail campaign will be able to customize their texts to your habits that well - it would destroy the economies of scale that make spam viable in the first place.

Also, performance of a filter cannot be measured with just one metric. Detecting spam is very easy, indeed trivial, if you simply classify everything as spam - but then the false positives (detecting real mail as spam) are intolerably high. Detecting nothing solves that problem, but then the false negatives (classifying bad samples as good) make your life miserable. A good filter must reach good values on both counts, which makes it much more complicated than just a super-sensitive detector of something.

So right from the start you have not just one 'List of bad words', but at least four lists, and not just one criterion, but at least two. So far, Bayesian filtering really is the simplest method that does this well. If you find better one, by all means let's hear it.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.