E-mail filtering
Encyclopedia
Email filtering is the processing of email
to organize it according to specified criteria. Most often this refers to the automatic processing of incoming messages, but the term also applies to the intervention of human intelligence in addition to anti-spam techniques, and to outgoing emails as well as those being received.
Email filtering software inputs email. For its output, it might pass the message through unchanged for delivery to the user's mailbox, redirect the message for delivery elsewhere, or even throw the message away. Some mail filters are able to edit messages during processing.
es. A less common use is to inspect outgoing email
at some companies to ensure that employees comply with appropriate laws. Users might also employ a mail filter to prioritize messages, and to sort them into folders based on subject matter or other criteria.
, either as separate programs (see links below), or as part of their email program (email client). In email programs, users can make personal, "manual" filters that then automatically filter mail according to the chosen criteria. Most email programs now also have an automatic spam filtering function. Internet service provider
s can also install mail filters in their mail transfer agent
s as a service to all of their customers. Due to the growing threat of fraudulent websites Internet service provider
s filter URLs in email messages to remove the threat before users click. Corporations often use filters to protect their employees and their information technology
assets.
. Outbound email filtering involves the reverse - scanning email messages from local users before any potentially harmful messages can be delivered to others on the Internet. One method of outbound email filtering that is commonly used by ISPs is transparent SMTP proxying
, in which email traffic is intercepted and filtered via a transparent proxy within the network. Outbound filtering can also take place in an email server. Many corporations employ data leak prevention technology in their outbound mail servers to prevent the leakage of sensitive information via email.
. Other times, keywords in the message body are used, or perhaps the email address of the sender of the message. Some more advanced filters, particularly anti-spam filters, use statistical document classification
techniques such as the naive Bayes classifier
. Image filtering can also be used that use complex image analysis algorithms to detect skin-tones and specific body shapes normally associated with adult-images (pornographic images).
Email
Electronic mail, commonly known as email or e-mail, is a method of exchanging digital messages from an author to one or more recipients. Modern email operates across the Internet or other computer networks. Some early email systems required that the author and the recipient both be online at the...
to organize it according to specified criteria. Most often this refers to the automatic processing of incoming messages, but the term also applies to the intervention of human intelligence in addition to anti-spam techniques, and to outgoing emails as well as those being received.
Email filtering software inputs email. For its output, it might pass the message through unchanged for delivery to the user's mailbox, redirect the message for delivery elsewhere, or even throw the message away. Some mail filters are able to edit messages during processing.
Motivation
Common uses for mail filters include organizing incoming email and removal of spam and computer virusComputer virus
A computer virus is a computer program that can replicate itself and spread from one computer to another. The term "virus" is also commonly but erroneously used to refer to other types of malware, including but not limited to adware and spyware programs that do not have the reproductive ability...
es. A less common use is to inspect outgoing email
Employee monitoring
Employee monitoring refers to any method of tracking what an employee does while at work. This may include the use of video cameras, keystroke logging, email filters, or even just watching or listening to the employee. As a result of increased technology, this has become a very important issue in...
at some companies to ensure that employees comply with appropriate laws. Users might also employ a mail filter to prioritize messages, and to sort them into folders based on subject matter or other criteria.
Methods
Mail filters can be installed by the userUser (computing)
A user is an agent, either a human agent or software agent, who uses a computer or network service. A user often has a user account and is identified by a username , screen name , nickname , or handle, which is derived from the identical Citizen's Band radio term.Users are...
, either as separate programs (see links below), or as part of their email program (email client). In email programs, users can make personal, "manual" filters that then automatically filter mail according to the chosen criteria. Most email programs now also have an automatic spam filtering function. Internet service provider
Internet service provider
An Internet service provider is a company that provides access to the Internet. Access ISPs directly connect customers to the Internet using copper wires, wireless or fiber-optic connections. Hosting ISPs lease server space for smaller businesses and host other people servers...
s can also install mail filters in their mail transfer agent
Mail transfer agent
Within Internet message handling services , a message transfer agent or mail transfer agent or mail relay is software that transfers electronic mail messages from one computer to another using a client–server application architecture...
s as a service to all of their customers. Due to the growing threat of fraudulent websites Internet service provider
Internet service provider
An Internet service provider is a company that provides access to the Internet. Access ISPs directly connect customers to the Internet using copper wires, wireless or fiber-optic connections. Hosting ISPs lease server space for smaller businesses and host other people servers...
s filter URLs in email messages to remove the threat before users click. Corporations often use filters to protect their employees and their information technology
Information technology
Information technology is the acquisition, processing, storage and dissemination of vocal, pictorial, textual and numerical information by a microelectronics-based combination of computing and telecommunications...
assets.
Inbound and Outbound Filtering
Mail filters can operate on inbound and outbound email traffic. Inbound email filtering involves scanning messages from the Internet addressed to users protected by the filtering system or for lawful interceptionLawful interception
Lawful interception is obtaining communications network data pursuant to lawful authority for the purpose of analysis or evidence. Such data generally consist of signalling or network management information or, in fewer instances, the content of the communications...
. Outbound email filtering involves the reverse - scanning email messages from local users before any potentially harmful messages can be delivered to others on the Internet. One method of outbound email filtering that is commonly used by ISPs is transparent SMTP proxying
Transparent SMTP proxy
SMTP proxies are specialized Mail Transfer Agents that, similar to other types of proxy servers, pass SMTP sessions through to other MTAs without using the store-and-forward approach of a typical MTA. When an SMTP proxy receives a connection, it initiates another SMTP session to a destination MTA...
, in which email traffic is intercepted and filtered via a transparent proxy within the network. Outbound filtering can also take place in an email server. Many corporations employ data leak prevention technology in their outbound mail servers to prevent the leakage of sensitive information via email.
Customization
Mail filters have varying degrees of configurability. Sometimes they make decisions based on matching a regular expressionRegular expression
In computing, a regular expression provides a concise and flexible means for "matching" strings of text, such as particular characters, words, or patterns of characters. Abbreviations for "regular expression" include "regex" and "regexp"...
. Other times, keywords in the message body are used, or perhaps the email address of the sender of the message. Some more advanced filters, particularly anti-spam filters, use statistical document classification
Document classification
Document classification or document categorization is a problem in both library science, information science and computer science. The task is to assign a document to one or more classes or categories. This may be done "manually" or algorithmically...
techniques such as the naive Bayes classifier
Naive Bayes classifier
A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions...
. Image filtering can also be used that use complex image analysis algorithms to detect skin-tones and specific body shapes normally associated with adult-images (pornographic images).
See also
- Anti-Spam SMTP ProxyAnti-Spam SMTP ProxyThe Anti-Spam SMTP Proxy server project is an Open Source, Perl based, platform-independent transparent SMTP proxy server available at SourceForge.net that leverages numerous methodologies and technologies to both rigidly and adaptively identify e-mail spam...
- Bayesian spam filteringBayesian spam filteringBayesian spam filtering is a statistical technique of e-mail filtering. It makes use of a naive Bayes classifier to identify spam e-mail.Bayesian classifiers work by correlating the use of tokens , with spam and non spam e-mails and then using Bayesian inference to calculate a probability that an...
- CRM114
- dSPAMDSPAMDSPAM is a free software statistical spam filter written by Jonathan A. Zdziarski, author of the book Ending Spam and other books. It is intended to be a scalable, content-based spam filter for large multi-user systems...
- information filtering
- MaildropMaildropmaildrop is a Mail delivery agent used by the Courier Mail Server. The maildrop MDA also includes filtering functionality.maildrop receives mail via stdin and delivers in both Maildir and mbox formats....
is an MDA (Mail Delivery AgentMail delivery agentA mail delivery agent or message delivery agent is a computer software component that is responsible for the delivery of e-mail messages to a local recipient's mailbox...
) for UnixUnixUnix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
systems. - Markovian discriminationMarkovian discriminationMarkovian discrimination in spam filtering is a method used in CRM114 and other spam filters to model the statistical behaviors of spam and nonspam more accurately than in simple Bayesian methods. A simple Bayesian model of written text contains only the dictionary of legal words and their...
- POPFilePOPFilePOPFile is a free, open source, cross-platform mail filter originally written in Perl by John Graham-Cumming and maintained by a team of volunteers. It uses a naive Bayes classifier to filter mail. This allows the filter to "learn" and classify mail according to the user's preferences. Typically...
- Policyd-weightPolicyd-weightpolicyd-weight is a mail filter for the Postfix mail transfer agent written in Perl, by Robert Felber. It allows postfix to evaluate mail envelope information and to score mail against several DNS-based Blackhole Lists before the mail is queued...
Postfix policy-daemon before SMTP DATA - ProcmailProcmailprocmail is a mail delivery agent capable of sorting incoming mail into various directories and filtering out spam messages. Procmail is widely used on Unix-based systems and stable, but no longer maintained; users who wish a maintained program are advised to use an alternative MDA, such as...
is an MDA (Mail Delivery AgentMail delivery agentA mail delivery agent or message delivery agent is a computer software component that is responsible for the delivery of e-mail messages to a local recipient's mailbox...
) for UnixUnixUnix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
systems. - SendmailSendmailSendmail is a general purpose internetwork email routing facility that supports many kinds of mail-transfer and -delivery methods, including the Simple Mail Transfer Protocol used for email transport over the Internet....
supports libmilter for mail filtering - Sieve (mail filtering language)Sieve (mail filtering language)Sieve is a programming language that can be used to create filters for email. It owes its creation to the CMU Cyrus Project, creators of Cyrus IMAP server....
is an RFC standard for describing mail filters - SpamAssassinSpamAssassinSpamAssassin is a computer program released under the Apache License 2.0 used for e-mail spam filtering based on content-matching rules. It is now part of the Apache Foundation....
- SpamBayesSpamBayesSpamBayes is a Bayesian spam filter written in Python which uses techniques laid out by Paul Graham in his essay "A Plan for Spam". It has subsequently been improved by Gary Robinson and Tim Peters, among others....
- SpamchekSpamchekSpamchek is the trade name for a range of managed email services developedand operated by ENIDAN Technologies GmbH in Herrliberg, Switzerland...
- Spamfighter
- Spam ReaderSpam ReaderSpam Reader is an anti-spam add-on for Microsoft Outlook, produced under shareware license. The program uses content filtering based on Bayesian spam filtering algorithm, whitelist and blacklist techniques, additional spam signs like embedded images from internet, attached executed files, and junk...
- White list#Email whitelists