Heuristic Rules

Heuristic rule analysis for Spam can be defined as a method of examining the content and contextual characteristics of a message for indicators of Spam probability (also referred to as the confidence level).

For example, these rules might include the following:

• Message header follows non-standard formats and hop patterns.
• Excessive use of punctuation, including multiple interjections or exclamations.
• Unsubscribe footer follows conventional formats to appear legitimate.
• Text syntax consistencies between paragraphs appear normal.

To effectively evaluate Spam probability, heuristic rules should include illegitimate characteristics and context as well as legitimate traits. These rules should not depend on a pre-existing signature, but rather should employ standardized identifiers that are consistent no matter what combination of traits a Spam technique employs. For greater accuracy, heuristic rules can also include category-specific attributes that allow the recipient to tune their preferences to differentiate between Spam and expected communications (for example, differentiating between inappropriate sexual content; and a user's weekly prenatal health care newsletter by adjusting sensitivities in the rules diligence).

Unlike database methods based on signatures, heuristic rules excel at identifying new, unreferenced Spam, which is increasingly the majority of Spam. Spammers have learned to circumvent signature methods by using variable randomization, which can thwart the objective identification techniques. However, the heuristic rules methodology has a subjective nature; therefore, the chance of false-positives increases because the grey area of identifying Spam does not provide an explicit pre-existing reference fingerprint. For heuristic rules to be effective, they must provide a relevant rule set and have the capability to be tuned to overcome the difficult grey area of Spam definition.

With an effective heuristic rules system, the majority of Spam should be easily identified with additional tools that adjust the subjective interpretation with user-specific customization during the rule evaluation process.

Difference between heuristic rules and database signatures for fighting Spam.

There is a few different methods to address the Spam problem:

Traditional methods include databases of Spam signatures (much like AntiVirus) where a pre-existing pattern (like an email address, phone number, or phrase) gives away the nature of the message, usually based on a human pre-determination that a message *is* Spam and should be included.

There are also public databases (RBL, ORBS, etc.) that track a black list of malicious spammer addresses and domains. However, these have been known to contain errors due to overly draconian diligence.

In more recent years, spammers have learned the secrets to circumvent around these signature methods by masquerading origin or using variables to mislead the signatures, so enter heuristic rules - a newer technique, but already showing positive results as they evolve.

Heuristic rules analyze content and context of messages. As a result, an explicit pattern is not required. In essence, qualitative aspects of the message reveal its Spam-like nature. Heuristic rules work more like artificial intelligence to produce a Spam probability rating. In addition, when they are properly trained via tuning to learn; the preferences of your environment to reduce the chance of error (false positives), can be a highly effective method of Spam blocking that achieve more accuracy over time as they learn to recognize your definition of Spam.