Identifying False Positives to Improve Email Security
In March 2016, the email security of former White House Chief of Staff John Podesta was breached. A spear-phishing email warning Podesta to change his Google password was caught and placed in his spam folder but mistaken by a staff member as legitimate. The email was opened and the password entered by the staffer, after which 20,000 pages of Podesta’s email were posted online. The staffer’s judgment that it was a false positive—i.e., not a malicious email, in spite of being found in the spam folder—threw the 2016 presidential election into turmoil and may have altered the course of history.
The willingness of users to pull malicious email out of a spam folder is a consequence of security software crying wolf. The more often legitimate emails are misclassified as malicious, the more likely users are to ignore warnings and invite the wolf through their door.
In screening for anything undesirable—whether through drug testing, lie detectors or email security software—there is a trade-off between avoiding false positives and preventing false negatives (failing to catch malicious behavior). The stricter the screening, the more likely it is to generate false positives.
Mass-emailed commercial spam, such as Viagra advertisements or unwanted newsletters, is relatively easy to spot and block, and a false positive is likely to do no more than send a newsletter to the spam folder. Stopping malicious email that is more carefully targeted—including Business Email Compromise (BEC) attacks—without false positives poses a much more difficult challenge. Unlike spam, the consequences of failing to block targeted attacks are dire, while blocking legitimate email could wreak organizational havoc.
The relatively low number of targeted email attacks makes them harder to detect than spam or other unwanted email. According to Cisco’s Talos, 85 percent of global email traffic in November 2017 was spam or malicious, while targeted attacks make up less than 0.7 out of every million emails, according to Agari’s research. There is no available dataset that sufficiently represents malicious emails.
BEC emails are far more difficult to spot than spam, for both users and traditional security technologies. Each BEC email is tailored to a specific email recipient. They often use highly targeted social engineering and sometimes benefit from inside knowledge gleaned from previous successful attacks on a company. Typically they are sent in small quantities, not in batches of millions, and use “business as usual” language. None of them screams “Viagra.” Adding to the challenge of designing models to identify this type of fraud is that it adapts rapidly.
So what approaches are most successful in minimizing both false negatives and false positives? Agari has established mechanisms designed not to identify bad behavior, but to identify deviations from ordinary (good) email communication. We don’t know what attackers will do next, but if we know what good email behavior looks like, we can spot anything that acts differently. We’ve found that these deviations from good are best detected using a combination of expert-designed rule systems corresponding to principal types of attack, along with machine learning methods to generalize among variants of one principal attack type.
The core machine learning models identify the “authenticity” and “trust” associated with each email. Authenticity refers to whether the IP address associated with the sender is in fact associated with the domain used. Trust is defined as the objective trustworthiness of an email, taking into account things such as use of “confusable” characters (such as a Cyrillic letter that looks like the letter o in “password”), deceptive misspellings, sender history and other metadata. From these a final risk score is generated. Rules are then applied to define what is ultimately classed as a fraudulent email.
While this method has proven very successful at capturing malicious emails, it runs into another problem: Organizations and their behavior change. Models looking for deviations from normal patterns of email communication will sometimes classify legitimate changes as fraudulent, generating false positives. For example, using new domains associated with partners risks triggering false positives. To accommodate new or rare partner domains we must classify partners based on measurable characteristics of their relationship with the organization we are defending, such as the volume and breadth of email communication.
For each type of legitimate deviation from normal behavior, a model must be developed to reduce false positives. This is an iterative process, with the model re-engineered and retrained until a high level of accuracy is achieved. Machine learning then enables us to generalize beyond those examples and automate pattern detection at scale.
Catching fraud and simultaneously keeping false positives to a minimum is a challenge that can be met with a network of models working in tandem, each focusing on a few nuances of email. Through these techniques, it is possible to confidently catch fraudulent emails and allow good emails to reach their desired recipients unimpeded, without manual oversight.
