Four Key Elements for Comprehensive Network Threat Detection

by Bricata on April 7, 2020

Today’s cybersecurity landscape is a mix of old and new threats. Many of the attacks that organizations encounter involve legacy malware and techniques that are repackaged and reused (this post from security researcher Patrick Wardle has a good example of this using malware from North Korea’s Lazarus Group). On the other hand, sophisticated attackers are constantly designing completely new attacks that evade today’s most advanced threat detection methods – as a matter of fact, researchers demonstrated how to create adversarial malware designed to evade machine learning detection at last year’s DEF CON AI Village contest. No single method detects all types of threats. History has demonstrated that successfully protecting your network is best done by employing multiple threat detection capabilities that inspect traffic from a wide variety of different perspectives.

Here are four key elements of comprehensive network threat detection:

Deep Packet Inspection (Signature-Based) Detection

It’s important to look for malicious traffic as it comes onto the network. A comprehensive network security appliance should use deep packet inspection (DPI) to evaluate network traffic against signatures (addressing, characteristics, and values) of known threats. Suricata is a popular and powerful open-source DPI engine for accomplishing this. Roughly 50-70% of malware hitting today’s networks can be detected by DPI and signature-based solutions, making it one of the easiest ways to identify (and stop) known bad traffic from traversing your network. Signature-based threat detection comes with the significant benefit of being fast and requiring the least CPU processing power of all available threat detection methods. This alone makes it a useful first step in blocking known threats.

Behavioral Anomaly-Based (Stateful) Detection

Looking deep into individual network packets is vital, but it’s not enough to protect against today’s advanced threats, especially ones that have been built to evade signature-based detection. As a result, organizations need to layer on behavioral anomaly-based detection (also known as stateful detection) capabilities. Zeek (formerly Bro) is a popular and powerful open-source stateful-detection engine for accomplishing this. These tools monitor how traffic behaves as it traverses the network, records this behavior in log files, and then compares current traffic with historical baselines to detect indicators of malicious activity and deviations from observed norms. For example, if a network normally connects to a particular site three times per day, and then one-day connections increases to 300 per hour, an exfiltration event is likely underway. If a script has been created to recognize this as abnormal behavior, an engine like Zeek will detect it.

Since stateful tools have a “memory” of what has happened on the network, they can detect threats that would evade signature-based detection. For example, a piece of malware split into three parts, each delivered in a separate transmission, wouldn’t trip a signature-based solution because each individual packet registers as benign. But a stateful tool would check past traffic and detect that all three pieces together are part of a set. Then it can flag them all as malicious and signal endpoint detection and response (EDR) and security orchestration and response (SOAR) tools to mitigate the threat.

Sponsorships Available

File Hashing and Detection

Looking at traffic is only part of the security challenge. Payloads are also important. A network security platform that automatically hashes every file that comes onto the network allows you to easily identify known malicious files. Tools like Zeek (and Suricata) can detect files within network traffic (attachments in emails, downloads from websites, etc.), extract them, and generate their MD5 and/or SHA1 hashes. Comparing these hashes against a feed of known bad hashes (from sites such as Google’s VirusTotal) to see if there’s a match can quickly identify malicious files.

Artificial Intelligence and Machine Learning Detection

If an unknown file isn’t flagged as malicious through file hashing, it could still be a threat, possibly one that hasn’t been seen before. Hackers use polymorphism techniques (like encrypting payloads with a different encryptor/decryptor pair in each malware instance or adding useless pieces of code) to change malware so it produces a different hash every time it is hashed. This is why you need other methods of protection. Artificial intelligence and machine learning (AI/ML) can counter some zero-day and polymorphic malware behaviors using methods that find indicators of malware without explicit signatures, rules or scripts for them. Comprehensive network security solutions should include the ability to send files to an AI/ML service like Cylance to provide additional security against some advanced or polymorphic threats.

Bonus Element: 2^nd Opinions from an External Sandbox

Threat detection tools require tuning to establish their optimal sensitivity to known and unknown threats. Make them too sensitive and they trigger on too many legitimate files, causing teams to waste time following up on too many false positives. Make them too permissive and they could let malware slip onto and across the network (false negatives). One option to help navigate this balancing act is to send files flagged as malicious by the AI/ML engine to an external sandbox to be detonated. Applying sandboxing to every file is too resource-intensive but using it to obtain a “second opinion” and eliminate false positives can reduce the workload significantly. Look for network security solutions that offer this option, either built-in or offered via an API connection to a third-party service like the Cuckoo Sandbox.

The best network security solutions use a variety of threat detection methods for the greatest chance of detecting all security threats. Having an intrusion detection platform that integrates all of these network threat detection techniques and allows security analysts to manage them in a centralized console is critical to delivering comprehensive threat detection and response that’s actionable.

To learn more about how Bricata provides all of these threat detection capabilities in a single platform, click here.

###