Encrypted Traffic Analysis Will Be Mandatory Soon

 Although today much of the internet traffic is encrypted, attackers can still exploit it. While the need to examine encrypted traffic is obvious, the way to carry out decryption often remains a conundrum. Decrypting traffic can introduce performance bottlenecks and introduce potential privacy and compliance issues if the traffic is fully unshrouded. Finding a way to maintain performance and ensure compliance while also being able to properly examine traffic is becoming critical.

Encrypted traffic needs to be examined to uncover potential functions for controlling botnets and malware that are often hidden within “secure” tunnels. Examining encrypted traffic will also help investigate various issues. Take, for instance, a workstation that abruptly started to communicate using an outdated encryption algorithm. Such is likely a clear sign of being compromised. Or consider users communicating with servers with untrusted certificates. The ability to analyze encrypted communications such as these is growing more crucial each day for the effective enforcement of security policies.

While only half of internet traffic was encrypted in 2017, today it is over 80%. The era of a fully encrypted internet is already knocking on the door and, naturally, professionals responsible for security and risk management in companies are paying more attention. Encryption complicates the use of traditional security technologies, such as firewalls, and also often makes their use impossible. If you do not know what is hiding in packets, you cannot fully protect the corporate network or individual workstations from malware.

Today, the analysis of encrypted communication should be part of the portfolio of network monitoring and security for every company. Some security solutions are adding such capability, providing the ability to analyze header information of encrypted traffic without having to open the payload. Thanks to this functionality, enterprises are now able to display important details of encrypted communication, including detecting hidden malware. However, the encrypted content cannot be viewed without decryption. So it is important to get as much information as possible when the communication is not yet encrypted during the process of establishing the connection when the exchange of encryption keys and certificates is being conducted.

An example of this connection setup is a SSL/TLS handshake, which is required for establishing encrypted communication during which different TLS parameters are available and visible, including the TLS protocol version used by the server, encryption set, server name (SNI) indication, certificate issuer, public key, certificate validity, JA3 fingerprint and more.

The connection data can then be analyzed or used in different ways to manage the security of the organization. Based on the data, one can receive notifications of changes and events or use it for automatic alerts that are linked to other actions (emailing, running a user script, sending a syslog or an asynchronous notification in the form of an SNMP trap, etc.).

JA3 Helps With Malware Detection

One of the easiest ways to detect malware and process indicator of compromise (IoC)  is to analyze JA3 fingerprints. Using JA3 method, one can easily create SSL/TLS fingerprints on any platform. It is much more effective to use JA3 fingerprints to detect malware within SSL/TLS than to monitor the IP or domain IoC. It does not depend on whether the malware uses domain generation algorithms (DGA)  or changes the IP addresses for each of its command and control (C2)  hosts, not even when it uses, for example, Twitter, to control it. Since JA3 detects a client application directly, it can detect malware based on how it communicates instead of what it communicates through. Thanks to this, special tools such as those in Flowmon, in cooperation with the publicly available JA3 fingerprint database, can detect potential threats from specific JA3 fingerprints in encrypted communication.

How To Audit Security Policies

Many companies rely on HTTPS communication and certificates issued by a certification authority for a given period to secure their internal communication or web presence. It is important to monitor the validity of the issued certificate to avoid a situation where data remains unsecured for some time. This can be elegantly solved by analyzing encrypted traffic, which provides, among other things, an overview of each certificate’s expiration. This allows one to monitor expiring certificates and completely avoid the problem of expired certificates altogether. One can also easily detect weak TLS 1.0 encryption with enough time to take all the necessary corrective steps.

Some security solutions provide encrypted traffic analysis on two levels. The first focuses on cryptographic evaluation, i.e. examines versions of the SSL/TLS protocol, cyber suite (encryption algorithms, key lengths) and certificates, while the second focuses on monitoring and security. It offers JA3 fingerprints for possible identification of malware or infected stations and ALPN for identifying protocols in encrypted communication and examines SNI and many other parameters.

Companies Need To Prepare for a Strategic Change

For reliable threat protection, companies eventually will need to incorporate security tools based on behavioral analysis, artificial intelligence and encrypted communication analysis. These tools promise to detect malware in real-time encrypted traffic without impacting network throughput or degrading application performance. It will also require changes to existing security strategies to stop man-in-the-middle threats or attempts to steal corporate data promptly.

New security technologies such as these will be indispensable for not just protective security, but also for auditing. The technologies will help detect communications that use outdated certificates in violation of company policy, control the encryption strength or reveal data encryption vulnerabilities. Most organizations today can only get to such detailed overviews at the cost of laborious and time-consuming methods.

In a way, we can apply the Socrates dictum about the unexamined life not worth living to network security. Unexamined traffic undercuts all of the other important security methodologies and makes them not worth having, providing a way for attackers and bad actors to gain access to resources right under the nose of security inside encrypted tunnels. These need careful examination and can be done largely without performance penalties and compliance exposure.

Artur Kane

Avatar photo

Artur Kane

Artur Kane works as a Technology Evangelist and Analyst Relations expert at Flowmon Networks. He has gained extensive experience in both customer and technically oriented positions, which he invests into raising awareness of network visibility and creating an educated community of enthusiasts.

artur-kane has 3 posts and counting.See all posts by artur-kane