Virtual Patching the Signal Sciences Way

by Matt Vanderpol on June 18, 2019

Virtual Patches: Security Duct Tape

Let’s get serious for a minute. Virtual Patching is Cyber duct tape. Originally coined by Intrusion Prevention System (IPS) vendors years ago, Virtual Patching has been known by different names including “External Patching” and “Just-in-time Patching.” Regardless of naming convention we’d like to provide our customers with a behind the scenes peek of how we develop virtual patches at Signal Sciences. We’ll also check out a DIY Virtual Patch using Signal Sciences Power Rules.

Leveraging our in-house expertise we focus on identifying and responding quickly to common vulnerabilities and exposures (CVE’s) less than 365 days old with known publicly available proof of concepts. Our official recommendation for CVE’s is: patch that vuln!

Unfortunately, real world business conditions are not always conducive to patching software in a timely manner. From bureaucratic boondoggles to chained dependencies patching servers has been a pain in the side of the industry for years.

An Automated Virtual Patch Pipeline

At Signal Sciences, we wanted an automated method of identifying and triaging CVEs for qualification to virtual patches. Initially this started as a python script running on an engineer’s computer. But we’ve evolved it to a relatively robust process for staying on top of the latest CVE related vulnerabilities.

At a high level the pipeline looks something like this:

Sponsorships Available

An AWS Codebuild job kicks off every six hours
- The code will first gather the last 100 updated CVE’s from: https://cve.circl.lu/api/last/100
With the last 100 CVE IDs we check two more things:
- Github for PoC code
- cve-search for linked PoC code from ExploitDB, Metasploit, or Packetstorm (Example: https://cve.circl.lu/api/cve/CVE-2019-6340)
Once we’ve identified a CVE that has new PoC code we will then triage the CVE internally and evaluate some criteria before moving forward with a patch:
- Is the nature of the attack something we can defend using a web application firewall (WAF)?
- What are the chances one of our current or future customers will be affected by the CVE?
- Is the exploit unique enough the virtual patch we do deploy won’t cause unnecessary false positives or other issues?
With a patchable CVE our team then deploys the new rule to production
- We notify customers via email of the newly available patch.
Additionally we have monitoring/logging in place to track:
- Corp/Sites/Users who have enabled the virtual patch
- Results of any hits on the Virtual Patch rule

We continue to iterate and improve this automated system to identify new CVEs.

The additional checks performed after a virtual patch is deployed assist us in proactively resolving false positives/negatives, identifying unique attackers/payloads, and overall help us to understand how our customers are using virtual patches.

Outside of the automated process outlined above we are still humans and miss things. We are always open to investigating and assisting with virtual patches not in our products, so please get in touch with [email protected] if your organization is struggling with a virtual patch we do not currently cover.

DIY Virtual Patching with Signal Sciences Power Rules

Are you a battle hardened IPS/IDS/WAF sysOpsSecDevEng Ninjaneer? Do you manually disassemble packets and block them on the wire regularly?

If you are, we have the tools that empower you to craft your own virtual patch rules.

Let’s run through a hypothetical DIY virtual patch using Power Rules.

Step 1: Read Beyond The Headlines

We’ve all been victim to impulsive panic when seeing a headline like:

“New Apache CVE has IT field in chaos!”

“PHP zero day strikes again, server farm on fire in the south!”

Now before we get too far into virtual patching we need to be clear there is only a small subset of CVEs which can be mitigated via a WAF. CVEs range from local privilege escalation in Apache (https://cve.circl.lu/cve/CVE-2019-0211) to vulnerabilities that occur in specific languages.

Let’s look at two contrasting examples:

EXAMPLE 1: local privilege escalation in Apache

“Serious Apache server bug gives root to baddies in shared host environments”

The words “serious”, “root”, “shared host environments” are all major causes for concern. The mention of “Apache” is a bit of a hint that a WAF might be able to help us out here, but upon further reading into CVE-2019-0211, we soon realize not only is this a local privilege escalation issue, but also does not affect non-unix systems. Depending on the organization those two key points of information should be enough to decide that WAF is not going to help here or the team may have already offloaded their responsibility for this issue to some purchased service.

EXAMPLE 2: vulnerabilities that occur in specific languages

CVE-2017-5638 a.k.a. “The Equifax” Apache Stuts Vulnerability: By abusing Object Graph Navigation Language (OGNL) in Apache attackers were able to achieve remote code execution. One intractable problem we have in application security is to know what the fix is, but struggling to implement the fix at scale in not only technically complex systems, but also complex human systems.

Step 2: PoC or Go Home

A proof of concept is the most basic requirement for developing a rule. The proof of concept will provide the necessary details to identify a unique request pattern and help the team to develop a request rule to protect a system until a patch is ready.

Let’s look at the PoC for CVE-2017-5638

https://github.com/rapid7/metasploit-framework/blob/master/modules/exploits/multi/http/struts2_content_type_ognl.rb

Specifically line 95: https://github.com/rapid7/metasploit-framework/blob/master/modules/exploits/multi/http/struts2_content_type_ognl.rb#L95

From the Metasploit module we are able to establish the exploit payload is dropped into the Content-Type header. Luckily Content-Types are relatively standardized.

Note the preamble of the payload is %{. Additionally there are some other keywords that look unique to the exploit and would likely never be in the Content-Type header in normal conditions (ognlUtil, opensymphony.xwork2, and multipart/form-data). Using this information we can come up with a DIY virtual patch.

Using the wildcard syntax we can look for requests where the Content-Type header starts with %{ and contains one of the keywords we identified in the PoC:

%\{*{ognlUtil,opensymphony.xwork2,multipart/form-data}*

This should be concise enough to block attempted exploits of this CVE without impacting normal operations. In the example we show the rule blocking immediately, but in any case where your team is experimenting with these sorts of rules we do recommend simply adding a signal for a period of time and moving to blocking once the team is confident in how the rule works.

Step 3: Profit

We continue to improve on the CVE pipeline to help provide timely and relevant virtual patches to our customers. Learning to stay calm and collected when headlines contain trigger words is paramount to any application security team’s success. Panic is one of the last things a security person should be introducing into a complex system.

With the process and approach we’ve outlined here, devops and software engineers with the responsibility for addressing known vulnerabilities can better sift through the noise to determine how actionable and relevant a new CVE is to their organization. Our Power Rules help customers to also build customized protections on their own time. Keep in mind though the best solution to vulnerable software is to deploy the patch.

The post Virtual Patching the Signal Sciences Way appeared first on Signal Sciences.