Serverless Security Forensic Data Analysis With PureSec

My two previous blog posts covered one of my favorite topics related to serverless security, the first blog post demonstrated how you can automate SQL Injection testing by using SQLMap together with ‘Lambda-Proxy’, a small utility developed and published by PureSec. In the second blog post, we demonstrated how you can immunize your serverless functions against SQL Injection attacks in less than 2 minutes, by using the PureSec SSRE. We ended the last blog post with a promise to demonstrate how you can leverage PureSec to perform security forensic data analysis – so, here we are.

For this short tutorial, we’ll use (almost) the same serverless function we used in the last episode – here’s a quick reminder:

We are using a Lambda function, which receives an SNS event, reads the ‘Message’ field from the event and checks it against a SQL database table (AWS RDS), which holds SNS messages that were stored previously. If the table has an entry with the same Message field, all the relevant SNS message data is retrieved. Unfortunately, the developer of this Lambda function made a poor development choice, trusted the event input blindly, and used the Message as-is, without performing any input validation first.

Sponsorships Available

Sponsorships Available

Sponsorships Available

Similar to last time, we will now run SQLMap with ‘Lambda-Proxy’, and hammer our function with the entire battery of SQL Injection exploit payloads. However, this time, we will not take the hacker’s point-of-view, but rather dig in the logs, to see what kind of visibility we’re getting, and whether or not it’s easy for us to spot the attacks in CloudWatch:

Since there are plenty of log lines, let’s zoom on a few of them:

Most of our CloudWatch log lines seem normal, with a few occasional syntax errors, related to the SQL queries failing. If you’re an app security expert, that could be indicative of an attack, however, to the untrained eye, these exceptions just seem like coding errors.  

Now let’s switch to the PureSec Serverless Security Dashboard:

On a first look, you can already see something weird – the line chart that demonstrates the correlation between function invocations, and PureSec alerts, clearly shows a spike in security alerts – this is odd. Seems like we’re being targeted. Let’s scroll down a bit, and look at the following statistics:

Ok, so we are learning that the attacker is targeting the AWS Lambda function named ‘sns-rds-db-checker-dev-sns_db_checker’. Luckily for us, this is the exact same function that is protected by PureSec, which is why we are receiving logs, and also why we are seeing all 1,300 exploit attempts getting blocked. From the second table, we learn the following:

All 1,300 exploit attempts were using the AWS SNS event as the source for the event triggers. All of them blocked. Ok, it’s time to launch our Splunk dashboard, and see the deep forensic data provided by PureSec:

The top chart, clearly shows some troubling spikes, related to serverless security events. Looking a bit below, we have the ‘Serverless Application Firewall’ event triggers, which clearly lay out the following information in front of us:

• Malicious event time
• Target function ARN
• Attack type
• Malicious payload

Based on this, it’s already very clear what’s going on – the attacker is attempting to inject malicious SQL queries (SQL Injection payloads) through the SNS event. By clicking on one of the events, we can jump directly into the deep forensic log data:

In the image above, you can see the actual exploitation attempt, as carried over the spoofed SNS event. Take a look at the element event.records[0].Sns.Message field – the attacker is using a Blind SQL Injection technique to attempt (and fail) data exfiltration!

Isn’t this awesome? Just a minute ago, we were looking at what appeared to be (almost) normal function invocation logs, with maybe some syntax errors, and within a few clicks, we already have all the data we need to conclude what really happened.

Detecting & Preventing Covert Data Leakage

Before we wrap up, I’ll let you in on a little secret, while I was developing my function, I made another mistake… I imported a 3rd party library called Synonym-Extractor:

This is a very popular python package, which enables developers to run extremely efficient pattern matching. However, someone (in this case it was one of my colleagues) managed to “poison” the git repository from which I was cloning the package. That person is after my AWS account credentials…as you can learn from the code below:

It seems like the person who poisoned this dependency, is trying to harvest my AWS Lambda function environment variables, and to transmit them over an encrypted channel (TLS) to a remote location ( in this case, to www.some.site )

Luckily for us, the PureSec SSRE is about to demonstrate one of it’s most powerful sides – the ability to spot data leakage attempts in real time, and to block them.

As a reminder, here’s the output of my function when it executes normally:

And here’s the output after I imported the poisoned dependency (with PureSec deployed)

The function execution was halted, and it seems that the TLS connection was aborted. Let’s use Splunk again, to run our forensic analysis – from the data below, it looks like the PureSec SSRE runtime protection module spotted and blocked the malicious behavior – an attempt to connect and leak data:

Let’s look at the data:

Yup – there was an attempt to connect to a remote domain. Let’s take a look at the TLS/SSL monitoring data:

It appears that my mischievous colleague tried to leak my environment variables via the URL of an HTTPS request.

Again, PureSec saves the day 🙂 – no harm done, the attempt to leak our sensitive data was spotted and blocked, and just as important, I was alerted in real time, and presented with all the forensic data I needed.