Finding Non-Trivial Web API Issues with Mayhem for API

Finding Non-Trivial Web API Issues with Mayhem for API

Web APIs have become increasingly important to the operation of modern business. Many business models for new products and services are constructed based on APIs such as billing, IoT and identity providers. Engineers are under pressure to deliver web APIs that are tested, observable, maintainable, scalable and secure! 

In order to support rapid delivery without sacrificing software quality, there has been an explosion in DevOps culture to ‘shift-left’ the application development lifecycle. That is, to move the processes that determine the quality and security of software as close as possible to the engineers writing the code. Linters, Unit tests, Integration tests, static analysis, DAST,  code review and other practices have embraced the shift-left culture and are a common practice in most CI/CD build pipelines. 

Manual tests tend to focus on the ‘happy path’ – which satisfies a code coverage % gate, but will miss out on testing inputs that are outside of the bounds of the happy path (such as integer overflow or divide by zero issues).

This is where fuzzing shines. A good fuzzing engine will automatically generate a comprehensive suite of inputs used to test function and robustness of the software it is targeting. By using fuzzing techniques to generate inputs and observing the response from the target software, it is possible to quickly iterate through multitudes of test cases to find weakness in functionality or security.

Consider a toy code example with bug that is only present after satisfying a sequence of cascading conditions:

 read (input)
if input[0] != ‘c’:
   if input[1] != ‘a’:
      if input[2] != ‘t’:
         throw exception
return “ok”

A fuzz tester will generate a large number of test cases  to execute against this block until it finds the crashing input, “cat”. 

Now, let’s take a look at a toy REST API, which requires a number of preconditions to be satisfied before an issue is triggered. 

This API is used to manage a collection of ‘Resources’, where each ‘Resource’ has its own collection of ‘Widgets’:

We can implement this with a very basic REST API:

 
##
# Get and Create Resources
##
GET /resources
POST /resources

##
# Get and Create Resource Widgets
##
GET /resources/{resource_id}/widgets
POST /resources/{resource_id}/widgets

After implementing the API, we are asked to extend it with two new calls:

• Deactivate widgets

• Transfer Deactivated widgets to another resource

##
# Deactivate a Widget
##
DELETE /resources/{resource_id}/widgets/{widget_id}

##
# Transfer inactive widgets to another resource
##
POST /resources/{source_resource_id}/transfer_inactive_widgets

In order to test the happy path for the ‘transfer’ call, we will need to write a test that performs the following requests:

 # 1. Create a new SOURCE Resource
POST /resources

# 2. Create a new DESTINATION Resource
POST /resources

# 3. Create a new Resource Widget
POST /resources/{resource_id}/widgets

# 4. Deactivate the widget
DELETE /resources/{resource_id}/widgets/{widget_id}

# 5. Transfer deactivated widget from SOURCE to DESTINATION
POST /resources/{source_resource_id}/transfer_inactive_widgets

This leaves us with a lot of corner cases / boundary cases to cover! What happens when ‘source_resource_id’ doesn’t exist? What happens if we try to transfer widgets, but none exist, or none are deactivated?

In a rush to get the feature out, we may just test the happy path and ship it! Customers need this yesterday! 

Here is some of the new code awaiting review:

1 @app.post("/resources/{source_resource_id}/transfer_inactive_widgets/")
2 def transfer_inactive_widgets(
3    source_resource_id,
4    target_resource_id,
5    db):
6 
7    source_resource = db.get(Resource, source_resource_id)
8    if not source_resource:
9        raise NotFoundException
10
11  for widget in db
12            .query(Widget)
13            .filter(Widget.resource_id == source_resource_id)
14            .all():
15      if not widget.active:
16          # Potential SQL Injection here!
17          db.execute(f"UPDATE widgets
18                       SET resource_id='{target_resource_id}'
19                       WHERE id = '{widget.id}'")

In the implementation above, there is a potential SQL Injection on line 17. But the only way to make it to line 17 is to have:

• A resource matching ‘source_resource_id’
• An ‘inactive’ widget that belongs to ‘source_resource_id’

Due to these constraints, the probability of a typical DAST scanner detecting the SQL Injection vulnerability is very low, if it is found at all.

How Mayhem for API goes deeper

Mayhem for API runs as a fuzzer against REST APIs. It generates requests from an API specification (OpenAPI/swagger) and inspects the responses in order to build a suite of test cases which successfully execute the API.

Each test case is a sequence of requests. 

 # Test case 1 
1. POST /resources
2. GET /resources

# Test case 2
1. GET /resources
2. POST /resources/{resource_id}/widgets
3. DELETE /resources/{resource_id}/widgets/{widget_id}

# Test case 3
1. POST /resources
2. POST /resources/{resource_id}/widgets

…

A valid test case is a sequence of requests where every request is responded to with a successful status code (HTTP Status Code 200/201/etc). An endpoint (path and method) is considered ‘Covered’ if it has returned a successful status code to at least one generated request.

The more endpoints that Mayhem for API can successfully cover, the more interesting issues it will be able to trigger. This follows the notion that successful requests put the API into a state where responses contain enough information, such as unique identifiers, to make more interesting requests against other endpoints that reference those same IDs.

These test cases are then modified to try to elicit an unhandled or unintentional error. The most common of which is a 500 Internal Server Error response. But other behaviors may be triggered by passing more malicious payloads, such as Server-Side-Request-Forgery and Command Injection.

Mayhem for API will also use the responses from each request in a test case in order to attempt to make new requests that use real object ids and their relationships.

For example, Mayhem for API may generate a test case that executes most of the happy path described earlier:

# 1. Create a new SOURCE Resource and save the new ID
POST /resources               

# 2. Create a new DESTINATION Resource and save the new ID
POST /resources

# 3. Create a new Resource Widget using resource id from (1)
POST /resources/{resource_id}/widgets

# 4. Deactivate the widget using the widget id from (3)
DELETE /resources/{resource_id}/widgets/{widget_id}

If this test case completes  successfully, a subsequent run will add a new request … this new request will trigger the SQL Injection seen in the source code above:

# 5. Transfer deactivated widget uses source resource id from (1)
#    Uses a string that triggers SQL Injection for target resource id
POST /resources/{source_resource_id}/transfer_inactive_widgets
       ?target_resource_id='; or 1=1

When Mayhem for API runs this new test case with the invalid request – the API responds with an error that indicates potential for a SQL Injection!

psycopg2.errors.SyntaxError: syntax error at or near "or"
LINE 1: UPDATE widgets SET resource_id=''; or 1=1'                  ...
                                           ^
The above exception was the direct cause of the following exception: 

The issue is found! 

While we may have been able to find this issue with purely random requests after a significant period of time, Mayhem for API will find it much faster. It does this by learning how the API operates while building a suite of test cases, and then modifying those test cases in order to elicit undesired behavior.

Try it yourself!

Clone the example source code repository referenced in this article:  https://github.com/ForAllSecure/mapi-coverage-example

Test this API and your own APIs by heading over to Mayhem for API and get started for free!