Why Non-Functional Testing is Equally Important to Functional Testing

by Tamulyn Takakura on December 2, 2020

Developers live in the world of features. So, it comes as no surprise that when it comes to testing those features, developers are mostly focused on functional testing.

Functional testing is when a feature is tested against a list of requirements that describe what the feature should be able to do. Those features are then accepted and marked as complete when these features are able meet their requirements. However, in today’s era of digital transformation, we posit that non-functional testing is equally an important developer metric as functional testing.

Applications are at the core of every business

Sponsorships Available

Applications are at the heart of every organization, pumping information throughout the business to drive objectives. Applications are a driving force behind business productivity — and at times, entire economies.

When business livelihood is dependent on applications, feature quality becomes a customer driven demand. That’s where non-functional testing comes into play.

While non-functional testing is often considered a security measure, it’s increasingly becoming considered a quality measure. It verifies and validates an application’s speed, scalability, reliability, and efficiency. Ultimately, it ensures that an application is robust and able to withstand unforeseen use cases.

Non-functional testing is, however, exponentially harder than functional testing. With functional testing, there’s a finite number of ways that a feature can be used. With non-functional testing there’s an infinite number of possibilities. So it just isn’t realistic and far too cumbersome to expect developers to test every possible misuse case, especially in today’s CICD world.

Addressing non-functional testing with machine efficiency

Fuzz testing is an effective solution for addressing those non-functional testing challenges. Unlike today’s approach, fuzzing doesn’t require a developer to manually craft every possible input. Not only is it able to do so automatically, but it is also able to come up with random variations of an input (on-the-fly) at speed and scale. As a result, fuzzing is able to help developers quickly root out reliability issues in their features — all as a part of their development process.

It’s true. Formerly, fuzz testing was a technique primarily leveraged by security researchers. However, we’re seeing this dynamic shift. In the past five years, we’ve seen fuzz testing overcome significant technical limitations that make it easier to use and allow it to integrate directly into developer workflows. As a result, there’s an increasing number of developers wielding this powerful technique. In fact, this summer, Dark Reading predicted that fuzz testing is a trend that will rise as organizations aim to create agile software development processes, such as DevOps.

Even repository managers and devops tool chains are beginning to tout their ability to integrate with fuzzing capabilities. In ForAllSecure’s case, Mayhem integrates into most CI/CD tools. We’re certainly not limited to these, but here’s just a few of the popular integrations leveraged by our customers: Jenkins, Travis, CircleCI, Concourse CI, Drone CI, Bamboo, and Team City.