Why Fuzz Testing Is Indispensable: Billy Rios

Why Fuzz Testing Is Indispensable: Billy Rios

I recently spoke to Gartner on the addition of fuzz testing to their Critical Capabilities for the Application Security Testing Magic Quadrant. In that conversation, one analyst shared that companies that implement fuzz testing programs never rip them out. Why? They’re just too valuable. 

This is a bold statement, especially in the world of application security where strategies are around tool augmentation and diversification, leading to frequent rotation of tools within product security programs.

So, in this series, I am going to look to my network to get validation and uncover more details on this observation.

I first reached out to fuzzing expert, Billy Rios. Rios is an author, researcher, venture advisor, and entrepreneur. He has led security engineering and product security programs at organizations with the most advanced fuzz testing programs, such as Google and Microsoft.  

See what Rios had to say about fuzzing in the interview below:

———–

Takakura: Does fuzzing matter?

Rios: Fuzzing allows organizations to find bugs before attackers do and there’s big value in that. And, it’s not just about finding bugs, it’s about finding interesting bugs, which makes fuzzing even more compelling. Eventually those bugs become evidence that allows teams to advocate for more fuzzing. In my experience, it’s hard to push on a technique that keeps bringing in bugs. So, yes.

Takakura: Where does fuzzing deliver value?

Rios: The whole point of fuzzing is automation, and that’s where fuzzing delivers value. Once fuzzing is strategically implemented within developer processes and it’s tailored to the product under test, everything just happens. Testing and validation happens automatically. And, the great thing is that once it’s in your program, it’s there and everything just works. At which point, why wouldn’t you use it?

Organizations will need to stomach that upfront one time implementation cost, but once they do, the fuzzing programs require little to minimal maintenance. This is why organizations that put in the effort to implement fuzzing don’t get rid of it. It’s also why it makes sense to work with a fuzzing vendor for the one time installation — so it can be deployed with the right strategy. It becomes a set it and forget it.

I neither have any recollection of any product manager or security engineer saying fuzzing is not worth it, nor any account of an organization that’s implemented fuzzing into their SDLC ripping them out — from Facebook to Twitter to Microsoft. 

I will caveat that fuzzing has to be done right. This is key. Fuzzing gets lukewarm responses when a half-baked job is done. Remember, the whole point of fuzzing is automation, so when it’s half automated, half manual, you start losing people. No one wants to get blasted with bugs, only to have to manually triage them. At that point, you’re just overwhelming security and development teams and they’re not handling them (the bugs). 

When organizations choose to implement fuzzing in the SDLC, they’re coming in with a different level of commitment. Fuzzing is most effective when it’s in the development process. This all contributes to the indispensability of fuzzing.

Takakura: What’s your philosophy on the “right” way to go about fuzzing?

Rios: When I look back at my experience at Microsoft, their fuzzing infrastructure was robust. It automatically kicks off and it was easy. It makes sense because they’re a product company. If you’re a product company, it would probably make sense for you to have this too.

But, there’s an art and science to fuzzing. It requires thoughtful architecture, ensuring that fuzzing happens at the right place and time in the development process to maximize efficiency. This can spark resistance. Anytime anything is implemented into a SDLC, there’s immediate hesitancy because they fear it will slow down releases. It’s a disruption all people and organizations hate. While there are methodologies that you can lean on here, there’s also experience and the eye needed to properly size the target under test — meaning estimating complexities, dependencies, and possible boobytraps — to assess the right fuzzing deployment strategy.

For those that want to bring fuzzing into their organization, but don’t have the experience to do so, I recommend starting with small projects and staying isolated from releases, then upgrading to larger applications and running adjacent to releases, and finally work yourself over to the SDLC. This gives organizations the data they need to inform the best implementation that will work for their product and SDLC. At the same time, you also have an opportunity to collect bugs that are directly relevant to your products, which will serve as justification later on.

Overall, I would recommend running a rudimentary fuzzer for a quick assessment. From there, I’d keep tweaking based on what’s found. This will be where you slowly learn based on what you find. Learn the pros and cons of fuzzing for your organization. But, the big advice I have is: Just start. Fuzzing is neat in that you don’t need permission to do it. It’s a myth that only Google can do it. If you can convince a developer or tester, they can definitely help because developers know the ins and outs of the application and testers are essentially already writing fuzzers.

Applications are able to become more sophisticated and modern when they’re fuzzed. If you’re not fuzzing or doing DAST, you’re behind the curve not just from a security standpoint, but also an innovation standpoint. So, just start fuzzing.

——-

Want to hear more from Billy Rios? Check out this FuzzCon TV, EP 1: Introduction to Fuzz Testing virtual fireside chat with Billy Rios and other fuzz testing experts in the field.

Want to learn how you can get started with fuzzing? Reach out to one of our security experts here. We’d be happy to help.