How to Cut Through Vendor Claims & Marketing Hype When Evaluating New Security Tools
As we’ve pointed out in a couple of recent blog posts, Machine Learning (ML) has been billed as a savior for short-staffed security teams — a silver bullet that can single handedly identify and mitigate every security threat automatically. As we usually do with silver bullet solutions, we’ve cautioned readers to distinguish between the hype and reality. While ML has many strengths and is here to stay, it’s only a part of the solution in the world of cybersecurity — not the solution itself. Human input is still essential to draw meaningful conclusions and define appropriate action.
In today’s post, we’re continuing to advise readers that it’s essential to go below the surface, to distinguish between the hype and reality, when evaluating a cybersecurity solution. Remember: A beautiful package may open up to reveal a beautiful can of worms. Keep your eyes open, investigate below the surface, and avoid nasty surprises.
Abstraction permeates the computing field, including security. Each abstraction layer makes the problem easier to conceptualize and reason about, but it also hides details and can make investigating deviances from the abstraction difficult. Machine Learning (ML) represents a powerful abstraction of programming via statistics and patterns. Like other abstractions, it makes the problem easier to think about: “The computer is learning how to handle this data.” Also like other abstractions, it makes deviances hard to debug: “Why did we miss that one?” Even if users have the propensity to investigate into the model, proprietary methods often prevent peering behind the curtain and hand tuning models.
We live in a world that tries to give more and more of our agency to computers. We curate our feeds with thumbs-up or thumbs-down, and hopefully that improves the content. Our attempts to tune the machine learning algorithm do not always bear fruit. Sometimes I still get horrible songs on Pandora. Facebook still shows me things I do not care about. I’ve only tuned Pinterest a little, so it shows me all sorts of weird things. Sometimes it’s funny, sometimes it’s frustrating. In a security setting it can be dangerous.
Take a moment to consider the motivations behind looking for a security solution. Now consider how vendors sell their products. Do those two align? Unfortunately, the money grab for funding leads to flashy demos and hyperbole, both from vendors and from internal stakeholders. The necessity of this presentation is nearly a fait accompli, but flashy demos and hyperbole are often not the goal for a security solution. This means that a lot of time and effort goes into making a fancy cover. Defenders forget the saying “Don’t judge a book by its cover,” and this can lead unscrupulous advocates of security solutions to spend inordinate amounts of time on the cover compared to the substance inside.
Enumerate some of that substance a defender wants from a security solution:
- Detect security incidents
- Achieve compliance
- Provide visibility into systems
- Support infrastructure in transition
- Operate at speed and scale
- Protect past, present, and future innovation
A demo can and should illustrate the capabilities mentioned above, but it can also sweep weaknesses under the rug. Also note that the various items listed above do not include the “how” of the solutions. Evaluating the “how” can help us make sure the solution adequately deals with the “what,” but it does not supersede the “what.” Whether a solution uses machine learning or the blockchain becomes irrelevant when compared to evaluating whether it meets the needs of the user.
We have a beautiful cover on our product here at Threat Stack, but it also provides the goods. Our platform has many powerful features that demo well, but we’re not just a whited sepulchre. We find that while our customers appreciate the sleek UI, they find the most value in the data we provide. That data can get accessed through the app, but also via our REST API, and our S3 bucket export. We do not lock your data inside a black box.
A Few Final Words . . .
Security will always require human expertise to one degree or another. If you’re looking to supplement your in-house security team, we’ve created the Threat Stack Cloud SecOps Program℠ , a series of services that are designed take the investigation workload off the organization and provide recommended actions based on contextualized insights from our Cloud Security Platform®.
The post How to Cut Through Vendor Claims & Marketing Hype When Evaluating New Security Tools appeared first on Threat Stack.
*** This is a Security Bloggers Network syndicated blog from Blog – Threat Stack authored by Nathan Cooprider. Read the original post at: https://www.threatstack.com/blog/how-to-cut-through-vendor-claims-marketing-hype-when-evaluating-new-security-tools
