If Shift Left Is Dissolving, How Should Security React?
For years, the idea of “shift left” has been a guiding principle for application security testing. The philosophy aims to discover bugs early in the development process by automating code analysis within development and staging environments. However, shift left is only one aspect of security testing—maintaining stable platforms requires a continuous view into production runtimes to anticipate real-world settings and improve response times. Plus, as development cycles are shortened and teams move away from waterfall approaches, not much “left” remains.
I recently chatted with Rickard Carlsson, co-founder and CEO of Detectify, to understand why shift left is dissolving and how security engineers can respond. According to Carlsson, the new continuous, progressive delivery style doesn’t mesh well with the old security playbook. To fit into this current development cycle, Carlsson believes teams must shift their thinking toward continuous testing and evolve the metrics used to gauge their security response and performance.
There’s Little “Left” Left
“There is no left,” says Carlsson. When fully integrated development teams run, test and deploy, they don’t really have much code waiting around in “left,” he explained. With continuous deployment, production may only be five hours or even 15 minutes away from the staging environment. From a visual point of view, the shift left concept is relevant only for waterfall releases, said Carlsson.
Furthermore, progressive delivery is also complicating the traditional release cycle, meaning that what appears to be “left” to one subgroup may be completely different from another point of view. These rapid iterations make it possible to quickly test for bugs in small subsets of user groups or introduce personalized features. The takeaway is that production is changing much more iteratively and rapidly than ever before.
“If we’re just pushing a change to 1% of users, is that production? Or a modern version of staging?” Carlsson asked. In this new paradigm, talking about shift left feels like an old analogy. This feeling is driven home by the fact that testing code in isolation rarely yields real-world results—to get complete insight into the implications of new changes, you really need to understand how they behave in production. Developing with feature flags and exposing A/B testing is where we’re going anyways, noted Carlsson.
The New Security Response Metrics
If development is shifting toward iterative tests in production environments, then what follows is more frequent remediation responses. Thus, the metrics used to gauge overall security performance changes slightly. It follows, then, that the industry has embraced remediation speed as a top concern.
Carlsson shared some metrics to consider to gauge the performance of a security culture in this new era of agile development:
- Number of staging versus production bugs: How many bugs were found in staging versus production? When you find something in production, how fast can you fix it?
- Maximum time to respond: Mean time to recover (MTTR) is a helpful metric, but charting the maximum time to respond is perhaps a better metric to watch. This could reveal large bottlenecks that impact a vast number of users for extended periods of time. If the maximum time to recovery is longer than a few hours, teams need to find the root cause to avoid future incidents.
- Criticality: How broad is the impact? Security monitoring systems infamously generate a lot of noise. This can hide important alerts and cause burnout. To avoid false positives, teams will require a system that rates vulnerabilities on their severity and routes such information automatically, Carlsson explained. Thus, you can still have wide observability into all issues while privileging severe incidents. “The more precise you can be in testing, the better you can be,” said Carlsson.
As you can see, much of modern security response hinges on speed. This is because when there’s a bug exploited in the wild, you typically only have a few hours to respond, noted Carlsson. Black hats use automation to crawl the web for open computing environments and known vulnerabilities, meaning it’s only a matter of time before every exploit is discovered.
And unfortunately, even if the organization is aware of the vulnerability, it doesn’t mean they will respond quickly. Take the infamous Equifax hack as a prime example. Equifax knew about the Apache Struts vulnerability for months yet failed to fix it. The longer the exploit was exposed, the more time bad actors had to access servers and exfiltrate data. Thus, what really matters is decreasing the fix time—from days to hours down to minutes, if possible.
You Own What You Ship
Hopefully, with iterative changes, faster remediations will be realized. This could allow even large companies to ship patches with greater alacrity. But another method to enable quick fixes is the idea of full-cycle development. Full-cycle development can be thought of as “if you ship it, you own it.” In theory, a full-cycle developer would oversee more DevOps/life cycle management responsibilities for a smaller footprint of applications.
In the old waterfall development world, security acted as more of a controller, said Carlsson. Security reviews were a gate that required long windows for review. Yet, modern engineering consists of diverse teams with various domains fully integrated. The old security check process doesn’t work with a modern way of building, noted Carlsson. Instead, in full-cycle development, engineers are responsible for responding to new threats as they emerge.
Of course, many DevOps tools now crowd the market, and relying on a single developer for security, testing, validation, performance, deployment, cost optimization and more for their code could be a lot to ask. For a DevSecOps culture to emerge, “Security needs to act as enablers,” said Carlsson. This will inevitably involve optimizing the handshake between programmers, SREs and traditional IT security roles to decide how an organization introduces new tools and processes into its workflow. It will also require alignment on setting an acceptable timeline for responding to events.
Security is On the Radar
“Security is always playing catch up to the new trend of the day,” said Carlsson. New, trendy languages pop up and we see the same vulnerabilities from the older language now must be resolved in the newer ones. Or, take the web development trend; from cookie handling to JSON Web Tokens (JWTs) and now, back to cookies. Now, people are building zero-trust architectures, yet microservices handlers can suffer from data overexposure if not properly architected. In short, as long as technology evolves, new vulnerabilities will continue to emerge.
Thankfully for security personnel, security is now on the radar. It’s gone from a niche field to being reported on by mainstream media on a weekly basis. The rise of global data regulations is necessitating more investment in compliance tooling and practices. And, the increasing cybersecurity threats are making security a pressing concern at the executive level. With increasing digitalization, we see more and more placed in digital value chains, prompting executive attention.
With so much digital reliance, security vulnerabilities can now affect massive amounts of people at a catastrophic level. “It’s the only disaster not bound by geographical scale,” noted Carlsson. For example, take the City of Baltimore, which in 2019 shelled out over $6 million to recoup losses from a ransomware demand. This hack hurt millions of taxpayers and robbed them of a budget for parks and public facilities. International cyberwarfare is known to be even more severe.
Now that the headlines are populated with many recent hacks, organizations are starting to think a bit more about how to better protect their sensitive data and infrastructure. Shorter release cycles, quicker remediation windows and an emphasis on full-stack ownership could help continuously monitor and adapt to new threats.
