Musings on Serverless and Application Security With Simon Wardley

While traveling home from ServerlessConf in San Francisco, I bumped into Simon Wardley and we engaged in an hour long discussion on serverless application security. I found the discussion extremely intriguing and thought it would be great to record an informal interview with Simon, and share it with our audience. So, without further ado, here it is: 

Transcript

Ory: Hello there. Thanks for listening in. This podcast is essentially an informal interview we ran with Simon Wardley.

Sponsorships Available

Sponsorships Available

Sponsorships Available

For those of you living under a rock somewhere, Simon Wardley is a researcher for the Leading Edge Forum, a global research and thought leadership program dedicated to helping large organizations reimagine their organizations and leadership for a technology-driven future. Simon is also lead practitioner for L.E.F’s Wardley Maps advisory service, which helps client anticipate market and ecosystem developments so they know where to go and why. In today’s conversation, we will be 

Hi Simon, it’s been a while since we last talked. What are you up to?discussing Serverless, Serverless, and some more Serverless. But most importantly, how serverless and application security can and should co-exist.

Simon:  I’m good. I was doing a research report from Serverless and that’s all done and dusted with the team, which goes through audits prior to publication to the L.E.F members. Quite an interesting sort of space to look at. Obviously, huge advantages to serverless and there’s a lot of new emerging practices.

It’s interesting when you look at the DevOps population that it actually splits into two factions. One of which is highly positive about Serverless and has considerable experience of it, and one which is highly negative of Serverless but has also little to no experience of it.

It’s a bit like the 2008 world where if you asked IT about cloud, there was that group of people who are a small minority, highly positive about the cloud experience, and the huge majority who are incredibly negative about it and thought the future was building your own virtual data centers.

So exactly the same sort of thing happening at the moment. Of course, in the emerging practice, well they’re emerging at the moment so no one really knows what they are obviously. Practices looking in terms of things like financial flow in applications, does potential security has potential, so new different ways of operating, monitoring, securing systems. All of that is the sort of new ground, new land. So, that’s interesting. It’s growth rate is phenomenal. Yeah it’s good. It has all the characteristics of being a major paradigm shift as well and exponentially growing.

Ory: Hold that thought. I’m going to actually ask you more formally about it. But first of all, I’d like to obviously thank you for agreeing to participate in this, I guess, guerrilla interview. I guess, in a sense it’s not really a formal interview but rather a continuation of this, I think, very interesting conversation we had at the SFO airport, right?

Simon: Yes.

Ory: Okay. I guess, we can start, and I wanted to start by asking you about this tweet I saw from you a while ago where you said that, “If you think serverless is just about tech, you’re very badly misinformed.” Can you elaborate on this a little bit?

Simon: Yes, so one of one of the interesting things is when you look through the history of change, you’ve got the process of how things evolve. So, you start off with the genesis of a new move. You get custom built examples, then you usually get products and rental services, and then it becomes more commodity and utility services. And by that, I mean there’s a change of characteristics related to something. So, you start off with something like as simple as electricity. You get the Parthian battery 400 AD or thereabouts; we’re still arguing whether it is or isn’t a battery.

Then you get, later on, custom built systems like the Hippolyte Pixii for generating electrical power then you get Siemens generators and eventually get Tesla Westinghouse AC utility. And it’s always the shift between those different stages, which makes major changes particularly the shift from product to more commodity utilities. When something goes from basically being a product to a more industrialized form. So, like electricity changed and not only enabled all sorts of new wonderful things to appear like television, radio, computing etc. standalone from the power enabled so much. It also tends to change practices as well. So, you get new forms of organization appearing.

So, that change occurred also with computer. So, you start off 1943 with the Z3, custom built systems like Lyons electronic office. You get to things like the first products, IBM 650. Constant improvement products, and then it becomes more commodity and utility like services. Like when you get that shift in product to commodity and we start seeing a change of practice and so we get the whole DevOps move in the past. If you look through the history, we have things like the American system of engineering, and these are all related to industrialization of underlying technology. 

So, what actually creates that change of practice so what we call coevolution of practice? What was the trigger? And it seems to be that when a change impacts MTTR.  So, if they talk about MTTR as Meantime To Recovery, if I were a company, as events occurring in my ecosystem or my computing environment, matters of bringing in new products or new features of products or there’s some economic change, my MTTR is my ability to recover and regain my position. And if I do it more quickly than anybody else, I gain advantage; if I’m slower I often have a disadvantage. And that depends upon your ability to observe the change orientate around it, decide what to do and act. And it’s that cycle, which is reflected in MTTR. 

So, if I’m really quick at sort of changing and observing, orientation, deciding, and acting, I can actually gain more advantage. If I’m very slow, everybody else gets the advantage over me. And so, when you look at a shift, say computer product to utility, that impacted MTTR in terms of our ability to act, in terms of providing compute resources. We went from taking weeks and months to set up a computer to seconds. So, I could quickly react and of course that changes practices. So, I go from scale up to distributed systems.

I go from disaster recovery test to designing for failure, and the introduction of things like Chaos engines. So, no more of this N+1 type stuff. Trying to make the single, super reliable, machine. I’ve now with distributed systems. I’m constantly monitoring for failure and if a component fails, I’m just recreating that and including that component back in. We can suddenly do continuous deployment because continuous deployment is meaningless if you’re waiting for several weeks or months for the server to turn up. But I mean, there is no continuous deployment. But then it’s like seconds, well then you can. Okay?   

So, you get that change of practice. So, when you look at something like Serverless, which we’re talking now basically the code execution environment shifting from a product stand like Lambda, so it’s much more of a utility. And it has those characteristics of being more of utility. You’re also seeing a change in MTTR. Now it’s not so much the ability to act. And certainly, there isn’t an advantage of in terms of underlying automation. It’s also ability to observe the environment because one of the beauties about serverless environments is billing for function. 

And so, what this means is that in the past, when I wrote on that occasionally, it was often difficult to know how much money that application was actually costing me to run and virtually impossible to know that within that application which consists of the namespace, and other functions, what the cost of those individual functions were. I mean nobody had that sort of information. Well one person did and I’ll explain that a bit later. They are billing for function and now enables you to actually have that information in terms of where I’m spending my money. So, I can basically take my application and break it down into much smaller components.

And I actually see where my capital is flowing in my environment. And the reason why I say the one person is because I built, my company built the first, I mean I was the C.E.O. for a Service Environment back in…

Ory: We’ll get there. I actually have a few questions about Zimki.

Simon: It was called Zimki. And the point is we notice this without any… Because suddenly because we could see the capital flow, we could see the functions, which ones were costing us the most money? So suddenly refactoring was done based upon capital flow. We were monitoring this stuff by capital flow totally changes. So, that’s one of the interesting things about Serverless. We should see an entire new set of practices, which combine development and finance together, and that’s what really, really will make it a big change.

Ory: Okay. So you already touched upon Zimki so I’ll go there. In many of your talks that I’ve seen you mentioned that Fontango developed this Zimki platform, sort of like a platform as a service, right?

Simon: So Zimki was what we would call a platform as a service, it’s a code execution environment where you write your application. The front and backend of your application in one language, JavaScript. We only supported one language. So, you wrote your front and backend all in JavaScript. It had basically a storage service, object storage service and various other message queue services. It was all billing by function.

So, when you wrote your application you could see the cost of the individual functions. Every single function you wrote in JavaScript could be exposed as a web service, very simply. And the actual code itself you just get a template for the expression of the function, and the billing was to the function. So, it was pretty close to what we would call a serverless environment.

Ory: Right, right. And did you put any thought or research into how this platform was to provide security for developers? Was there any security baked into it? Or was it just still exploratory, cutting edge?

Simon: We had thousands of developers building on this as well. I have a security background and my security officer when I was the C.E.O of the company. Certainly, I put some effort into mechanisms, ways of securing this environment. This was 2005. I mean, Amazon was launched in 2006. And of course, we were excited, because underneath Zimki we had a system called Borg, which was our basically, equivalent to EC2, which we are able to convert later to EC2 APIs. And therefore, run Zimki on top of that. And so, it didn’t take us long to do that as well. So, there was certainly some thoughts to security. But remember it’s 2005, 2006. I would hope that there would be vastly more today than there was then.

Ory: Well, I actually asked this because I noticed that even though we’re already after I think at least twenty years of mileage in web application security, it’s still seems to me that most folks prefer to, first for example, learn serverless and build applications, experiment with it. And only then people start remembering that there’s this thing called Security.

Simon: Yeah so a bit of background. I used to be a security officer for Harrods. And I also did some security work with various banks. And so, SQL injection attacks and all this sort of stuff. I mean, you would think that by now they would not exist anywhere. It’s just like the simplest of all things. But they do. I’m always disappointed. And yeah, I do agree that people often… security is sort of added on. Whereas it actually should be an integral part of the process. I mean, with one of the things we have like Zimki. I mean our security officer was involved in the whole process right from the very beginning.

In terms of there were lots of arguments and disagreements and a lot of talking about ways people could exploit the system to attack others etc. In terms of doing actual code scanning, I mean, we were nowhere near that state at that time. Remember we had sixteen different lines of other business and we are a relatively small company as well. So, this 2005, 2006. Obviously, when we’re talking about today in building large-scale systems on Lambda. Then code scanning to get rid of the obvious flaws in what you’re actually building would be hopefully a norm. But, I’m never surprised when I discover that it isn’t. So, this sort of stuff that you’re working on in terms of looking at the inputs and the outputs coming from an actual function and the actual flow of the function. I hope it will become more baked into systems.

Ory: I think personally that when it comes to experimenting with new technologies, engineers or people who like technology, and even the most security-minded ones have a tendency to sort of procrastinate or always postpone anything related to security.

Simon: Okay. So it’s an interesting choice of the word new. Because it’s not really so much new as a transition. Because we’ve had the products back. In terms of Lambda and .NET etc. We’re really shifting to a much more—it’s provided as a utility. So, it’s a transition. But it’s not as if the code execution environment, the idea of a platform. It’s not like Lambda has invented that. But it has existed for an awfully long time. But now, it just provides a more utility form. So, you’d hope that the lessons learned will be translated. But also, because it is a transition, there are different set of practices which will now appear and will be required.

I mean you’re doing security to the function and that’s very different from doing security to just simply the application. And so, there’s a whole new set of factors. I’d hope that some of the sensible practices in the past would translate into this transition from product to utility. Unfortunately, past experience tells me that people will do the same sort of things they’ve done in the past like for example in the early days of cloud, people would try and take entire legacy of states. And shift them onto something like the EC2 and expect it to operate like their data center did. I mean that’s not going to happen. You have to check. 

Ory: So we published actually like a serverless top ten security guide. Hopefully developers will read this before starting to migrate or develop new applications. So, on the topic of DevOps and you touched upon it a lot. And we actually talked about it in the airport when we met. It seems to me that with the shift to function as a service, or serverless, developers now are responsible for designing and deploying the code. And they don’t depend really on any external, traditional team in the IT sense.

Traditional IT security teams, or IT operation teams. In fact, one would argue probably that organizations are about to experience or have a lot of what I referred to as Cloud Shadow IT. Where people sort of post or upload unregistered or ungoverned functions to the clouds. Any thoughts on that? How would you plug the traditional infosec teams into this process and make sure the developers are sort of following corporate, security best practices?

Simon: Well, first of all, the interesting thing about security best practices is those practices themselves will have to change. With the shift in product to utility of the code execution environment. That’s just the typical coevolution of practice. There’s an entire new sets of ways of securing that we’ve got to learn about, and also new vectors of attack that we’ve got to learn about. So, the past best practice, there will be some of it kept, and some of it will need to change because of the changing technology landscape. So, that’s the first.

The second thing is monitoring and securing doesn’t go away just because you’re using serverless. It’s just a different form. I’m no longer concerned about tin. I’m concerned about, say if I’m monitoring by capital flow, when money is moving in my I.T. estate, that still needs monitoring but it’s a different set of monitoring. And they’ll be changes therefore to the monitoring operations as a security. They won’t disappear, they still exist, but I’m not concerned about tin anymore.

All the lower order systems become less visible. In the same way that most companies had a “president of electricity” at some point in the past. Most companies don’t need that anymore. That function is being replaced, is being changed, but they still have in large companies – lots of involvement with electrical engineering. Whether it’s the use of electricity in different ways, in terms of lighting or whatever it happens to be as well. So, it’s just there will be a change of practice. It doesn’t go away, there will be a change of practice.

Those practices are emerging so we’ve got to learn about them at the moment. I mean how do you include with the current…? You get involved. An event like serverless, what I would like to see at a serverless event…if the company is sending… they don’t just send the developers. Send somebody from security, send somebody from operations, and send somebody from finance. Send your financial controller, because they’re all going to have to learn and work together. Now the combination is very much strongly between finance and development. This is what I suspect is going to happen. But you still need to operate and you still need to secure that environment.

Ory: So for many years there seems to be an attempt to push security upstream, or some call it shift left, and have developers build and test secure code. However, due to the existence of traditional IT security teams’ involvement, security solutions were still sort of bolted on after the fact. Do you think that since serverless leaves us with only control over the code, we now have an opportunity to finally bake intelligent security into our applications rather than rely on external devices like firewalls? 

Simon: Well, first of all, that sounds great and obviously, I would like that to be the case. The one counter to this is—when I used to work, you don’t have to tell me what it’s like today. But when I worked in the security industry, which was a long time ago, there was a certain type of security officer who was like, my role is to get involved, educate, and help people. And that is the sort of thing that’s needed in the serverless world.

But then there was another type which I need to monitor and look for breaches, and they didn’t want to get involved. They didn’t want to get involved because there was this attitude that if I got involved, and it went wrong, I’m now responsible. And it’s actually easier to be on the outside. Just like, yeah, I’m just going to, it’s like a critics; I don’t want to get involved in creating the film. I just want to be a critic at the end because that’s an easier job to do. I have to say it’s rubbish.

Ory: They usually turn into naysayers. They say no to anything you ask.

Simon: Well this was happening then. And it was quite some time apparently. I’m hoping the security industry as a course obviously progressed a lot since then and is very, very focused on the positive side. In terms of getting directly involved and so serverless should allow for new emerging security practices. So, I don’t see it as a negative for security I see it as positive.

As an opportunity for security professionals to get involved in the space with the development teams that are learning about them in practices and developing those, by bringing past practices and translating them. To this more transformed sort of utility world. I see it as a positive, except for the people who want to be gatekeepers. Who want to sit there just going no, no, no. So, they’re going to hate it but then, well.

Ory: Okay, another topic I wanted to chat about was—and this is somewhat of a sensitive topic but it seems like the modus operandi for generating awareness around application security, and around cyber security in general. Until today, was to either find a very big, very public disaster and use that? Like the Equifax fiasco for example, or by using scare tactics to influence the decision makers. That was how traditional security vendors used to work.

Simon: That is also how traditional vendors, the use of fear, uncertainty and doubt is how traditional vendors also try and stop their clients, or customers from adapting to a new world which they’re not actually involved in. But in some cases, it’s used to make people panic about things in terms of, well I’ve got to take care of this, and in some cases, it’s used to make people panic about something that’s changing. There’s lots of games played in the space.

Ory: So where I’m taking this is, with everything we’ve seen until today and you’ve seen a lot when it comes to security breaches and things like that and the way software security evolved in the past twenty-thirty years. To me it seems like serverless is again, starting on the wrong foot. It seems like developers are rushing to build and deploy stuff without paying much attention to security. I’ve been interviewing prospects and customers and they have hundreds of functions already deployed and just now they’re starting to think about, wait a second, these functions are facing the Internet and I’ve done nothing to secure them. Any idea why is that?

Simon: To some extent, that’s always human nature, isn’t it? It’s a bit like the automobiles. The rush, to that “IP enabled”. Some may have interconnection, Internet connection on my car. And the thing is the car is controlled by engine control units, they have their own protocol for communication. But between those different engine control units it has never been exposed to the Internet before. So, that suddenly enabled my car on the Internet. Wouldn’t that be a cool thing to have? No!.

Ory: We’ve done it much earlier when we took mainframe COBOL applications, banking applications and using Java, connected to the Internet and now, those are now on the internet right?

Simon: It’s like somebody was saying they’ve got this IoT oven and because of some problem, they can’t now switch it off except for the mains because some service is being hacked into. I mean that’s an oven that could burn down a house. I’m just waiting for the IoT enabled pacemaker. Let’s just connect that to the Internet, I’m sure somebody’s done it. There are all sorts of areas where we tend to rush into stuff. And security is an afterthought.

I wish that wasn’t the case. I wish it, so I’m hoping in the serverless world, in the move towards…you don’t want to stop the move to serverless and there’s a lot of exploration and one of the things about that exploration is about the practices. So, the reality is we don’t actually know what the security practices for the serverless world should be. We are still learning about those practices around that space.

So, there’s going to have to be a bit of gambling and a bit of experimentation. But you’d hope that some of those things which are sensible, obviously sensible. Things like not having exposing yourself to SQL injection attacks. Simple things like that will actually be in place. I know they won’t. But you’d hope so. So, there’s always a bit of experimentation; otherwise, there’s always a bit of risk. I mean security at the end of the day is about balance and risk. There’s the risk of being attacked. And of course, there is the risk of doing nothing and allowing your competitors to get to gain advantage. And you’ve always sort of rebalancing. It’s a hard job balancing those risks. Sometimes you have to be no but a lot of the time you have yes, and well the time you should be pushing a company. So actually, from a security point of view I would probably argue that security should be banging on at the company about going serverless, and about learning those practices. Because that is where the world is going to be going. So, might as well get involved, learn about those emerging practices. Take what we’ve already got, apply what is sensible, realize that not everything is going to be perfectly working in this transitional stage, so it’s going to be a bit of heartache and all the rest of it as well. So, there is increased risk there, but there’s also the risk of doing nothing as well than what our competitors are doing in that space. So, with my security officer hat on in the past I would probably be encouraging adoption.

Ory: It’s interesting because now that we broke, in serverless, if we broke everything into functions, very similar to how you say that now it’s a matter of developer and finance working together, might be similar for security. Now I don’t have to apply security for my entire application, everything is broken down into functions. I can prioritize where I want to invest my resources and money securing things, and where you know I have things that I don’t really want to secure, or the risk is not worth the cost of securing the function.

Simon: Totally agree in terms of involving security but the difference between dev and finance and dev and security, is that dev and finance, because of that change of MTTR, Meantime To Recovery, your ability to observe, your ability to act. And that is the thing which fundamentally changes practices, even changes organizational structure. That’s the big thing that really turns it into a large paradigm.

So, it’s the same with computing from product to utility. I mean you take operating systems, they work from products more of a commodity sort of… but this doesn’t have a big change to MTTR. It’s a bit like containers as well, again not a huge change to MTTR but computing from product to utility or code execution product to utility, big change to MTTR. And the observability is the big thing as well. And so that’s why I talk about dev and finance together. Obviously, security should be part of that as well, but the security doesn’t impact MTTR in the same way.

Ory: But given that security is basically a risk versus how much is this going to cost me if somebody hacks into my application, what would be the loss? I have a more granular way of handling security and deciding where, and when I want to apply it.

Simon: Absolutely. So, we’re going to see a change of practice related to security and serverless. Absolutely true.

Ory: Okay. That’s all the topics I had for today.

Simon: My pleasure.

Ory: I’m really glad we had the chance to continue the discussion. We still have to maybe have a discussion around how to abuse–protest signs and what was it? Drones.

Simon: I love that idea. I mean they’re doing SQL injection attacks through protest signs. Because of some sort of camera and recognition software and conversion to text and some sort of function built which doesn’t actually—I mean that example it’s just perfect, isn’t it? I mean you know that somewhere that is going to happen.

Ory: And so, I promise that when I produce that demo video. I’ll allow you to be there and stand with the sign that crashes the drone.

Simon: Absolutely. Absolutely. Yeah I definitely want to hold the, oh yes I would be delighted to hold the crash-drone-sign. But again, that’s one of the beauties about this sort of changes. So, it’s like with electricity shifting from product to commodity. We get this explosion of new things. Like television and radio, computing, etc. becomes possible. And they create new opportunities in the world. But they also create new forms of risk associated with it as well. And as they evolve and they shift from product to commodity they cause an explosion of higher order systems like computing. We’ve got this wonderful new stuff being created at phenomenal speeds.

But of course, they should create new sorts of risks. Our past risks maybe that’s a little bit on why is in certain circumstances particularly to do with power supply. But they create also new risks as well.  So, it’s always a fascinating world, particularly the shifts from product to commodity. And what drive the major revolutions, the industrial revolutions, the agricultural, the oil revolution, the automobile. All of these big shifts are connected to that and they’re all related to new practice as well. So, it’s an exciting time. It was exciting with cloud in terms of infrastructure. It’s exciting now, but in terms of serverless.

Ory: Yeah, what you call risk I think I call it a challenge.

Simon: Well, it’s also an opportunity as well.

Ory: Okay. Thank you very much for this.

Simon: My pleasure, delight speaking to you. Take care.

Ory: You too, bye, bye.

Simon: Bye, bye.

Ory: Those were musings on application security and serverless with Simon Wardley and yours truly, Ory Segal. Don’t forget to check our website https://www.puresec.io