No, 1,000 engineers were not needed for SolarWinds

Microsoft estimates it would take 1,000 to carry out the famous SolarWinds hacker attacks. This means in reality that it was probably fewer than 100 skilled engineers. I base this claim on the following Tweet:

When asked why they think it was 1,000 devs, Brad Smith says they saw an elaborate and persistent set of work. Made an estimate of how much work went into each of these attacks, and asked their own engineers. 1,000 was their estimate.

— Joseph Cox (@josephfcox) February 23, 2021

Yes, it would take Microsoft 1,000 engineers to replicate the attacks. But it takes a large company like Microsoft 10-times the effort to replicate anything. This is partly because Microsoft is a big, stodgy corporation. But this is mostly because this is a fundamental property of software engineering, where replicating something takes 10-times the effort of creating the original thing.

It’s like painting. The effort to produce a work is often less than the effort to reproduce it. I can throw some random paint strokes on canvas with almost no effort. It would take you an immense amount of work to replicate those same strokes — even to figure out the exact color of paint that I randomly mixed together.

Software Engineering

The process of software engineering is about creating software that meets a certain set of requirements, or a specification. It is an extremely costly process verify the specification is correct. It’s like if you build a bridge but forget a piece and the entire bridge collapses.

But code slinging by hackers and open-source programmers works differently. They aren’t building toward a spec. They are building whatever they can and whatever they want. It takes a tenth, or even a hundredth of the effort of software engineering. Yes, it usually builds things that few people (other than the original programmer) want to use. But sometimes it produces gems that lots of people use.

Take my most popular code slinging effort, masscan. I spent about 6-months of total effort writing it at this point. But if you run code analysis tools on it, they’ll tell you that it would take several millions of dollars to replicate the amount of code I’ve written. And that’s just measuring the bulk code, not the numerous clever capabilities and innovations in the code.

According to these metrics, I’m either a 100x engineer (a hundred times better than the average engineer) or my claim is true that “code slinging” is a fraction of the effort of “software engineering”.

The same is true of everything the SolarWinds hackers produced. They didn’t have to software engineer code according to Microsoft’s processes. They only had to sling code to satisfy their own needs. They don’t have to train/hire engineers with the skills necessary to meet a specification, they can write the specification according to what their own engineers can produce. They can do whatever they want with the code because they don’t have to satisfy somebody else’s needs.

Hacking

Something is similarly true with hacking. Hacking a specific target, a specific way, is very hard. Hacking any target, any way, is easy.

Like most well-known hackers, I regularly get those emails asking me to hack somebody’s Facebook account. This is very hard. I can try a lot of things, and in the end, chances are I cannot succeed. On the other hand, if you ask me to hack anybody’s Facebook account, I can do that in seconds. I can download one of the many hacker dumps of email addresses, then try to log into Facebook with every email address using the password “Password1234”. Eventually I’ll fine somebody who has that password — I just don’t know who.

Hacking is overwhelmingly opportunistic. Hackers go into it not being sure who they’ll hack, or how they’ll hack. They just try a bunch of things against a bunch of targets and see what works. No two hacks are the same. You can’t look at one hack and reproduce it exactly against another target.

Well, you reproduce things a bit. Some limited techniques have become “operationalized”. A good example is “phishing”, sending emails tricking people into running software or divulging a password. But that’s usually only the start of a complete attack, getting the initial foothold into a target, rather than the full hack itself.

In other words, hacking is based a lot on luck. You can create luck for yourself by trying lots of things. But it’s hard reproducing luck.

This principle of hacking is why Stuxnet is such an incredible achievement. It wasn’t opportunistic hacking. It had a very narrow target that could only be hacked in a very narrow way, jumping across an “airgap” to infect the controllers into order to subtly destabilize the uranium centrifuges. With my lifetime experience with hacking, I’m amazed at Stuxnet.

But SolarWinds was no Stuxnet. Instead, it shows a steady effort over a number of years, capitalizing on the lucky result of one step to then move ahead to the next step. Replicating that chain of luck would be nearly impossible.

Business

Now let’s talk about big companies vs. startups. Every month, big companies like Apple, Microsoft, Cisco, etc. are acquiring yet another small startup that has done something that a big company cannot do. These companies often have small (but growing) market share, so it’s rarely for the market share alone that big companies acquire small ones.

Instead, it’s for the thing that the startup produced. The reason big companies acquire outsiders is again because of the difficulty that insiders would have in reproducing the work. The engineering managers are asked how much it would cost insiders to reproduce the work of the outsiders, the potential acquisition candidate. The answer is almost always “at least 10-times more than what the small company invested in building the thing”.

This is reflected by the purchase price, which is often 10-times what the original investors put into the company to build the thing. In other words, Microsoft regularly buys a company for 10-times than all the money the original investors put into the company — meaning much more than 10-times the effort it would take for their own engineers to replicate the product in question.

Thus, the question people should ask Brad Smith of Microsoft is not simply how many skilled Microsoft engineers it would take to reproduce SolarWinds, but also how many skilled Microsoft engineers it would take to reproduce the engineer effort of their last 10 acquisitions.

Conclusion

I’ve looked at the problem three different ways, from the point of view of software engineering, hacking, or business. If it takes 1,000 Microsoft engineers to reproduce the SolarWinds hacks, then that means there’s fewer than 100 skilled engineers involved in the actual hacks.

SolarWinds is probably the most consequential hack of the last decade. There are many eager to exaggerate things to serve their own agenda. Those types have been pushing this “1,000 engineer” claim. I’m an expert in all three these areas, software engineering, hacking, and business. I’ve written millions of lines of code, I’ve well known for my hacking, and I’ve sold startups. I can assure you: Microsoft’s estimate means that likely fewer than 100 skilled engineers were involved.

*** This is a Security Bloggers Network syndicated blog from Errata Security authored by Robert Graham. Read the original post at: https://blog.erratasec.com/2021/02/no-1000-engineers-were-not-needed-for.html