SBN

Lessons from the Massive UK Power Outage

In the midst of rush hour on Friday, August 9, 2019, the power went out London, and in large sections of England and Wales. This blackout caused major disruptions.

The Telegraph (UK) reported: “People at the ­hospital said that parts of it were left in “complete darkness”, with sirens going off when the power cut hit.

A hospital spokeswoman said it was “too early” to say whether there was a fault, but assured staff ‘helped keep patients safe’ during the power ­outage.

Rail commuters experienced ­disruption and delays, while motorists found some traffic lights out of action. 

Around 300,000 homes and ­businesses were affected in London and the South East, a UK Power ­Networks spokesman said.

Western Power ­Distribution said around 500,000 ­people were hit in the Midlands, South West and Wales, with power restored shortly after 6pm.

Northern Powergrid, which serves Yorkshire and the North East, said 110,000 people lost power. …”

According to the BBC:  “The power outage happened at about 17:00 BST on Friday, National Grid said, with blackouts across the midlands, the south east, south west, north west and north east of England, and Wales.

National Grid said its systems were not to blame and the outage was caused by the loss of two generators.

Industry experts said that a gas-fired power station at Little Barford, Bedfordshire, failed at 16:58, followed, two minutes later, by the Hornsea offshore wind farm disconnecting from the grid. …”

The disruption caused chaos and anger from residents and politicians alike, who demanded an investigation into the incident.

Not a Cyber Attack – According to Authorities

The Daily Mail (UK) announced that the government has launched an investigation into causes of this blackout.

“Fury at power cut that brought Britain to its knees: Government launches probe into mystery simultaneous failure of wind farm and gas-fired power station as officials insist there is ‘no evidence’ of a cyber attack

  • Blackouts were reported in London and the South East yesterday, as well as Midlands and the North East
  • The chaos was caused by a gas-fired power station and a wind farm both going off-line yesterday afternoon  
  • Trains across the country ground to a halt as a result of the power loss, which caused knock-on delays
  • Energy watchdog Ofgem has called for an urgent report from the National Grid about yesterday’s chaos 

According to another source (Euronews.com): “The power outage experienced in the United Kingdom by thousands of homes on Friday was not caused by a cyberattack, the electricity transmission network of Great Britain said on Saturday.

Duncan Burt, operations director at National Grid, said the power outage occurred when two power stations failed almost at the same, leading the system to cut off power in some parts of the country in order to preserve the rest.

He said the company was ‘very confident that there was no malicious intent or cyberattack involved’ but added that the loss of two power plants was a ‘very, very rare event’ and that the last time something similar happened was in 2008.”

But members of LinkedIn’s Information Security Community were not so sure and expressed fascinating opinions and scenarios about this power loss incident. Here were some of their comments on Saturday after the event:

  • David O’Berry (Co-Founder, VP, CISO, Chief Product & Security Innovation Officer at PreCog Security – and former South Carolina Security lead): “I am not a big believer in ANY explanation given by Big Government or Big Biz at the moment…it’s just hard to trust knowing current state in these various CIP areas.”
  • Frank W. Holliday (Senior Data Protection & Information Security Consultant at tacticx Consulting GmbH): “Two electric generators with malfunction at once were the communicated reason for the blackout of Midengland’s electricity. Well it was just a real blackswane event or also a plausible just exercising cyberattack squad for the even bigger target (U.S. electricity nets). I wish for the blackswane event.”
  • Matthias Maier (Matthias has a account EMEA Director of Product Marketing for Splunk): “I was surprised how 10-15m power outage can create across a country so much chaos… at least the news made it to one… in other countries are more often power outages and people keep calm and if train signals and other processes do not work properly after a power outage it shows the need for more regular planed power outages to get less vulnerable long time… disaster recovery…
  • Caston Thomas (President, Interworks, LLC and Co-Host of the Internet Advisor Radio Show – WJR Detroit):  “Skeptics’ viewpoint… ‘Bears are attracted to honey. Would they ever admit that they are vulnerable?’”
  • Kevin Shane Fitzgerald (Director of Business Development for Excipio Consulting and Creator of the Fitzgerald Foundation): “Penetration testing. Lets see how they react if we do this….ok…duly noted…now…lets add this….ok, i see ….what if we do this..ok, then they do this…and then this…and then this. If this then go to…… programming…engineering…reverse engineering. Business by business…block by block…city by city…. Grid by grid…. Seems…to Happen more n more now don’t it?”
  • Dan Walsh (Principal, Act 1 Security): “Not caused by cyber attack does not mean not caused by a cyber event. Two power plants, separated by many miles and utilizing different generating technologies go down simultaneously and are offline for <30 minutes – definitely digital wrongdoing. What we can we learn: promote these events as evidence of our ever-increasing dependence on digital – the what/how matters more than the who.”

What Immediate Lessons Can We Learn?

So what can we learn from the events this past week? A probe into the UK National Grid promised to learn lessons from the blackout. According to the BBC:  “Regulator Ofgem has demanded an ‘urgent detailed report’ into what went wrong.

It said it could take enforcement action, including a fine, after train passengers were stranded, traffic lights failed to work and thousands of homes lost power during the blackout.”

Here are a few initial takeaways:

Lesson 1 – Are Effective Backup Plans In Place?

According to experts, backup systems worked well. Two power stations disconnected from the grid “near simultaneously”.

Mr Burt said: “What happened then is our normal automatic response mechanisms came in to help manage the event, but the loss of power was so significant that it fell back to a set of secondary back-up systems which resulted in a proportion of electrical demand across the country being disconnected for a short period to help keep the rest of the system safe.”

He added: “Those events happened very, very quickly, in a matter of a few seconds, maybe a couple of minutes maximum.

That sequence of events is entirely automatic, we think that worked well, we think the safety protection systems across the industry on generators and on the network work well to secure and keep the grid safe.”

Lesson 2: Prepare Now For The Inevitable

There seems to be a huge fascination in this country for when a “Cyber 9/11” or whether a “Cyber Pearl Harbor” is coming. Many experts have predicted that it will happen at some point, but in the meantime, we can learn a lot from major incidents like this.

This incident is another wake-up call. We can learn from how people reacted. Test incident response assumptions. What if the power was out even longer or over a wider area?

I find this report very helpful reading from the Insurance Journal in 2013 about the Blackout of 2003. Here’s an excerpt:    

“In its final report on the causes of the blackout, the U.S.-Canada Power System Outage Task Force identified poor vegetation management, computer failures, inadequate training and lack of real-time situational awareness of grid conditions as the main factors behind the disaster.

First Energy was harshly criticized, but the task force identified institutional failures across the industry, particularly in setting and enforcing reliability standards, and coordinating across the grid. No fewer than 46 recommendations were made to prevent the blackout recurring (“Final Report on the August 14, 2003 Blackout” April 2004).”

 Hopefully, a similar report will be produced about this outage. In the meantime, review (or develop) a Cyber Disruption Response Plan.

Lesson 3: We Are Vulnerable & More Power Outages Are Coming

We know that hurricanes, ice storms and other natural disasters cause power outages every year. Utility companies plan for these scenarios and practice recovery efforts.

But something different is happening lately, and more unexplained outages are happening at airports, in major cities and even country-wide around the world. We know that some of these are the result of hostile attacks against critical infrastructure, even if this was not what caused this U.K attack.

What is clear is that much more needs to be done globally in this area, and we need to care and make this a top priority.     

Whatever the causes, the data shows (from several studies) that power outages are becoming more frequent in the USA as a result of weather, aging infrastructure and additional factors.  Other studies show the same trends worldwide.

Final Thoughts

  Even assuming this power outage was not caused by any type of cyberattack, we must pay close attention to details before, during and after the incident.  

I am very interested in the reactions from the security community, and the lack of trust in government and industry is already appearing when these type of situations occur with critical infrastructure outages. How will the public respond in an even worse emergency? 

There is no doubt in my mind that bad actors watched closely when this happened and took notes regarding public reactions, communications capabilities, response times and much, much more. Thankfully, the duration of this outage was fairly short, unlike the Blackout in Michigan in 2003, which I describe here and here.

Nevertheless, as bad as these incidents are, none rises to the level of a major multi-week or even multi-month blackout on a wide scale. The closest thing we have seen to that scenario is the hurricane devastation in Puerto Rico.   

Are we prepared for more disruptions? Only time will tell.