Very few NERC entities are taking advantage of the cloud for workloads originating on the operational side of the organization. There are two reasons for this:
1. NERC entities with high and medium impact BES environments believe (with ample justification) that they will violate multiple CIP requirements if they implement or utilize BES Cyber Systems (BCS), Electronic Access Control or Monitoring Systems (EACMS), or Physical Access Control Systems (PACS) in the cloud.
2. Many entities don’t believe their systems will be secure in the cloud, because of risks that only affect cloud-based systems. Some that believe this also believe that no BES systems at all, no matter who owns or operates them, should ever be implemented in the cloud (I’m sure lots of people outside the NERC community would say this as well. After all, every man, woman and child in North America – except for off-the-grid types in cabins in Montana – has a stake in the security of the power grid!).
Regarding the second point, it is impossible to assert unequivocably that it is safe to implement systems that monitor and control the power grid in the cloud. My only answer to someone who believes this is, “You have a valid concern. The only way to decide this is for the power industry to a) identify the major risks posed by implementing control systems in the cloud, and b) work with cloud service providers to identify mitigations for those risks.” See below for more on this topic.
Regarding the first point, my answer was previously, “You’re correct. Nothing in the current CIP standards forbids use of the cloud. However, if a NERC entity with high or medium impact BES systems wishes to implement or use BCS, EACMS or PACS in the cloud, their cloud service provider (CSP) will never be able to furnish them with the evidence they need to prove compliance with each of the 50-120 applicable CIP Requirements and Requirement Parts (about 120 apply to BCS, about 70 to EACMS and about 50 to PACS). Without that evidence, the NERC entity will probably be found to be out of compliance with at least some of those requirements. Since I don’t see any movement to change that situation, I don’t recommend you plan on ever being able to use the cloud.”
However, things changed. One change is that software vendors have come to realize that, by making their software available as SaaS (software as a service), they can offer better performance, upgradability, and resilience than they can for their on premises versions. Even better, they can do all of this for a much lower end user cost. This is mostly because when software moves to the cloud, activities like hardware maintenance, patching, and upgrading are assumed by the SaaS provider, who can provide them at much lower cost than the end user could. In fact, someone from a major software vendor, with many large clients in the power industry, told me it’s impossible to make money with on-premises software today.
Thus, increasingly more software products, including products used by NERC entities to operate and secure the Bulk Electric System (BES), are primarily available in the cloud, while the on premises versions are less functional and more expensive - if they are available at all.
Another change is that NERC entities are realizing that software delivered as SaaS is inherently more secure than software installed on premises. This is because newly discovered vulnerabilities are usually fixed by the SaaS provider as soon as a patch is available; the end user doesn’t have to do anything to fix the vulnerability, including applying a patch. In fact, the end user may never even know that the vulnerability existed[i].
A third change is that new cloud-based security monitoring services have appeared that can gather information on new threats in real time from sources all over the world. The service provider can use that information to protect their customers from new threats almost as soon as the threat is detected. Because of the EACMS problem, NERC entities with high or medium impact BES environments usually are unable to make use of these services. Moreover, when an on-premises security monitoring service moves to the cloud (which is also happening much more frequently), it usually becomes unusable by NERC entities, due to the same problem.
In other words, it is no exaggeration to say that the security of the North American power grid has been negatively impacted by the fact that the current CIP standards and definitions effectively prevent implementation or use of BES Cyber Systems (BCS), Electronic Access Control or Monitoring Systems (EACMS), and Physical Access Control Systems (PACS) in the cloud by NERC entities with high or medium impact BES environments. Should this situation not change, it is inevitable that this trend will continue and in fact accelerate.
What can we do to change this situation? As you may know, the NERC Risk Management for Third-Party Cloud Services Standards Drafting Team has been at work for close to 18 months. However, the team seems to have decided that to make it possible to implement BCS, EACMS and PACS in the cloud, they need to draft new versions of all the current CIP requirements. These new requirements will apply both to on-premises and cloud-based systems.
While I think the current CIP requirements, which of course only apply to on premises systems, need to be rewritten from the ground up, that alone will require a massive project of at least seven years (including getting a new Standards Authorization Request, since the current Standards Drafting Team’s SAR doesn’t mention anything about this goal). However, writing a set of CIP requirements that apply at the same time to on premises and cloud-based systems is very likely impossible, and in any case doesn’t address the real problems.[ii]
Fortunately, the problems of BCS, EACMS and PACS in the cloud can all be solved very simply if we keep in mind that nothing in the current CIP requirements prevents deployment or use of those systems in the cloud today. The three problems are all due to unintended effects of word choice by the Standards Drafting Team (SDT) that developed CIP version 5 (which is the foundation of today’s CIP standards). We only need to make a small number of corrections to words in a few requirements and definitions (along with a few new definitions). There’s no need to rewrite a single CIP requirement, let alone all of them.
I believe that making the following seven changes will achieve the above goal, although this obviously needs to be discussed by an appropriate group (either the current SDT or a new one, if the current SDT doesn’t want to take on this project). Moreover, I also believe these changes can be drafted, approved by NERC and FERC and implemented by October 1, 2028, the enforcement date of CIP-015 – although that assumes work starts on drafting these changes soon.
I’m proposing seven changes:
1. The term “System” needs to be defined to include both on-premises and cloud-based systems (it’s not defined now).[iii]
2. A new term “Cloud BES Cyber System” (or similar wording) needs to be defined (perhaps by basing this definition on the BES Cyber Asset definition, while replacing “Cyber Asset” with “System”).
3. A new requirement needs to be added to CIP-002, which requires the Responsible Entity to identify its Cloud BCS. Because there is no reason why a Cloud BCS needs to be classified as high, medium or low impact, the requirement does not have to reference the bright line criteria in Appendix 1 of CIP-002. Therefore, CIP-002 Requirements R1 and R2 and Appendix 1 should not need to change, at least not very much.
4. A new term, “Cloud Electronic Access Control or Monitoring System” (“Cloud EACMS”), needs to be defined. It will probably be based on the current EACMS definition, but with “Cyber Asset” replaced by “System”.
5. A new term, “Cloud Physical Access Control System” (“Cloud PACS”), needs to be defined. It will probably be based on the existing PACS definition, but with “Cyber Asset” replaced by “System”.
6. The existing definitions for BCS, EACMS and PACS need to be revised to make clear that they only apply to on-premises systems. This will allow almost all the existing CIP requirements to remain unchanged. Any requirements for Cloud BCS, Cloud EACMS, and/or Cloud PACS will be different from the on-premises requirements, since “cloud-only” risks are very different from risks to on-premises systems.
7. The three requirements that apply to BCSI (BES Cyber System Information) - CIP-004-7 R6, CIP-011-3 R1 and CIP-011-3 R2 - need to be slightly changed. These requirements need to be made applicable to BCS, Cloud BCS, EACMS, Cloud EACMS, PACS and Cloud PACS, rather than just BCS, EACMS and PACS, as they are today. I don’t believe that any more changes are required to these requirements.
These changes (perhaps with modifications) will address the first problem listed at the beginning of this post: the problem with CIP compliance if BES systems are deployed in the cloud today. However, the second problem remains: risks that only arise when BES systems are implemented in the cloud. Two examples of this are the risks posed by multi-tenancy in SaaS and widespread CSP outages. A large number of these risks have been identified already; more will be identified every year.
Risks like these are the most important to be addressed. However, as I stated in this post, trying to address them by developing mandatory NERC CIP requirements is probably impossible, but in any case will literally take decades. We need to figure out a practical way to address what I call cloud native risks. I think there needs to be a nonprofit organization (preferably an existing one. NERC would be a possibility, as long as this were kept separate from the standards area) that organizes representatives from the electric power industry and platform CSPs, SaaS providers, and cloud-based managed security service providers (collectively, the CSPs) to discuss these risks, identify mitigations for them, and develop voluntary guidelines for the CSPs to follow. If you would like to discuss this organization, please let me know.
If you would like to comment on what you have read here, I would love to hear from you. Please email me at [email protected] or comment on this blog’s Substack community chat.
Tom Alrich’s Blog, too is a reader-supported publication. You can view new posts for one month after they come out by becoming a free subscriber. You can also access my 1300 existing posts dating back to 2013, as well as support my work, by becoming a paid subscriber for $30 for one year.
[i] This isn’t necessarily a good thing, since if one SaaS provider discovers a vulnerability in their product and patches it without reporting it in a new CVE record, other software providers (both SaaS and on premises software) will never learn about the vulnerability, and the scanner vendors will never include it in their scans. There is now a proposal before the CVE Program to allow a new CVE record to make clear (in a machine-readable fashion) that the product listed is no longer affected by the vulnerability, although other products may be affected.
[ii] The reasoning behind both of these assertions can be found in this recent post, as well as a few more posts that I want to put up soon.
[iii] There may also need to be a NERC definition of “cloud”, unless there’s agreement in the NERC community that the term is well enough understood that no definition is needed.