Fri, Aug 14

I have just a few 😊 questions for the Cloud CIP Standards Drafting Team

Recently, I participated in an online meeting with many staff members from NERC entities with CIP compliance responsibilities. At the meeting, one of the members of the Project 2023-09 Risk Management for Third-Party Cloud Services Standards Drafting Team provided information on the recently posted CIP “100 series” standards, which address use of cloud-based systems subject to CIP compliance.

This post describes the admission made by the SDT member at that meeting: Despite the SDT’s repeated assurances that a NERC entity that doesn’t want to use cloud-based systems in its high or medium impact BES environment will not need to comply with the 100 series standards, they haven’t mentioned the fine print (perhaps because they didn’t realize it themselves until recently).

If you read that (and the only place I’ve seen the fine print is in my post just linked), you’ll learn that entities that want to use cloud-based software that meets the definition of EACMS (Electronic Access Control or Monitoring System) or PACS (Physical Access Control System) will have no choice other than to comply with the 100 series standards – which, if you take a look at the draft standards and definitions that were posted last month, won’t be a piece of cake, to say the least.

As background, there are three problems that constitute the “Cloud CIP” problem; they were laid out (not in exactly these words) in the original Standards Authorization Request (SAR) that led to the SDT being constituted in 2024. These are:

1.      NERC entities with a high or medium impact BES environment can’t utilize BES Cyber Systems (BCS) that are installed in the cloud while maintaining compliance with all (or even most) NERC CIP requirements.

2.      NERC entities with a high or medium impact BES environment can’t utilize EACMS that are installed or made available in the cloud while maintaining compliance with all (or even most) NERC CIP requirements.

3.      NERC entities with a high or medium impact BES environment can’t utilize PACS that are installed or made available in the cloud while maintaining compliance with all (or even most) NERC CIP requirements.

The original SAR made it clear that by far the two most important of these problems are the second and third ones; the SAR requested – nay, begged – that the SDT focus on those two problems first. Unfortunately, that didn’t happen. Thus, a NEC entity that wants to utilize a cloud-based SIEM or MFA service, or PACS service, to protect their on premises medium or high impact systems will be exactly where they are today: SOL (which doesn’t stand for “system operating limit”, BTW).

During the Q&A at the end of the meeting, a person who introduced himself as new to the NERC CIP world asked a simple question (which I am paraphrasing, since I’ve forgotten the original wording): “It seems like you’re not sure about a lot of this.” Unfortunately, this person hit the nail on the head. This was confirmed in another meeting I attended this week, in which a drafting team member was discussing some of the fundamental questions they’re running into now, as they are starting to draft more of the 100 series standards, beyond the five they’ve drafted so far. Why didn’t they nail these issues down two years ago when they started meeting, or at least when they decided to rewrite the existing CIP standards – which activity was never mentioned in either of their SARs?

Comments on the initial posting are due next week. The comment form asks some very specific questions about the standards, most of which require a yes or no answer (with the option of adding freeform comments at the end). These questions all assume the framework of what the SDT is proposing is basically sound and it just needs some fine tuning. That’s far from being the truth. Here are some questions I intend to ask the SDT in my comments:

1.      After starting work in the summer of 2024, why did it take you until only two or three months ago to finalize definitions for fundamental terms like “BES Cyber System or Service”? That should have been the first thing you did. How can you even discuss requirements when the terms addressed in those requirements haven’t been at least preliminarily decided on – as evidenced by the fact that the white paper you published last December doesn’t have any definition at all? The team that drafted CIP version 5 starting in 2011 (CIP v5 was the only complete rewrite of the CIP standards before now, but the 100 series is far more ambitious than v5 was) had already defined the fundamental term in v5 – BES Cyber System – in a “concept paper” in 2009.

2.      You have – ahem! – pushed the boundaries of CIP (or even NERC) compliance with concepts like System Security Plan and the idea that a NERC entity can choose, for each on-premises system that meets the BCS definition, whether to have it comply with the existing CIP standards or the 100 series. Have you ever formally run these ideas by the auditors, as well as the NERC lawyers, to determine whether changes to the Rules of Procedure will be needed to make them feasible? After all, the CIP v5 SDT spent at least a full day with NERC auditors within about 2-3 months of commencing work in January 2011. As you’ll see below, even that didn’t protect them against later pushback from FERC.

3.      I know the answer to the above question: No, we haven’t formally run our ideas by the auditors (despite promising me to do that at least a few times). Then, why did you ask NERC entities to comment on five draft standards and a number of draft definitions as they’re doing now, when you weren’t even sure they will pass auditor muster? Surely you know, since you collectively have decades of SDT experience, that the draft standards will never even make it to the first ballot if the auditors haven’t signed off on them. You are probably literally wasting NERC entities’ time, since once you’re revised the standards to address the concerns the auditors raise, you will have to re-post them for comment. The last thing you should want to do is submit standards for balloting when there are still problems that can be fixed, since it’s inevitable that the Ballot Body (i.e., the NERC entities who want to vote on this) will find all sorts of problems in what you submit, even if the auditors, lawyers, Alexander Hamilton and Thomas Jefferson have all signed off on them beforehand.

4.      Perhaps you’ve heard that the CIP v5 SDT included, in all its drafts of the CIP v5 standards, the words “identify, assess, and correct” in just about every requirement. These were there to address what was considered the number one problem with CIP versions 1-3 (v4 was approved but never implemented): “zero-tolerance” auditors who insisted that even small deviations from a requirement were violations. The idea of IAC was that an entity wouldn’t be assessed only on whether they had complied with the strict wording of the requirement; instead, they would need to a) identify any potential violations, b) assess what problem caused the PV, and c) correct each problem found. If they did all that, there was no harm, no foul. I and lots of others thought this was a great idea, but FERC didn’t. When they approved v5 in 2013, they ordered that wording be taken out. They did that on the grounds that “No standard can dictate how it will be audited.”

5.      I think what you’re advocating now, especially the System Security Plan, is quite similar to “identify, assess and correct”. While you certainly have no channel by which you can get feedback on this issue – or anything else, other than perhaps the time of day – from the FERC Commissioners, this is certainly something you need to discuss with the NERC lawyers. The last thing you want to happen is to have FERC, 2-4 years from now, completely remand the 100 series (since they never ordered it in the first place, unlike almost every other change in the NERC CIP standards) due to this problem. This will mean you will have completely wasted 4-5 years of your time, as well as the industry’s time. Won’t that be a bummer?

6.      Did you ever figure out what the fundamental “cloud CIP problem” is – specifically, the issue at the root of the three problems I listed earlier? Why do you think that the wording of the current CIP requirements is at fault, since none of them even mentions the cloud, let alone forbids its use? The problem is with the definitions of the terms BCS, EACMS and PACS, since they don’t take into account any system based in the cloud. These three definitions can easily be changed in a day (even allowing for a long lunch break). In this post last December, I outlined eight simple changes (almost all to definitions, plus one new definition: “system”). All eight of the changes could easily be drafted within 1-2 weeks, not two years and counting. They would most likely sail through approval, since NERC entities wouldn’t have to change any CIP practices or documentation. If you started to draft those changes today, they could probably be in effect early next year.[i]

7.      After having needlessly decided to develop new versions of all the existing CIP standards, you compounded that error by deciding that the new versions you were developing need to apply both to on premises and cloud-based systems; moreover, they need to replace the current CIP standards in the future. Who asked you to rewrite the CIP standards? I and others have been advocating that since before the CIP version 5 standards (the basis for today’s standards) were implemented in 2016. We did this because we could see that cybersecurity is fundamentally risk management, so the standards need to be made risk-based (as the SDT is saying now).

8.      However, the reason I haven’t been pushing hard to make all of CIP risk-based is that the current NERC enforcement regime, which is encoded in the Rules of Procedure but probably other documents as well, does not handle risk well at all. For example, the new vulnerability management requirement in the 100 series, CIP-105 Requirement R6 Part 6.3, mandates “A plan to mitigate prioritized cyber security vulnerabilities.” What if an entity’s plan reads, “We recite a certain mantra as a group every morning; we find this protects us against attackers exploiting cyber vulnerabilities. Since no system in our ESP has ever been hacked, we think this is a very effective mitigation”? Of course, this is a ridiculous assertion, but what can the auditor point to in the requirement that will allow him or her to prove the entity has violated it?

9.      Conversely, what if an entity follows a reasonable patching program, but the auditor insists that the only acceptable mitigation plan is one that strictly implements zero trust throughout the entity’s OT environment, even for ten-year-old systems that have no ability to implement zero trust? How is the entity going to counter this, especially since the SDT decided that they don’t have time to develop implementation guidance – and since no other documents will provide guidance that must be considered in any audit dispute? As was often the case when CIP version 5 was being implemented (and will almost certainly be much more the case after the 100 series is implemented, if what we’ve seen so far isn’t changed), there will probably be lots of heated compliance discussions that will never be resolved, except through a fragile truce (i.e., a cease-fire).

10.   Why are so many 100-series requirements written to apply to cloud-based systems (most of which will be under the complete control of the cloud service provider) as well as on-premises systems, even though the CSPs have made it clear from the beginning that they will never provide any compliance evidence other than audit reports - which they’ll give to any customer that asks for them? And even though they’ve also made it clear that they won’t negotiate contract terms with any customer not named Uncle Sam (or maybe Aunt Samantha)?

11.   Continuing this thought, since the only possible evidence for most 100 series requirements will be audit reports, why not just require the entity to look at those? If they were about to sign a contract with Clem’s Cloud Services and Screen Door Repair, which probably isn’t FedRAMP authorized or SOC 2 Type 2 certified, this step will hopefully dissuade them from doing that. But if the entity long ago signed a contract with one of the Big Two CSPs (and my guess is 80-90% of NERC entities did this long ago for systems on the IT side of the house), reviewing the latest audit reports will just confirm to them that there’s no reason even to question the original decision. This won’t be compliance, it will be compliance theater. There will almost never be any doubt that the entity won’t be found in violation of any requirement. It’s like the Lake Woebegone effect: All the children are above average.

12.   Drafting team, I’ve heard you’re now starting to rewrite the BCSI (BES Cyber System Information) requirements: CIP-004-7 R6 and CIP-011-3 R1 and R2. Please stop. A drafting team started working on those requirements in 2019, to achieve their objective of making use or storage of BCSI in the cloud “legal” under CIP. Their changes were approved by FERC in 2022 and implemented on January 1, 2024. Those changes have been almost completely ignored since then, mostly because neither NERC nor any of the Regional Entities has taken it upon themselves to explain them to the CIP community. Fortunately, that SDT left a brief but workable guidance in their Technical Rationale, included in their filing to FERC in 2021 (see page 11, the second full paragraph). I hate to see you throw that away, when what’s needed is just real guidance from NERC.

Finally, here is the entire “Deliverables” section of the revised SAR that you drafted in 2024, which was approved by the NERC Standards Committee at the end of that year. This is, of course, the SAR you’re supposed to be following:

The following describes the proposed deliverables for this project:

1) The DT will strive to minimize impacts to existing requirements for on-premises systems and assets under the existing CIP-002 through CIP-015 suite of standards.

2) The Drafting Team will consider risks related to cloud services for CIP applicable systems, including but not limited to:

·        Procurement / supply chain controls

·        Reliability / operational risk / resilience

·        Compliance / enforcement risk

·        Data sovereignty

·        Life cycle

·        Key management

·        Cloud ramping / communications  

·        Concentrated Span of control

·        Reliance on indirect services

·        Multi-tenancy

·        Regional considerations

·        Blackstart scenarios

Note: I numbered the two deliverables.

Has the drafting team delivered the first deliverable? Only if you ignore the “fine print” that I mentioned at the beginning of this post. That is, they didn’t change any current CIP requirement except CIP-002; on the other hand, they also didn’t take the simple steps needed to allow current on-premises-only users to utilize the cloud without having to make the big leap to the 100 series (which will of course apply to their on-premises systems if they’re protected by a cloud based EACMS or PACS). My guess is most entities with only on-premises systems, who would otherwise utilize a cloud-based EACMS (e.g. SIEM or MFA system) or PACS will simply not do that, rather than make the huge investment of time required to understand and document compliance under the 100 series standards, for all of their on-premises systems.

And how about the second deliverable? Those 12 items are almost all purely “cloud risks” – i.e., risks that arise when BES systems are deployed freely in the cloud. I was expecting risks like these would be the focus of the SDT’s work, but I was very disappointed when I realized that the SDT wasn’t going to address them, perhaps for years. In fact, when they posted the initial standards and definitions for comment, they also officially stopped consideration of CIP-116, the standard that was going to address cloud risks. It’s not clear when or if they will resume consideration of that standard, but it will very likely not be in the first version of the 100 series, absent some drastic change.

Instead, as you can see, the SDT is focusing entirely on rewriting the existing CIP standards, which don’t address any purely cloud risks. Therefore, to facilitate use of the cloud by NERC entities, the SDT is no longer considering the risks that are most important. Instead, they are spending their time developing requirements that are already addressed by the CSP’s FedRAMP authorization and ISO 27001/SOC 2 Type 2 certification.[ii] This isn’t compliance; it’s compliance theater.

There’s one more problem that I just realized when I read the SDT’s comment form. Since this post is already very long, I’ll save that for a new post, hopefully tomorrow. I’ll also include my ideas for what the SDT should do, not that I have any great illusions that they’ll do it.

Tom Alrich’s Blog, too is a reader-supported publication. You can view new posts for three months after they come out by becoming a free subscriber. You can also access all of my 1300 existing posts dating back to 2013, as well as support my work, by becoming a paid subscriber for $30 for one year (and if you feel so inclined, you can become a founding subscriber for $100). Whether free or paid, please subscribe. 

If you would like to comment on what you have read here, I would love to hear from you. Please comment in my chat or email me at [email protected].


[i] Also, implementation of what I’m proposing will come into effect at least two years before what the SDT is proposing, even if the implementation period is the same. This is because the SDT’s complicated compliance model requires a change in CIP-002. There are already two new approved versions of CIP-002, versions 7 and 8, scheduled to take effect on July 1, 2028. Since nobody wants three new versions to take effect the same day, this means the earliest that the 100 series standards can take effect is January 1, 2029. But even that isn’t likely, since it will require a NERC entity to make two major revisions to its CIP-002 compliance program in one year. To say the least, that’s not going to be a popular suggestion. Meanwhile, the eight simple definitional changes I’m proposing don’t require any change to CIP-002.

[ii] This is unfortunately like the old joke regarding a man who comes home at night and sees his neighbor on his hand and knees, looking for something under the streetlight. He goes over and asks what he’s looking for. The neighbor answers, “My house keys”. The man asks where the neighbor last saw his keys; the neighbor gestures toward the dark lawn. The man asks why he’s looking under the streetlamp if the keys are probably on the lawn. The neighbor explains, “Because the light’s better here.”

1