Note from Tom:
I have moved to Substack as my primary blog platform. If you want to see all my new posts, as well as my 1200+ legacy posts dating from 2013, please support me by becoming a paid subscriber to my Substack blog. The cost is $30 a year. Thanks!
About 11 months ago, I wrote this post that asked the question how much time would elapse between the day when the Risk Management for Third-Party Cloud Services Standards Drafting Team (SDT) started drafting one or more new CIP standards designed to make medium and high impact BES assets (BCS, EACMS, PACS, etc.) deployable in the cloud, and the day when the new standards come into effect.[i]
To estimate how long that process would take, I drew from my experience observing (and participating in) meetings of the CSO706 SDT, which drafted CIP version 5 (as well as versions 2-4 before that). I estimated the whole process would take 6 ½ years, which was up almost two years from my estimate in January 2024.
I also pointed out there was a significant question whether, besides new or revised CIP standards, there will also need to be changes to the NERC Rules of Procedure. Since I wasn’t sure at the time that RoP changes would be needed, and because nobody I talked to (even people in the NERC ERO) knew for sure how such changes are made, I didn’t even try to estimate how much time they would require.
Today, I no longer have much doubt regarding this question; the answer is Yes, RoP changes will likely be needed. My guess is it will take two or three years of effort by somebody within NERC to accomplish the changes (which will probably need to be approved by a NERC ballot body and by FERC).
Since there’s no way today to know who at NERC would be in charge of this process or how long it will take, I’ll assume that RoP changes will require 1-2 years of the drafting team’s time and 1-2 years of somebody else’s (perhaps NERC Legal) time – in fact, my guess is there will need to be two or three changes, all quite substantial and requiring a lot of discussion with, and persuasion of, the NERC membership. Thus, I now think the SDT’s whole effort (i.e., the seven steps listed in the November post linked above) will require 7 ½ to 8 ½ years of the SDT’s time.
When I made the 6 ½-year estimate, I considered the Cloud CIP and the CIP v5 efforts to be similar, since they both required fundamental changes to all or most of the CIP standards; therefore, I based that estimate on the previous team’s experience. However, since I wrote that post, the SDT has started their standards development process (they started in January of this year). I’ve been able to attend some of the team’s meetings and talked with at least a few drafting team members outside of the meetings. Therefore, I can now create a more realistic estimate of the required time.
The team has yet to develop a detailed plan of what exactly they intend to work on (the revised SAR they submitted and got approved at the end of last year is quite good, but it lacks details on how they will go about developing the new and revised “Cloud CIP”[ii] standards). In fact, their short-term goal is to develop a white paper describing their current plans (at a high level, of course) and solicit questions and comments on it from the NERC community. This is a good idea, and I suggest that they not do any further standards drafting (other than what’s required to complete the paper) until after they’ve had the chance to receive and review all the comments.
However, I want to point out something I’ve only come to realize over the past couple of weeks: My 6 ½-year estimate of the job ahead of them is, if anything, too short. I say this because the SDT’s most important task is to identify and assess risks to the Bulk Electric System that will arise from allowing systems that operate the BES to be implemented in the cloud
In their SAR, the SDT provided a good list of cloud risks they want to consider, including:
1. Procurement / supply chain controls
2. Reliability / operational risk / resilience
3. Compliance / enforcement risk
4. Data sovereignty
5. Life cycle
6. Key management
7. Cloud ramping / Communications
8. Concentrated Span of control
9. Reliance on indirect services
10. Multi-tenancy
11. Regional considerations
12. Blackstart scenarios
I identified three more risks – all of which have already led to significant cloud breaches, like Capitol One – in this post. Most importantly, in this post I pointed to an extremely serious cloud risk, for which a security researcher recently demonstrated an exploit. If sufficiently generalized (which would be very difficult, although it isn’t impossible), this could lead to the ultimate nightmare of cloud Platform-as-a-Service (PaaS) providers: a situation in which many cloud tenants could in principle be exploited to attack many other tenants.
The SDT’s risks and mine sum to 16 – and I’m sure it won’t be hard to discover more as well. How long will it take to do the following for each risk?
A. Confirm the risk is a real one. Since the SDT discussed their list of 12 risks extensively and the four risks I identified are based on successful attacks like the Capital One breach, I am sure they will all be found to be real, as well as others.
B. Determine whether a new CIP requirement (or in some cases, more than one requirement) is needed to mitigate the risk;
C. Draft the requirement;
D. Draft any definitions that are required for the requirement to be clear (one big mistake made by the CIP v5 drafting team was leaving some important terms not defined, including “programmable”, used in the definition of Cyber Asset, as well as “routable” in the definition of External Routable Connectivity; the latter plays a huge role in determining the number of CIP requirements the entity must comply with).
E. Respond to multiple rounds of comments on the new requirement during the balloting periods. The comments are almost guaranteed to be voluminous, perhaps even more than the 2,000 pages that the CIP v5 SDT received during just one of the four balloting periods they went through. This is because the “Cloud CIP” standards will be some of the first mandatory cloud use standards ever developed, outside of the military. This drafting team is breaking a path for all industries, not just electric power;
F. Modify the draft requirement in response to comments received in each ballot cycle (the SDT in principle must reply to every comment, although they can group similar comments together. Also, many NERC entities simply defer to their trade association, e.g. “Utility XYZ supports comments submitted by EEI”, etc.).
Based on my observation of the CIP v5 drafting team (as well as other SDTs), I see no way that the above tasks won’t take at least three months per requirement, meaning that, during the entire drafting/approval/implementation process, the team will spend at least three months per requirement addressing the above tasks – and it could well be longer than that. As I’ve already stated, these will be the first non-military and non-intelligence cybersecurity requirements for users of cloud services (that I know of, anyway). There are bound to be long discussions and, in many cases, arguments at every step of the way about each requirement and how well it mitigates the threat it purports to mitigate.
The only step remaining is to multiply 16 requirements by 3 months per requirement. The result (for those who skipped third grade) is 48, which is exactly four years. That’s my bare minimum estimate of the time the SDT will need to spend on accomplishing the six tasks just listed, for each of the 16 cloud risks.
But that won’t be the end of the story. New cloud risks are being discovered all the time (sometimes the hard way due to a breach and sometimes the not-so-hard way by having a researcher – or the CSP itself – discover a vulnerability on their own). Whenever a serious new cloud risk is discovered (I’m sure this happens at least a few times a year), the only way for NERC to address it as part of the CIP standards today is to a) write a SAR to develop a new requirement to mitigate the risk, b) constitute a drafting team to draft the requirement, c) go through three or four ballots and comment periods before the requirement is approved (probably requiring at least six months), d) send it to FERC and wait at least 3-6 months for them to approve it, and finally e) make the requirement effective after a 3-6 month (at least) implementation period.
This whole process will take 2-3 years, although it can be accelerated if the risk is especially serious. Needless to say, it wouldn’t be realistic to do this, even if only one new cloud risk were discovered every year. What’s most likely to happen is that a lot of less-than-absolutely-critical risks will simply be ignored by the NERC CIP standards.
There is a precedent for this. For example, the first cases in which an infected laptop was attached temporarily to a network and ended up spreading a virus to the computers attached to the network were reported in the late 1990s. Yet, when did CIP-010 Requirement R4 (which required controls on Transient Cyber Assets) become effective? Early 2017.
Another example: Ransomware has caused huge damage and cost a lot of money for at least 12-15 years, yet when did the NERC CIP ransomware requirement come into effect?...Sorry, that’s a trick question. There’s no CIP anti-ransomware requirement, nor is there any requirement under development. In fact, there isn’t any talk about developing one (FERC conducted a conference on ransomware a few years ago, but never ordered NERC to develop a new requirement). Why isn’t there a requirement? That’s easy: Nobody in the NERC CIP community, or anyone at FERC, wants to go through the dreaded NERC standards development process. They would rather leave ransomware protection to the individual NERC entities to implement on their own (and I’m sure all NERC entities have extensive ransomware protection now. While at least a few NERC entities have experienced a ransomware attack on their IT network, none that I know of has experienced a ransomware attack on their OT environment.
Summarizing…
I think that drafting and gaining approval for CIP requirements that are based on the 16 cloud risks listed above will require, at a bare minimum, four years. Is this included in the 7 ½ to 8 ½ year estimate that includes Rules of Procedure changes? Almost certainly not, meaning my new estimate for the SDT to accomplish everything they need to do is 11 ½ to 12 ½ years. However, I must admit it could be longer than that, since it’s likely that addressing the cloud risks alone will take more than four years.
In addition, some permanent mechanism will need to be put in place to a) identify important cloud risks that appear in the future and b) incorporate them into the list of cloud risks that were identified by the SDT; that is, the list of cloud risks to be addressed needs to be a living document that’s constantly updated. Of course, there’s no way this mechanism could be created without first changing the Rules of Procedure (it is likely that other changes will be required as well).
The SDT wants to develop new CIP standards that will be based on the existing standards, but will include both cloud-based systems and on-premises systems and will be entirely objectives-based.[iii] I have long believed that the CIP standards need to be revised from the ground up and made entirely objectives-based (although the term I normally use is risk-based. The two terms mean effectively the same thing), so I’m not opposed to the SDT doing this. But this is a big task, and for the SDT to take on this task as well as address cloud risks would mean they will be at work for well over a decade.
Even more importantly, a new SAR will be needed for the project to develop the new “hybrid” CIP requirements applicable to both on-premises and cloud-based systems. This project will be very different from the project described in the SAR they’re working on now. That SAR focuses on the need to allow NERC entities with high and medium impact BES environments to use the cloud, but doesn’t require that whatever standards are developed be applicable to both on-premises and cloud-based systems; that will be a very hard trick to pull off, although it might not be impossible.
More importantly, while NERC entities didn’t provide a lot of input to the SDT when they were developing the revised SAR last year, it’s certain they will be clamoring for attention when they hear a new SAR is being developed for revising the existing standards; there will be plenty of comments in favor of these revisions, although there will be wide differences on what individual entities ask for. This is the main reason why the SDT shouldn’t embark on the course of action they’ve chosen, without first getting extensive public comments and drafting a new SAR. Hopefully, the SDT will figure out a way to keep the entire new project down to a manageable total of no more than say four years.
So, if this SDT changes course and develops modifications to the existing CIP standards but doesn’t add new standards to address cloud risks, how will cloud risks be handled by NERC? I will continue this discussion in another post soon, coming soon to a blog near you.
If you would like to comment on what you have read here, I would love to hear from you. Please email me at [email protected] or comment on this blog’s Substack community chat.
I’m now in the training business! See this post for more information.
[i] Note that BES Cyber System Information (BCSI) has been “legal” to store and use in the cloud since January 1, 2024, when the revised standards CIP-004-7 and CIP-011-3 came into effect. However, because there has been so little guidance on compliance with these two standards that BCSI is hardly being used in the cloud at all today - at least compared to how much it could be used.
[ii] Cloud CIP is my name for the team, not the SDT’s.
[iii] Since this SDT is quite concerned about the bad experience of the CIP Modifications SDT in 2018 (in fact, at least two members of the current SDT were members of that SDT in 2018), besides the new set of CIP standards, the old ones will still be available to NERC entities that don’t want to change. When I outlined my ideas for “Cloud CIP” in 2023, I included the same “two track” system. I think it’s essential, since the large IOUs that opposed the idea of having to revisit their entire CIP investment in 2018 are unlikely to be willing to do that now, either. I think the SDT wants to make the transition for those entities as smooth as possible, which is quite wise.