Tue, Nov 10

The deepening morass

Last Friday, I presented my concerns about where the current “Cloud CIP” standards drafting process is leading to the ERCOT CIP Working Group (CIPWG); we had a good discussion afterwards. Since then, I’ve revised the slides, partly based on that discussion but also to incorporate things I would have said on Friday had there been time. Here is the text from those slides:

1. Why I’m so concerned

◦        This presentation is based mostly on the slides I presented at the Nov. 6 ERCOT CIPWG meeting, although I’ve expanded it based on the Q&A after my presentation and subsequent developments. It’s also based on this post.

◦        I’m quite concerned about the NERC 2023-09 “Cloud CIP” drafting team. I know many of the team members and like them all. But that being said…I think they’re heading for a train wreck, which will have wider implications for NERC and CIP.

◦        The team has been meeting since the summer of 2024 and hasn’t yet produced a final first draft of any of the 12 or so new standards (plus 8 or 9 new definitions) that they’re planning to put in front of the NERC Ballot Body in January. Of course, this makes it impossible to say anything certain about what NERC entities will have to vote on, but it also means that considerations like auditability, quality and safety appear to have been placed second to speed. Of course, this isn’t a good situation.

2. The “100 series”

◦        The proposed new standards are  numbered CIP-102 through CIP-115. This is because they’re supposed to be “supersets” of CIP-002 through CIP-015, intended to apply both to on premises and cloud-based systems.

◦        However, it’s hard, and perhaps impossible, to write standards that apply simultaneously to on premises and cloud-based systems, since the former are physical devices, while the latter can’t be mapped to devices at all. That’s probably why in 2 ½ years the SDT has drafted no new standards or definitions.

◦        This year, NERC approved a set of significant improvements to the standards development procedures: the MSPP. FERC very much supports the MSPP and has ordered NERC to start implementing it in January.

◦        I agree that the improvements address some of the most serious problems in the standards drafting program. In fact, it would be much better if Project 2023-09 switched over to the MSPP, but that’s not happening. Instead, at NERC’s urging, the team is pulling out all the stops to finish drafting the 100 series before the end of this year, specifically to avoid having to work under the MSPP. Go figure.

3. Cutting corners (or more)

◦        Due to the holidays, there are no more than 6 working weeks left in 2026. NERC is pushing the team to cut corners in any way possible.

◦        One corner they’re cutting is the SDT is not going to put the standards up for NERC entity comment before they’re finalized and submitted for the first ballot in early 2027. This is unacceptable. NERC entities need to be allowed to comment on the standards they’ll have to comply with before they’re finalized. They haven’t yet had this opportunity, and they probably won’t have it. ote: The 5 draft standards posted in July were billed as only suggestions of what might finally be posted. The fact that only 5 standards were posted meant there was no way to understand how important concepts like System Security Plan, and being able to choose which standards to comply with, will work in practice

◦        While this might not be against the NERC Rules of Procedure, NERC is supposed to be a completely open and transparent organization, not one that doesn’t allow the membership to look at new standards until it’s too late to make any substantial changes. It never even occurred to me that a situation like this could occur, but here we are.

4. Cutting corners (2)

◦        A second cut corner is the SDT won’t produce Implementation Guidance documents for the 100 series standards. This is almost guaranteed to cause problems once the standards are implemented, since if a NERC entity and an auditor have differing interpretations of a requirement, this is the only document that can “break the tie”.  Given the large number of problems like undefined terms and vague wording that are sure to be in the draft standards, this is a serious problem.

◦        A third cut corner is that the SDT has never met with CIP auditors to determine whether the proposed standards are completely auditable; there are serious questions including auditability and evidence required from cloud service providers. These can’t be resolved in the balloting process.

5. Will balloting fix these problems?

◦        SDT members have repeatedly pointed out that NERC entities will be able to read and comment on the 100 series standards when they’re posted for the first ballot in January. However, at that point there will be no way to address fundamental problems – of which I believe there are at least 4 or 5 – that require a “back to basics” look at the structure of the 100 series, without ending the balloting and sending the SDT back to Square One. That will be very messy.

◦        One of the biggest problems is: What is the role of the “System Security Plan”? This plan is supposed to include many of the requirements in the 100 series, but the plan will itself be a requirement. If an entity violates one of the other requirements, does that mean they’ll also violate the SSP requirement? That sounds like double jeopardy. But if not, will it be possible to violate for example a few other requirements, and still not violate the SSP requirement? If so, where’s the floor for an SSP violation? 3 violated requirements? 5? 10?

6. Here’s what to expect

◦        This summer, the SDT posted this trial definition for “Electronic Security Zone”. This concept will replace ESP, for both on premises and cloud-based systems. This will most likely change when the draft standards are finalized, but NERC entities probably won’t see those changes until just before they vote. For lack of better information, NERC entities need to assume that the definitions they will be asked to approve will read like this one.

◦        As you can see, a lot of options have been crammed into this definition, making it hard to understand and audit. Plus, I’ve highlighted phrases that need to be defined or explained in Implementation Guidance, although that probably won’t be available.

“A defined logical trust boundary that groups applicable systems based on common security requirements, impact levels, or operational security objectives and enforces controls for access, communication, and monitoring. The boundary of a Cyber Security Zone is established through logical security controls, including security policies, identity and access management controls, authenticated communication pathways, and monitoring capabilities, and is independent of physical location or underlying network topology.”

7. Using SIEM in the cloud

◦        By far the most important driver for making the cloud fully “legal” for many entities is the fact that many on premises SIEM systems either have moved or will move to the cloud. In some cases, the system will only be available in the cloud, while in others it will be available on-premises as well – but often at a higher cost.  reduced functionality (for example, most upgrades only being available for the cloud version), or both.

◦        You may have heard that you won’t have to move to the 100 series if you don’t want to. That’s true, but according to one leader of the SDT, if you don’t move to the 100 series for both on premises and cloud-based systems, you probably won’t be able to utilize any CIP systems based in the cloud, including SIEM.

◦        Thus, even if you just want to use SIEM in the cloud, you may still have to abandon your current CIP program and move entirely to the 100 series, both on prem and in the cloud.

8. There’s a much simpler solution…

I believe the SDT could fix the SIEM problem with three simple changes to definitions:

  1. Modify the current EACMS definition so it only applies to on premises systems,

  2. Develop a new definition for “Electronic Access Control System” (EACS)

  3. Prohibit use of EACS in the cloud.

While this might not be the exact solution, there is definitely a simple solution that will take weeks to develop, not years. Since I now think that BCS and PACS in the cloud are too risky (due to AI bot swarms), fixing SIEM is as far as we should go in the cloud now.

9. The worst problem with the 100 series

◦        The worst problem with the 100 series is that the SDT decided this year not to address the list of 12 “cloud services” risks listed on pages 3 and 4 of their SAR, including problems like multi-tenancy of SaaS databases, key management, blackstart, and data sovereignty.

◦        Instead, they’re addressing non-risks like whether the CSP conducts PRAs for their employees and whether they patch their systems regularly. Is it likely any major CSP doesn’t perform these basic tasks? Why do we need requirements?

◦        Today, I believe bot swarms pose the biggest cyber risk to the power grid. This needs to be added to the list. There are probably others as well.

◦        While it’s likely that at least some of these risks can’t be addressed through the standards development process, it’s important that a drafting team consider them all and determine how each one should be handled. Since this SDT doesn’t want to address them, a new SDT should be constituted for that purpose.

10. What should this SDT do?

The SDT should do four things before the 100 series standards are sent to NERC for quality review (prior to posting for ballot):

  1. Post the standards and definitions for a 3–4-week comment period for all NERC entities; make appropriate changes based on their suggestions.

  2. Develop an Implementation Guidance document for each standard.

  3. Meet with NERC Regional auditors and ask them to give their opinions on whether the individual requirements are auditable, then make changes based on their suggestions.

  4. Get a NERC opinion on whether any Rules of Procedure might be violated.

Since these are basic items that most entities subject to CIP compliance are likely to require and that other CIP drafting teams have provided, it isn’t likely the standards will ever be approved by the NERC Ballot Body if these are absent.

11. A new SAR/SIR

Because the SDT has put aside consideration of cloud services risks, and because two other urgent cloud-related CIP needs have arisen, I believe a new SAR is needed. It should include:

◦        Developing a streamlined solution to the problem of “SIEM in the cloud” described in slide 9. This will most likely require a new CIP standard.

◦        Developing a solution to the problem of renewables Control Centers implemented in the cloud, described in this blog post. This most likely requires a new CIP standard, and modifications to some existing requirements.

◦        Consideration of the cloud services risks described in slide 10 (as well as other cloud services risks that might be identified). Deciding which of these can be addressed through new CIP requirements. Perhaps drafting new requirements.

◦        The analog of the SAR concept in the MSPP changes coming into effect in 2027 is Standard Initiation Request (SIR). Because the MSPP will be a big improvement,  it’s probably appropriate to develop a SIR, instead of a SAR.

12. A snag in implementing the 100 series

◦        The SDT has discussed in their meetings, but I don’t believe has pointed out publicly, that they have a big scheduling constraint.

◦        This is because they not only feel the need to create a whole new set of standards, but they also believe they must change the existing CIP-002 (along with creating CIP-102). This will be version CIP-002-9.

◦        I doubt the 100 series will ever be approved by the NERC Ballot Body. But if the standards are approved by NERC and then by FERC, the new CIP-002-9 will have to wait behind CIP-002-7 and CIP-002-8, which will both come effect on July 1, 2028. This means the 100 series can’t come into effect until 2029 at the earliest. There’s probably no way to get around this problem, short of making fundamental changes to the 100 series standards so that the CIP-002 change isn’t needed.

Tom Alrich’s Blog, too is a reader-supported publication. You can view new posts for three months after they come out by becoming a free subscriber. You can also access all of my 1300 existing posts dating back to 2013, as well as support my work, by becoming a paid subscriber for $30 for one year (and if you feel so inclined, you can become a founding subscriber for $100). Whether free or paid, please subscribe. Of course, if you can’t afford the $30, please email me and I’ll give you a complimentary one-year full subscription (this offer doesn’t apply to bots). 

If you would like to comment on what you have read here, I would love to hear from you. Please comment in my chat or email me at [email protected].

 

1