NERC CIP: Time to make your voice heard!

 

Since last summer, I have been participating in a small, informal discussion group called the Cloud Technical Advisory Group (CTAG). It is composed of NERC and regional staff members (including a few current or past CIP auditors), staff members from NERC entities, representatives of two major cloud service providers, a few consultants like me, and one longtime staff member (and NERC CIP expert) from a four-letter federal commission.

As its name implies, the group’s “charter” is to discuss the problems that are preventing NERC entities with high and/or medium impact BES Cyber Systems (BCS) from fully utilizing the cloud, and to do our best to move the ball forward to finally address these problems. One positive step was the approval in December by the NERC Standards Committee, of a Standards Authorization Request (SAR) intended to lead to a complete “normalization” of cloud use by entities subject to the NERC CIP Reliability Standards.

I’ll be honest: This issue has been around since the cloud first became important, but it is only within the last 3-4 years that it’s received wide attention in the NERC community. I believe this was probably because not being able to make full use of the cloud was initially seen as primarily a missed opportunity to save time and money: “Gee, wouldn’t it be great if we could move all these systems to the cloud and not have to install and maintain them ourselves?” However, it seems that both NERC ERO staff and NERC entities were reluctant even to think too hard about the big changes to the CIP standards, and perhaps the NERC Rules of Procedure, that might be required for this to happen.

One important example of this, which has been discussed for years, is the fact that NERC entities with high and medium impact BCS are not currently able to utilize cloud-based network security services - i.e., services that operate a big SOC that monitors the entity’s networks and internet activities. These services become more and more valuable as they grow their customer base over time, since they can “see” so much traffic worldwide that is not visible through individual networks; thus, they can identify new threats much more quickly.

However, a NERC entity with medium and high BCS can’t utilize these services now, because by doing so, the cloud-based server would then become an Electronic Access Control or Monitoring System (EACMS). This means – among other things – that the server would have to be enclosed in a Physical Security Perimeter (PSP) operated by the entity.

That would have huge consequences; for example, the cloud service provider (CSP) would probably have to install card readers at all entrances to any one of their data centers that held any part of a BCS owned by the entity. All employees would have to badge in and out to that card reader, and they would have to do the same to separate card readers for every other NERC entity with medium or high impact BCS housed, in whole or in part, at any of those data centers. You get the idea: this is impossible for any CSP (and I could go on and on about the other impossible things the CSP would have to do).

One longtime NERC CIP auditor, now retired, told me about six years ago that an entity with high impact BCS in his Region had started using the services of one of the original cloud-based security monitoring services (which is one of the leaders in that field today) to monitor its networks, including its Electronic Security Perimeters (ESPs).

The auditor had to tell the entity to rip out everything they had put in place to use that service and instead install EACMS to do network access monitoring locally, in a PSP the entity could control. He said it “broke his heart” to have to do that, since he knew the entity’s level of security would decline because of this – and they would have to spend a lot of time maintaining on-premises devices that wouldn’t be needed if they could use the monitoring service. Of course, to this day, the entity is still using the on-premises “solution”.

I must admit that six years ago, I wasn’t particularly bothered by what the auditor told me, since I knew the changes to the CIP standards that would be required to allow this entity (and all similar NERC entities) to fully use the cloud were simply out of the question. I blamed the entity for their problems, since they should have known better than even to try such an outrageous stunt.

However, in the last year or so, the discussion has changed. Now, it’s much less about missed opportunities to save time and money and more about actual damage being done both to operations and to securityof NERC entities with medium and high impact CIP environments. And now it isn’t just one or two entities that are complaining about this; more and more are complaining all the time.

But there’s an even more serious consequence of this problem, beyond diminished security. The big problem now is that NERC entities are hearing more and more from their software suppliers (including software for real-time operations in medium and high impact CIP environments) that they are moving to the cloud (i.e., becoming SaaS). The supplier might commit to continued support for their on-premises version for a few more years (and not always even that), but they usually make clear that their development dollars are going to the cloud. From now on, if the NERC entity wants to have all the new bells and whistles, they will have to use the cloud version.

When I joined the CTAG last summer, this problem was growing, with of course no end in sight. But even given that, there still wasn’t a sense that this was now not just a nice thing for the to-do list, but an urgent problem that needed to be solved soon. That is, there wasn’t until…last week’s meeting.

At that meeting, the NERC people were clearly very concerned. They said the complaints about the cloud issue are now pouring, not just trickling in. However, since the SAR was approved in December, that means the clock is now ticking for a solution to the cloud problems to be in place. Of course, BCSI in the cloud was always one part of that solution, and it became reality on January 1. Unfortunately, the changes required for BCS, EACMS, and PACS (Physical Access Control Systems) to be in the cloud require much more thoroughgoing changes to the CIP standards, and perhaps even to the NERC Rules of Procedure (which I don’t believe has been the case for any previous change to the CIP standards) than did the BCSI changes.

So, at last week’s CTAG meeting, we looked at the question of how much time would be required between today and when a full solution to the cloud problem would be drafted, approved by NERC and FERC, and ready to be implemented by NERC entities. Here is my timeline:

1. When the Standards Committee approved the SAR, they assigned it medium priority. They did that because there are over 20 other standards development projects (across all the NERC standards, not just the CIP standards) already in progress. Therefore, nothing at all will happen to the project before this July.
2. In July, there will likely be a call for drafting team members. However, it won’t be for the standards drafting team that will draft any new standards for the cloud, but for the team that will draft the final SAR that will guide the drafting team, and get it approved by NERC. Of course, when that team is constituted, they will hold meetings and submit draft SARs to votes by the NERC ballot body; there will likely be multiple ballots required, each with its own comment period, followed by comments by the drafting team, etc. I would be surprised if that could all be accomplished in six months, but let’s assume it will be.
3. Now, we’re at the beginning of 2025. At that time, NERC can solicit nominations for a Standards Drafting Team (SDT). When that team is constituted, they can start meeting to draft the new standard(s) for the cloud. Of course, they will then have to go through multiple ballots and comment periods before final NERC approval, followed by approval by FERC (which can take over a year by itself - and given the major changes that will be required for this project, that is likely to be the case).
4. How long will that process take? The CIP version 5 standards were a fundamental rewrite of all of CIP. The CSO 706 SDT had previously drafted (and passed) CIP versions 2, 3 and 4. They started work on CIP v5 in January 2011; FERC approved v5 in November 2013, close to three years later.
5. However, CIP v5 included the “bright line criteria” for identifying BCS. These criteria were originally developed for CIP version 4 during 2010 (v4 was approved by NERC and FERC but was never implemented. Long story). That effort took at least six months, so let’s say the whole drafting and approval process for CIP v5 took 3 ½ years.
6. Given that “Cloud CIP” will constitute an equally fundamental change in the CIP standards as CIP v5 did, it’s safe to say that 3 ½ years is a good estimate for the time the process will take, starting with seating of an SDT in early 2025. Thus, we can expect FERC approval of Cloud CIP by about July 2028.
7. After FERC approves the new standard(s), there will be an implementation period of probably one to two years. But, given that there are many NERC entities that want to be able to use the cloud as soon as possible, there will undoubtedly be some provision for them to start complying with the new standards earlier than that. So, let’s say that, six months after FERC approves Cloud CIP, NERC entities will be able to implement BCS and EACMS in the cloud. That means the beginning of 2029, i.e., five years from now.

However, there’s an elephant in this room. Since it is very possible that changes will need to be made to the NERC Rules of Procedure, we have to allow at least a couple of years for that (I have no idea what’s required for an RoP change, but I’m sure it requires NERC and FERC approval and perhaps balloting as well). It would be nice to think that the RoP changes could be drafted and approved in parallel with the new standards, but it’s hard to see how the RoP changes could even be drafted until the new standards are finalized.

So, if we get lucky and there are no major glitches along this path, you can expect to be “allowed” to deploy medium and high impact BCS, EACMS and PACS in the cloud by early 2031. Mark your calendar!

Perhaps you may think that eight years is a little long to have to wait for the cloud to be completely “legal” for NERC entities. I can assure you that the CTAG members thought the same thing last week (and my estimate at that meeting was five years, not eight). In fact, the leader of the group (a very respected CIP expert employed by one of the Regional Entities) said this was “completely unacceptable”. Then there was discussion of “In case of emergency, break glass” provisions that might be invoked to allow full cloud use – for those that want it – much sooner than eight years from now.

The meeting closed with a recommendation that NERC entities be urged to make known to their Regional Entities that this problem is no longer just a missed opportunity to save money, but a cause of significant harm, both to reliability and security. Moreover, this harm is rapidly growing.

How this will be accomplished needs to be decided. But there should be no doubt about the need to accomplish this in some way. It’s time to make your opinion known!

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at [email protected].

I lead the OWASP SBOM Forum. If you would like to join or contribute to our group, please go here, or email me with any questions.