The great physicist Wolfgang Pauli was often approached by someone who thought they’d made a great discovery that would upend physics – like he did many times – and wanted him to agree with them. Sometimes, he would patiently prove to the person (often a graduate student) that they had missed a crucial step in their math. But other times, the person (an armchair physicist) would simply be throwing words around without having put in the years of study that would help them understand what those words meant. All Pauli could say to them is that they were “not even wrong”. There was no way their statement could be proven either right or wrong, so it wasn’t worth considering any more.
In fact, a scientific finding needs to meet two criteria: It can’t clearly violate a scientific principle like the conservation of energy, but it also needs to be “falsifiable”. That means there needs to be some way by which the finding can be proven wrong. For example, if I state that the moon is made of green cheese, I can be proven wrong in many ways. But if I state that people in North Carolina are friendlier than people in Wyoming, how could I ever be proven right or wrong? It doesn’t matter how many surveys you take or how many times you visit those states; you can never prove my statement is either right or wrong. It’s a waste of time even to try to verify that statement.
Which brings me to the draft NERC CIP 100 series standards for cloud use. Four or five of these standards (there will ultimately be about 12 or 13, and the Standards Drafting Team (SDT) is just starting to work on the others) were released in July, along with some definitions and some other documents. I’ve already pointed out several serious problems with what’s been released (e.g., here and here), but I now realize that one of the biggest problems – found throughout the standards and definitions that have been released so far – is that there’s usually no way that a NERC entity could be found non-compliant with a requirement. Here are some examples:
1. In this post, I pointed out that the fundamental term in the 100 series – the term “BES Cyber System or Service” (BCSS) – depends on two undefined terms: “system” and “service”. The first sentence of CIP-102 Requirement R1 requires the NERC entity to identify “services and systems, that affect or support the reliability of the BES...” Suppose an entity decides they don’t have any BCSS because they don’t have any services or systems that meet that criterion, yet their auditor doesn’t agree with them; he or she believes the entity has at least one service that meets the criterion.
Since there’s no definition for service or system, how could the auditor possibly prove their point? They will just have to state there’s no finding with regard to this requirement (by the way, there are many other undefined terms in the 100 series; this will lead to a lot of problems. This happened the last time the CIP standards were completely rewritten as CIP version 5. But in that case, there were only a few important undefined terms. I’ve just identified two of them and I suspect there are a lot more).
2. One of the most common causes of findings in the current standards (at least, it was when those standards were first enforced as CIP version 5 in 2016) is that a NERC entity didn’t include all its BES Cyber Systems within its Electronic Security Perimeter (ESP). This is required by CIP-005 Requirement R1 Part 1.1, “All applicable Cyber Assets connected to a network via a routable protocol shall reside within a defined ESP.” Any violation of R1.1 was (and is) easy to prove, since it can easily be verified using network diagramming software. Conversely, if an auditor thinks an entity has violated R1.1 and the entity disagrees, it won’t take a long discussion for them to figure out which one of them is right.
However, in the 100 series, the term ESP has been replaced with “Electronic Security Zone” (ESZ). Its definition is “A defined logical trust boundary that groups applicable systems based on common security requirements, impact levels, or operational security objectives and enforces controls for access, communication, and monitoring. The boundary of a Cyber Security Zone is established through logical security controls, including security policies, identity and access management controls, authenticated communication pathways, and monitoring capabilities, and is independent of physical location or underlying network topology.”
I think you’ll agree that this definition allows an ESZ to be defined in many ways. If the auditor and the entity disagree about whether a particular system is included in an ESZ, the auditor will need to show that, no matter how the entity defines this ESZ, that system is still outside of it. I can see that easily turning into a week-long exercise. This means that in practice NERC entities will seldom if ever be found non-compliant with CIP-105 Requirement R1.1.
3. My third example is really many examples, since it is likely to describe many of the requirements in the 100 series, including the standards that haven’t been drafted yet. It stems from the problem that, more than any other, led to the 100 series being drafted: PACS and SIEM tools that used to be available entirely on premises are increasingly moving into the cloud. This is due to the huge cost decreases, as well as increases in flexibility and reach of services, that are available in the cloud. Even if the vendor continues to offer an on-premises version, it will often be more expensive and not updated as regularly, if at all.
Yet, if a NERC entity with a high or medium impact BES environment wants to continue using one of these services after it has moved to the cloud, they will face the problem that, since the cloud based system meets the PACS or EACMS definition, all of the CIP requirements that apply to a PACS or EACMS will still apply to that system. This means the vendor will need to provide evidence that they complied with each part of requirements like CIP-010 R1 Configuration Management, even though that means providing evidence about individual physical and virtual devices in cloud data centers – something that no CSP can do.
Since this problem just has to do with definitions, it would probably be well on the way to being solved by now if the SDT had been willing to even consider making the eight small changes (almost all to existing definitions) that I suggested last year. Instead, they have rewritten four or five of the existing CIP requirements to remove any need for evidence about individual devices. They intend to draft and submit the other 100 series standards, as well as get feedback – and make changes that will inevitably be required - from NERC entities, NERC auditors and NERC lawyers by the end of the year, so that all the standards will be ready to start the likely one year process of balloting and re-balloting them in January (while this is less unrealistic than their deadline up until a month ago, late September, it’s still quite unlikely they will make it. I assume they’re doing this to show they have a sense of humor).
Here’s the problem: Even when all of the existing CIP standards are rewritten to apply to both the cloud and on premises systems, the cloud service providers – which I define as platform CSPs and SaaS providers - still won’t provide evidence of compliance with any of the CIP requirements. This is because their whole business model is based on not doing special favors for individual customers, unless they happen to be the federal government, the military, the intelligence agencies or the major AI companies.
For example, CIP-105 Requirement R4 Part 4.2 reads, “Default accounts are identified and the associated credentials are changed, disabled, or protected from unauthorized use.” One of the Measures shown for this requirement part is, “Listing of accounts by account types showing the enabled default or generic account types in use.” Do you think that AWS or Azure is going to provide this level of evidence for you? Not to keep you in suspense, the answer is No. I’ve verified this with knowledgeable people at both organizations. The best they will do, in response to specific security requests, is give you the link to their latest ISO 27001 or SOC 2 Type 2 audit report and tell you the evidence that they properly address risks from default or generic account types is in there – which is almost certainly true for all the requirements in the 100 series.
If CIP-105 is approved and implemented and you face an audit, what are you going to do when you’re asked to provide evidence for compliance with CIP-105 R4.2? You’ll tell the auditors that the CSP wouldn’t provide the required evidence, but you will also show them the section of the CSP’s SOC 2 Type 2 audit report that shows they have the proper controls in place. The auditor will almost certainly agree that this constitutes evidence of compliance (since they have heard the same story from every NERC entity they’ve audited on the 100 series), even though the wording of the CIP requirement will never exactly match the “equivalent” control in the audit report.
The above scenario will be repeated for most of the requirements and requirement parts in the 100 series. In fact, unless you have intentionally violated one of the requirements, it’s quite hard to think of how you wouldn’t receive a perfect audit score for every audit.
Of course, it’s nice that you won’t be failing audits and you will be home in time for dinner most weeknights, but what is all this accomplishing? If all the evidence is in the CSP’s audit report and if no NERC entity is likely to use any CSP other than the Big Two when BES systems are involved, why even bother to audit the 100 series? Even better, given that the only risks addressed by the 100 series are those that are also addressed by the existing standards – but in a device-oriented, sometimes prescriptive manner – why even go through the process of approving and implementing the 100 series?
That process will almost certainly take 3-4 years starting today. Even worse, at the end of that process, the real risks from the cloud (such as the 12 or so cloud risks that were listed in the SDT’s revised SAR) will remain unaddressed (the SDT worked a little on a standard to address cloud risks, which was I believe called CIP-116, but decided a couple months ago to put it aside because they didn’t have time to get it done. It’s all a matter of priorities…).
Folks, this isn’t compliance, it’s compliance theater. However, the difference between this and legitimate theater is that the price of admission – meaning the huge effort that will be required to understand and document compliance with the 100 series standards, even though the evidence requirements will be minimal – will be much higher. But the worst part is that the actors – the NERC entities and the NERC enforcement staff – won’t have the satisfaction that what they’re doing is meaningful and important, because it won’t be. And if you think job satisfaction isn’t important both to you and the people you work with, think again.
Tom Alrich’s Blog, too is a reader-supported publication. You can view new posts for three months after they come out by becoming a free subscriber. You can also access all of my 1300 existing posts dating back to 2013, as well as support my work, by becoming a paid subscriber for $30 for one year (and if you feel so inclined, you can become a founding subscriber for $100). Whether free or paid, please subscribe.
If you would like to comment on what you have read here, I would love to hear from you. Please comment in my chat or email me at [email protected].