EnergyIoT Article 3 – IoT Architecture Big Picture
By Stuart McCafferty, Contributions from Eamonn McCormick and David Forfia
Disclaimer: The viewpoints in this article and others in the series are the personal views of the authors and in no way are meant to imply or represent those of the companies they work for.
This is the third in a series of articles introducing an EnergyIoT Conceptual Model that leverages modern architectural elements, such as virtualization, containerization, orchestration, rich semantic information models, message buses, DevOps microservices, and the cloud. This architecture:
- Is elastic and scalable
- Is service-oriented
- Is event-driven
- Enables edge computing as well as more traditional centralized computing
- Is data-centric
- Is secure
- Uses Internet Protocol (IP) addressing scheme
- Uses rich semantics for communication (not points or register-based) – interoperability
- Is extensible and designed to be adapted in the future
- Is redundant and fail safe
- Can be deployed on premise, in the cloud, distributed on the grid, or any combination of those options
- Employs distributed hierarchical control and coordination
- Is designed with expected communication losses
The future grid must transform from a top down to a bottom up hierarchical paradigm and become much more nimble and adaptable. The systems need to be able to scale to thousands, hundreds of thousands, or millions of assets. The technologies implemented need to become more aligned with Industrial Internet of Things (IIoT), to include virtualization and containerization, standard information models and message buses, DevOps techniques, microservices, and the flexibility and scalability provided by cloud. The systems should be designed with the intent of fully leveraging the latest mobile and wired communication technologies (e.g. 5G) delivering improved communication capabilities but continue to safely run autonomously when communication is interrupted.
New assets need to provision themselves and become active, dispatchable system elements quickly and simply. The system needs to be secure and resilient to local outages, whether the failure is device, weather or an intentional physical or cyber-attack, and prevent cascading events. Transmission and distribution systems must remain in balance by leveraging both distributed generation and demand response DER resources. Customers of all sizes and new business entities should be able to choose to participate in markets allowing open access. All of this must be done carefully and practically - enabling "brown field" legacy assets and systems to continue to operate while the transformation occurs. The EnergyIoT Conceptual Architecture we will be describing in this article series is designed with these fundamental ideas as its guiding principles.
What is happening here?
In its most simplistic terms, the energy industry is experiencing the merging of IT and OT. This transformation is the poster child for the Industrial Internet of Things (IIoT). When considering the sheer scale, huge volume of data, opportunities for analytics and Artificial Intelligence, the amount and distance of distributed assets, the electric power industry is literally the “Poster Child” for IIoT.
Figure 1- Energy, the Poster Child for IoT
The conceptual drawing in figure 2 is a simple representation of some very complex system of system relationships. The figure is abstract enough to easily represent the big picture, but also be able to consider each of the components individually and have a good understanding of what role they perform, the data they generate, and how each element interacts with other ecosystem components. The architecture is event-driven and data-centric. At its core, the architecture is loosely-coupled and includes state-of-the-art security and access control techniques that minimize the probability of cascading events due to natural disruptions or a physical or cyber-attack.
Figure 2- EnergyIoT Conceptual Model
(developed for this series with the GridWise Architecture Council)
Each grouped set of items within the two clouds and box are the “domains” of the conceptual architecture. The two IT domains (Energy Systems Cloud and Energy Services Cloud) include virtual components of the EnergyIoT ecosystem – software, middleware, core services, and data. The IT domains can live anywhere – on-premise at utility data centers, hosted in the cloud, or deployed on physical grid assets. The OT domain includes the physical components – the grid, sensors, and machines. The physical and virtual need to work hand-in-hand.
The Stack View
Another way to consider the power of this architecture is through a technology stack view.
Figure 3- Energy IoT Reference Architecture
(developed for this series with the Gridwise Architecture Council)
Figure 3 shows the Operational Technology (OT) layer of physical grid Assets at the bottom part of the stack. The Services/DevOps layer resides in the middle and abstracts the Assets from the Roles layer on the top. The Roles layer includes the businesses, consumers and other stakeholders who ultimately use and operate the assets. The key point of this diagram is that most of what is represented already exists. In fact, the green-colored services in the middleware can be found today on any of the major cloud vendor’s sites. The yellow-color shaded rectangles are new services that must be developed and tuned specifically to support the operational and compliance requirements of the electric power industry. There are probably some new services yet to be discovered, but a large number of building blocks are available today. It will not be a simple or instantaneous transition and inevitably there will be problems to be solved, but assembling an energy-specific Services layer and developing the missing yellow data-centric and associated services in Figure 3 can help propel industry-wide adoption of the EnergyIoT architecture ecosystem we will be discussing throughout this series of articles.
The OT Domain includes all the physical assets that make up the electric power grid. This includes generators, wires, substations, switches, poles, transformers, capacitor banks, remote terminal units, taps, solar panels and inverters, energy storage and any other devices yet to be invented. As DER penetration grows, the grid will need to become more nimble and adaptable – a “neural grid” – that has assets that communicate and cooperate with one another to support stability and resilience. It includes the following sub-domains:
- Sensors and Measurement: Required for situational awareness for systems, devices and people.
- Bulk Generation: Needed to support bulk and baseline power and ancillary services for reliability.
- Telecom Infrastructure: Essential to support grid asset interaction.
- Aggregators and Community Choice: Provides a means for all in the community to participate regardless of financial means.
- Smart Homes, Buildings & Cities: Required systems and assets to communicate and/or dispatch assets in homes, buildings, and cities.
- Distributed Energy Resources: Enables all forms of distributed generation, energy storage, or demand response to operate and interconnect safely.
- Security: Prerequisite physical and cyber security delivered through applications, devices and processes.
- Electric Transportation: Necessary systems and devices to support any form of electrified transportation.
- Neural Grid: Vital technologies that provide for safe, reliable, nimble and adaptive grid in timeframes faster than humanly possible.
This domain will be discussed in more detail in “EnergyIoT Article 4 – Operational Technology (OT) Domain”.
The Energy Systems Cloud Domain is a utility perspective of the application systems, devices, knowledge and processes required to plan and operate the grid. It is broken out by its sub-domains:
- Planning: Systems to enable forecasting, and short-term, and long-term planning.
- Construction and Maintenance: Required systems to support construction, geo-locating, and maintaining infrastructure.
- Comms and Security: Required systems for voice and data communications, and physical and cyber security services.
- Customer: Required systems to enable customer engagement, program enrollment, interconnect, settlement, and billing.
- Markets: Required systems to enable transmission and future distribution markets.
- Operations: Required systems to enable safe and efficient operations of the transmission and distribution grid networks.
This domain will be discussed in more detail in “EnergyIoT Article 5 – Energy Systems Domain”.
The Energy Services Cloud or DevOps Domain is the heart of this architecture. The “Green Cloud” services provide a “common abstraction layer” between the physical and virtual, dramatically simplifying when changes occur to the OT physical model. Interoperability and streamlined communications between services, systems, and grid assets is delivered as a service. It supports an “energy-specific” rapid development environment, data repositories, and analytics. Based upon the services provided by the major cloud vendors, these existing structures, tools, and services support the DevOps techniques, scale, and virtualization necessary to support a DER-rich, bi-directional energy flow, dynamic and adaptive grid. As a “data-centric” architecture, the Energy Services Cloud is designed with data and supporting data services as the centerpiece of the entire architecture. This domain includes the following services:
- Smart Contracts, Digital Ledger Technology: Provide trustless, immutable transaction tracking systems utilizing technologies like Blockchain.
- Structured Data: Allow storage and retrieval of relational data.
- Unstructured Data: Deliver storage and retrieval of raw data files like NoSQL.
- Digital Twin Agent: Enable the virtual emulation of assets and groups of assets.
- Security and Identity Management: Automates encryption, identity management, relationship management, and access management.
- Source Code Management: Allows developers to maintain source code development, modification, and collaborative development while ensuring configuration management.
- Orchestration: Provision, execute, and choreograph processes.
- Microservices: Standardizes services to containerize (and orchestrate virtualization), communicate and translate between standards, middleware operations, and other common tools for energy-specific DevOps environment.
- Artificial Intelligence and Analytics: Amplifies operation and design engineer’s abilities for analysis and optimization.
This domain will be discussed in more detail in “EnergyIoT Article 6 – Energy Services (DevOps) Domain”.
Conclusion
Recognize that this is a completely different architectural construct from today’s hub and spoke architecture. The grid of the future is coming quicker than you think. New methods and technologies to scale and support rapid grid configuration changes are needed. This EnergyIoT Conceptual Architecture is designed to leverage today’s most advanced technologies, uses existing standards, and leverages the services from cloud providers to provide extensibility and scalability that is future-proof. This is a moonshot opportunity for our generation. It cannot be done by one company alone. It will take an army of people with similar vision and passion to solve today’s electric grid problems – and create the inclusive, de-carbonized, plug-and-play “grid of tomorrow”.
Let’s start a conversation.
The Article Series will continue with deeper dives into each of the three different domains. Please see our next article on Thursday, May 2 on Energy Central, “EnergyIoT Article 4 – Operational Technology Domain” for more details on the physical grid assets.
The rest of the article series can be found here:
About the Authors
Eamonn McCormick is the CTO at Utilicast, a leading energy industry consultancy. Eamonn is a passionate believer in the bright future of the energy industry and the importance of collaboration as the foundation for solving for our current industry challenges. He is a results driven technology leader with a track record of success. He has implemented strategic technology change at several large energy companies over the last twenty years in the areas of wholesale markets, transmission and energy distribution primarily. In addition Eamonn is currently chief architect of the Energy Block Chain consortium.
Stuart McCafferty is an accomplished Smart Grid technical executive with an innovative history, strong relationships in the utility and vendor communities, business and partner development, platform and solution design, go to market planning and execution, and practical application of existing and emerging/disruptive technologies. Prior to B&V, he was VP of EnergyIoT for Hitachi America, where he led the architectural design of a distribution system platform supporting microgrid and Distributed Energy Resource (DER) related businesses. At B&V, Stuart supports the utility, technology, and vendor communities in strategy and pragmatic application of DER that combines IoT best practices and technologies with energy standards and protocols.
Thought leader in the Internet of Things (IoT), Big Data, Cloud Computing, Artificial Intelligence (AI), Machine Learning, and connected home with practical application within the Smart Grid ecosystem. Expert in utility IT/OT and the application of DER and microgrids for resilience, economics, and reliability.
Stuart is a US military veteran, Air Force Academy graduate, an Energy Fellow for community resilience at the National Institute of Standards and Technology (NIST), an Energy “Expert” for Energy Central, and Vice Chair of the Open Field Message Bus (OpenFMB) user group.
David is the Chair of the GridWise Architecture Council since 2015 and has been a council member since 2013.
The GridWise Architecture Council (GWAC) is a team of industry leaders who are shaping the guiding principles of a highly intelligent and interactive electric system. The Council is neither a design team, nor a standards making body. Its role is to help identify areas for standardization that allow significant levels of interoperation between system components. More about the Council can be found at www.gridwiseac.org
David is the current chair of the Technical Advisory Committee and a former member of the Board of Directors of the Smart Electric Power Alliance. He was also Chair of the SGIP Board of Directors from 2015 until 2017, and as a board member beginning in 2011.
In his current role, he is the Director of Technology Architecture and IT Transformation at the Electric Reliability Council of Texas (ERCOT). He began his career at Austin Energy Director of Information Technology Services for Austin Energy and was Deputy Director and Chief Information Officer for an $18B pension fund. He holds a BBA from the University of Texas at Austin and an MBA from St. Edward’s University.