EnergyIoT Article 5 – Energy Systems Cloud Domain
Image by Stuart McCafferty - No copyright
- May 9, 2019
- 792 views
EnergyIoT Article 5 – Energy Systems Cloud Domain
By Stuart McCafferty, Eamonn McCormick, David Forfia
Disclaimer: The viewpoints in this article and others in the series are the personal views of the authors and in no way are meant to imply or represent those of the companies they work for.
This is the fifth in a series of articles introducing an EnergyIoT Conceptual Model. This article focuses on the virtual Energy Systems Cloud Domain within the Model, which is highlighted in the image above by the thick red outlined cloud on the left-hand side. The Energy Systems Cloud Domain we envision:
- Provides services interoperably with any authenticated and authorized subscribers preventing data and functionality silos. Leverages message bus communications such as pub/sub
- Uses agents to connect to grid assets through an Energy Services Cloud
- Achieves interoperability using rich semantic information model standards
- Is cyber secure
- Reduces time and costs for systems integration using Energy Services Cloud services and standardized data definitions
- Accommodates opportunities for innovation, new systems and capabilities
Within the EnergyIoT Conceptual Model, at its core, the Energy Systems Cloud does not connect directly to physical Operational Technology (OT) devices. There are good reasons for this. First, if using a direct connection to a physical grid asset, when the capabilities of an asset change for any reason (bug fix, upgrade, security change), every device and its connection to system components need to be changed, tested and re-provisioned. Conversely, by using the Energy Services Cloud as an abstraction layer using virtual machines and “containers” (Docker) and orchestration (Kubernetes) combined with Digital Twin Agents can be used as a deployment mechanism. Using these technologies, problems with managing version control are solved and deployment is simplified as the software can be changed once and instantly propagate to any system using that newly modified asset type. Second, by leveraging the Energy Services Cloud abstraction layer, the data is decoupled from the systems and creates much richer opportunities for analytics and interoperability between other systems and services.
Planning groups and the systems they use provide utilities, System Operators and third parties with forward-looking predictions of how the grid network will perform and what the grid needs to operate within operational tolerances.
Long Term Planning (LTP)
LTP performs planning on a time horizon of greater than 30 days. It is typically found within the utility’s Distribution Planning Group. This group performs analysis and planning of the electric distribution system with the primary objective of ensuring that distribution infrastructure can support future grid services for load and distributed generation. New analyses performed as part of LTP include “capacity planning” and “non-wires alternatives” that consider implementation of Distributed Energy Resources (DER) (e.g. energy storage, microgrids, solar) instead of traditional infrastructure constructor or upgrades such as “re-conductoring.”
Short Term Planning (STP)
STP groups and the systems they use have planning time horizons less than 30 days. The STP works closely with the Distribution Grid Operator (DGO) and has real-time system information access. This group is responsible for the analysis and planning of the electric distribution system with the primary objective of supporting the DGO in providing week-ahead, day-ahead, hour-ahead, and/or real-time grid services for existing load and distributed generation. As more distributed resources are introduced on the grid, this new role will address the real-time operational planning function similar to a transmission system operator (ISO/TSO), but is focused on the electric distribution system.
Weather and Load Forecasting
Weather and load forecasting is mandatory with LTP and STP. Weather and load forecasting to predict both short term and long-term grid behavior and needs. Obviously, weather dependent solar photovoltaic (PV) and wind generation output profiles are directly affected by the amount of solar radiation or the speed of the wind. Forecasting systems continue to evolve allowing more granular capabilities with greater accuracy. Metrology devices for weather are now readily available and relatively inexpensive.
Customer systems manage the relationship between customers and the utility or third-party provider. This includes the types of services provided to each customer as well as the electrical and (potentially) market relationship of individual customers to their local grid and other actors.
Customer programs are services that customers can voluntarily participate in through a utility offering. Utilities use these programs to manage peak loads, comply with energy efficiency and other policy mandates and to implement new rate designs to incent customer behavior. By participating in these programs, customers are normally offered some form of additional financial benefit by providing a service to the utility or a third party. This can be in the form of demand reduction (Demand Response and Energy Efficiency) or Distributed Generation. Some programs, like the popular thermostat customer programs found at numerous utilities, are under scrutiny since the customer can unpredictably stop participating simply by changing the thermostat setting prior to the Demand Response (DR) event and still achieve the program reward without actually reducing their demand when it was most needed. By introducing a market to the distribution networks, bidding four degrees into a DR market would have no meaning. Instead, a Home Energy Management system or similar application would, for example, bid 2KW of load reduction from 5PM to 6PM. Some customer programs like the thermostat example are not likely to be useful in a distribution market where bids are KW, KWH, or some other ancillary services market metric. These customer programs will be replaced by other aggregation services and a an open market.
Metering systems are literally the “cash register” for utilities. Most of today’s Advanced Metering Infrastructure (AMI) systems use a polling methodology to retrieve Kilowatt hour (KWh) usage and generation information and transfer it into a head end system that checks data quality. The data is used by Measurement and Verification (M&V) and Customer Information Systems (CIS) to create and present bills to the customer. These systems are one of the “data silos” discussed in Article 2 - Architectural Challenges to the Energy Transformation. These systems are difficult and expensive to make interoperable and create latency issues providing timely data to other systems. In the EnergyIoT architectural paradigm, metering systems will evolve to include self-reporting meter Digital Twin Agents that will be able to deliver timely, accurate, secure, valuable data for authorized market, M&V, billing, and analytic systems.
Today’s interconnection agreement systems are inadequate to support the grid of the future. These systems are a critical piece for identifying third-party and customer DER in front of and behind the meter that includes DER type, capability range, capacity and geospatial information. This information is used to support electrical connectivity and power flow models to accurately predict local grid status properties and identify grid needs for both the long-term and short-term planning teams. Even the most efficient Interconnect systems can take several days to approve customer DER connections and create formal contractual agreements for grid connections. Other systems can take weeks, months or much longer causing some customers eventually give up. In order to transform to a carbon-friendly, efficient, open, and DER-rich ecosystem, this must change. Changes will be required to interconnect systems and processes, and will also require that DER vendors build assets that self-announce, self-define, self-provision and bind themselves to operational communication and/or simulation capabilities of Digital Twin Agents. More on this in Article 6.
Measurement and Verification (M&V) System
In today’s once a month relationship between customer and utility, M&V systems are simply based on meter reads. In the transformed grid that include distribution markets, this must change. Customers will bid energy - KWh whether usage or output - or some other measurable ancillary service to support distribution grid operations. The Distribution Market Operator (DMO) M&V system will determine whether the contractual obligation was met by the customer as expected during the time period for the bid. The customer must have his own M&V system that includes metrology devices that can be used to perform validation. It is likely that the Energy Services Cloud will have M&V services that can instantly determine whether the DMO and customer systems are in agreement and flag the ones that are not for human review. These services will be new opportunities for vendor companies to provide building measurement hardware and cloud settlement services that dramatically simplify and ensure fairness when open, democratized markets are enabled.
Customer Information (CIS), Settlement, and Billing Systems
CIS, settlement, and billing systems have been around for a long time - since customer’s needed to pay for electricity. CIS systems track customer names, addresses, contact information, and customer program enrollments. Settlement systems use M&V data to determine bills for each customer. Billing systems prepare customer bills and collect payments. Although there will be some evolution with these systems, they will adequately support an EnergyIoT transformation.
Operations systems are the heartbeat of the DGO and manage the real time operations of the distribution grids. The systems included in operations address:
- Situational awareness – real time visibility/monitoring of grid status
- Control – management of grid assets to maintain the transmission and distribution networks within acceptable operational tolerances
Within the EnergyIoT architectural paradigm, the mechanisms used by operational systems to communicate with grid assets will be much different than the traditional SCADA communications used today. The use of message buses, common semantic message payloads, and Digital Twin Agents will simplify communications, allow for much faster integration of new grid assets, and enable more autonomous operations to occur on the grid to enhance a more nimble and adaptive grid. This will be discussed in more detail in Article 6 - Energy Services (DevOps) Domain - The Heart of the Ecosystem.
Transmission ISO/RTO Systems
Transmission Operators perform operational functions that include planning and transmission operations. In the top-down architecture that currently exists, the ISO plans for capacity needs, qualifies generation and load providers to ensure they can meet contractual requirements for power supply, and manages operations to deliver enough quality power to radially flow down to meet the loads. Operational needs are met through Energy Management Systems (EMS) which can perform direct or indirect dispatch of generation and demand resources such as grid scale energy storage or demand resource load resources. In a bottoms-up EnergyIoT architecture, Transmission Operators will continue to fulfill the same role, but planning functions will be more granular at the distribution level and will either be shared or moved to the DGO with a goal of reducing uncertainty and ancillary services costs at the bulk generation level. The amount, mix, and location of generation will also change as more Distributed Generation will be generated near the loads and bulk generation is driven more and more to a mix of renewables and energy storage.
Distribution Grid Operations Systems
DGOs are responsible for the real-time operations of the electric distribution system within its jurisdiction. This role is currently supported by the Electric Distribution Operations group at the Utility. Since the DGO operates the distribution grid, it is also responsible for distribution reliability and safety.
At today’s modern utility, situational awareness and control is accomplished through a Distribution Management System (DMS) or an Advanced DMS (ADMS) that includes outage management functions. These systems can be very expensive and may cost hundreds of millions of dollars and require a lot of “care and feeding” as even minor grid configuration changes require customized modifications and system integration routines.
Further complicating the DGO role, dated Supervisory Control and Data Acquisition (SCADA) telemetry systems are used to provide direct communication connections between grid assets and operational systems. This point to point communication mechanism is difficult to change, complicated to configure and requires highly specialized and often overworked SCADA engineers to create a “point” within the SCADA system and write custom code to convert that meaningless point to engineering units that can be digested by other systems. Troubleshooting with these systems is complex with a variety of potential failure points ranging from configuration, connection, integration, data integrity or the physical asset itself, which could be a hardware malfunction or firmware issues. In the EnergyIoT architecture, SCADA systems will either need to evolve to support semantic language messaging payloads or, more likely, be replaced by modern message buses and operational Digital Twin Agent solutions.
Distributed Energy Resource Management Systems
Perhaps the biggest challenge facing utilities today is managing DERs in a coordinated way that promotes rather than impedes grid reliability. Even if the utility knows the DER is deployed (which most do not), utilities can only directly control registered DER in front of the meter. This is not the fault of DER Management System (DERMS) providers but rather a by-product of centralized command and control operations, regulatory rules requiring the utility’s reach to stop at the meter, disparate communications systems that include utility telemetry systems, cell net, and public internet, but most of all, the continued use of legacy registry-based protocol standards like ModBus, OPC, and DNP3. Using these legacy standards rather than rich semantic information model messaging standards AND requiring direct connection for each system to the physical asset, creates brittle relationships to any asset configurations that takes time and money to connect to and maintain. Instead, by using existing semantic standards like IEC 61968/70 (CIM), IEC 61850, OpenFMB, and IEEE 2030.5 as messaging payloads and marrying systems to Digital Twin Agent “adapters” to translate the messages into whichever protocol and proprietary configuration the vendor used to control the DER asset resolves these issues. This is a fundamental principle of the EnergyIoT architecture and one of the primary reasons that the Energy Services Cloud (DevOps environment) is a radical yet practical need for the electric power industry going forward.
The legacy electricity wholesale market system has enabled utilities, Independent Power Producers (IPPs), and now “qualified” third-party aggregators to trade and potentially make money in bulk energy and ancillary services markets. As currently designed, there are significant barriers for smaller players to participate is these markets.
The economics of DERs have changed and can now be purchased by businesses and homeowners at prices that have justifiable Returns on Investment (ROI). This is especially true in areas where energy prices are increasing and the price of DERs decreases. Regulators, ISOs and utilities are studying the concept of creating a Distribution System Operator (DSO) that would animate new distribution markets and engage DER owners. Imagine what a transparent, open, and broadly inclusive market would do for the electric power industry. It would accelerate the adoption of DER, create opportunities for innovation and new businesses, and completely change the way the grid operates and the way that consumers, producers and machines interact with it. As conversations about DSOs increase in number and intensity, it is inevitable that distribution markets will emerge in the coming years, allowing even homeowners to participate in the grid and driving faster and faster adoption of DER.
Transmission market pricing comprises four (4) algorithm components:
- Energy Market – power “product” sold and purchased
- Ancillary services – consists of necessary services such as reactive power and voltage control to ensure power quality and supply/demand imbalance thresholds are within tolerance
- Reliability or capacity – a market to make available generation asset to dispatch if needed
- Transmission congestion – address costs associated with delivering power over inadequate power lines and losses associated with transmitting power over long distances
The types of transmission market contracts include auctions, real time or “spot”, and bilateral agreements. In many transmission markets, prices are based on a grid location and the available resources and capacity to meet the local grid needs. Locational Marginal Pricing (LMP) is calculated based on a “node” on the transmission system and reflects the cost of delivering power to that specific location.
The transformation to an EnergyIoT architecture is not expected to change the requirements or behavior of transmission markets but may introduce new technical requirements for data exchange between the existing market and the EnergyIoT architecture.
EEstablishing new DSO functions that include an open and transparent distribution market is another evolutionary catalyst required to create a “new energy economy” that empowers utilities, consumers, producers, third-party aggregators, technologists, and new business models to create more efficiencies, cleaner and cheaper power, better reliability and more resilience.
The spectrum of DSO models ranges from highly centralized transmission-level models to highly decentralized peer-to-peer models. In-between those two extremes, exist models that called “nodal” models, which represent a physical location on the distribution network such as a T-D substation or a feeder, which becomes a separate nodal market instance for the actors on that Local Distribution Area (LDA) node.
In the EnergyIoT model, it is anticipated that there will be market services that support a wide variety of different retail market types across the spectrum described previously. These reusable and configurable services will support the calculation of pricing, bids, and settlement. These services do not currently exist. See DSO Models for Utility Stakeholders - DSO Organizational Models for a Digital, Distributed Modern Grid for more information on Distribution Market models.
Communications and Security
TThe evolution to 5G networks will have a profound and positive impact on utility operations. We are already seeing early deployments of 5G in Chicago and Minneapolis. 5G is coming quicker than you think – and systems should be designed for much faster (up to 20x over 4G LTE) fiber-like speeds and always-connected IoT devices. System designs must account for expected communication losses with the ability to continue operations autonomously in the last authorized command set received.
Cyber security is obviously a major topic and concern for grid operators. Cyber security systems include private/public key management, identity management, and access management. These systems support message encryption, ensuring that people and assets are who they say they are, and only allowing authorized and authenticated actors access to systems, services, and assets.
Network & Telecom Management
Network and telecom management systems provide configuration and health monitoring capabilities for operational and market needs. It may include predictive analytics and other analytics that search for tampering, sabotage, or other bad actor activities that target grid telecom networks.
Equally as important as cyber security concerns, physical security systems are there to protect physical assets from bad actors. These systems could be public safety applications that perform facial or other biometric recognition to flag potential intruders or provide access to restricted areas to authorized users.
Construction and Maintenance
The electric power infrastructure is the biggest interconnected machine in the world with over 360K miles of transmission and 6.3M miles of distribution lines in the US alone. Aging infrastructure needs to be replaced and maintained and systems are needed to manage the people, technology and the assets being modified. New infrastructure is needed to meet the growing needs of customers and also to take advantage of new technologies to reduce costs, increase reliability and resilience, and move towards a carbon-free electric power ecosystem.
Asset Management systems are a repository of asset information with functions that support the lifecycle maintenance of the assets. This includes design, construction, operation, maintenance, upgrades, replacement and decommissioning. The system is also used to coordinate across the assets and work tickets to manage risk, optimize costs and maximize performance. With millions of assets on the grid, Asset Management systems are a critical and necessary piece for safe and reliable operation.
Workforce Management systems main function is to forecast and schedule work tickets for utility and contractor staff. These systems minimize travel time between work sites to create maximum use of staff time and optimize the amount of time spent on productive work. Workforce Management systems continue to become more and more sophisticated, leveraging new technologies and creating mobile solutions that track progress and allow work dispatchers and workers to communicate in real time.
One of the most critical systems within utilities and in support of the EnergyIoT architecture model is the Geographic Information System (GIS). These systems provide precise geospatial locations of all of the assets deployed. Not only is this helpful when dispatching a crew to repair a problem or avoid damaging assets during construction or maintenance, they also have the more important role of providing asset location information to grid connectivity and power flow models. The connectivity information is used by those systems to simulate power flow within a certain grid territory to predict future behavior based on the assets available, weather, historical loads, and potential failures. This information, in turn, is used to forecast “grid needs” over some period of time (real time, minutes/hours ahead, days/weeks ahead, months/years ahead). In the dynamic, adaptable grid of the future, the GIS system and associated connectivity and power flow models are critical in supporting grid operations and market activities. Knowing precisely where assets are located cannot be under-emphasized. In fact, geospatial asset data are required information in all seven (7) of the EnergyIoT Energy Systems Subdomains – Planning, Customer, Operations, Markets, Communications and Security, and Construction and Maintenance.
Frankly, the way the current Energy Systems architecture is constructed is a big part of the problem the industry is experiencing in scaling, integrating, and incorporating new distributed assets into the ecosystem. The deployed systems collect data, store it in proprietary databases, require “spaghetti” interfaces to integrate with other systems, and create latency when sharing data. They are designed as siloed systems.
Today’s most successful and profitable technology companies learned years ago that it’s not about the systems, “IT’S ABOUT THE DATA!!”. In the EnergyIoT Conceptual Model, system names like DMS, SCADA, or AMI are not sued - the system functions are used instead with the deliberate intent of driving the discussion around the functionality and data requirements, and not leap to solving the industry’s needs with legacy systems used in the past. As an industry, investment decisions are driven by conversations about which legacy systems are needed. Instead, the conversation should start around the true data requirements. As an example, many industry conversations begin with “We need an AMI system,” rather than “We need to collect and store energy usage data for M&V, settlement, billing, and analysis.” Or, “We require a GIS system,” rather than “We require precise geo-spatial locational data for every asset for power flow modeling, operations, and planning.” Investment discussions must START with the conversation about data. This will lead to scalable, message-based, interoperable systems that are easier to maintain and evolve, cost less to implement, and create real stakeholder value and opportunity.
Introducing a modern, data-centric architectural construct is needed. The next article will describe the centerpiece of the EnergyIoT architecture, the Energy Services (DevOps) Cloud domain. This new architectural “layer” democratizes data, provides services for systems and OT assets to abstract and simplify communications, and includes a DevOps environment for developers to create energy-specific applications, services, systems, and solutions.
This is the fifth in a series of EnergyIoT articles being published on Energy Central. On Monday, May 13, we will publish the sixth article, “EnergyIoT Article 6 – Energy Services (DevOps) Domain - The Heart of the Ecosystem”on Energy Central and LinkedIn.
 The concept of Digital Twin Agents has been discussed internally by the authors at great length. There is not unanimous agreement, but there is consensus. We will introduce the author's concept of the Digital Twin Agent with much greater detail in Article 6 - Energy Services (DevOps) Cloud Domain - The Heart of the Ecosystem.
The rest of the article series can be found here:
About the Authors
David Forfia, Gridwise Architecture Council Chair
David is the Chair of the GridWise Architecture Council since 2015 and has been a council member since 2013.
The GridWise Architecture Council (GWAC) is a team of industry leaders who are shaping the guiding principles of a highly intelligent and interactive electric system. The Council is neither a design team, nor a standards making body. Its role is to help identify areas for standardization that allow significant levels of interoperation between system components. More about the Council can be found at www.gridwiseac.org
David is the current chair of the Technical Advisory Committee and a former member of the Board of Directors of the Smart Electric Power Alliance. He was also Chair of the SGIP Board of Directors from 2015 until 2017, and as a board member beginning in 2011.
In his current role, he is the Director of Technology Architecture and IT Transformation at the Electric Reliability Council of Texas (ERCOT). He began his career at Austin Energy Director of Information Technology Services for Austin Energy and was Deputy Director and Chief Information Officer for an $18B pension fund. He holds a BBA from the University of Texas at Austin and an MBA from St. Edward’s University.
Eamonn McCormick, Chief Technology Officer, Utilicast
Eamonn McCormick is the CTO at Utilicast, a leading energy industry consultancy. Eamonn is a passionate believer in the bright future of the energy industry and the importance of collaboration as the foundation for solving for our current industry challenges. He is a results driven technology leader with a track record of success. He has implemented strategic technology change at several large energy companies over the last twenty years in the areas of wholesale markets, transmission and energy distribution primarily. In addition Eamonn is currently chief architect of the Energy Block Chain consortium.
Stuart McCafferty, IoT Architect, Black & Veatch
Stuart McCafferty is an accomplished Smart Grid technical executive with an innovative history, strong relationships in the utility and vendor communities, business and partner development, platform and solution design, go to market planning and execution, and practical application of existing and emerging/disruptive technologies. Prior to B&V, he was VP of EnergyIoT for Hitachi America, where he led the architectural design of a distribution system platform supporting microgrid and Distributed Energy Resource (DER) related businesses. At B&V, Stuart supports the utility, technology, and vendor communities in strategy and pragmatic application of DER that combines IoT best practices and technologies with energy standards and protocols.
Thought leader in the Internet of Things (IoT), Big Data, Cloud Computing, Artificial Intelligence (AI), Machine Learning, and connected home with practical application within the Smart Grid ecosystem. Expert in utility IT/OT and the application of DER and microgrids for resilience, economics, and reliability.
Stuart is a US military veteran, Air Force Academy graduate, an Energy Fellow for community resilience at the National Institute of Standards and Technology (NIST), an Energy “Expert” for Energy Central, and Vice Chair of the Open Field Message Bus (OpenFMB) user group.