Coordination: A Discussion About the Operating System for a Smart Grid
Boiled down to its essence, the Smart Grid concept is all about coordination. Smart Grid proponents note that it will spur the development of microgrids, distributed generation, demand response, electric vehicles, storage and renewables by allowing these technologies to operate seamlessly and efficiently alongside central station generation. Indeed, all of these things are possible, but it's also necessary to sort out exactly how that coordination will take place. Just as the iPhone (or any other "smart phone" for that matter) has an operating system that coordinates operation of all the useful applications it can provide, a Smart Grid needs an operating system that coordinates the actions of market actors, large and small, from central station generating plants to the gadgets that many expect will someday help consumers manage energy usage in their homes and businesses.
In all of the many discussions I've read about, listened to and participated in, there's been no real consensus about what this operating system -- which allows disparate technologies connected at the transmission and distribution levels work together in a seamless fashion -- would look like, though there are three schools of thought on the matter:
- A centralized control paradigm in which the utility or the grid operator exercises direct control over suitably equipped devices (supply and demand) that are made available on a voluntary basis. A/C switches are the best known example of this idea.
- A centralized market in which loads and resources, either individually or via an intermediary, are required to offer supply and/or bid demand into a retail or wholesale market. Prices and quantities sold by generators and purchased by loads are determined by whether their bids and offers clear the market. In real time as energy is actually delivered and consumed, they are also expected to comply with operating instructions (dispatches) from the grid operator. This is the mode under which most RTOs and ISOs operate today.
- Decentralized decision making, in which loads and resources react to transparent prices broadcast by a central market operator and are charged/paid for what they consume/produce. The grid operator exercises little or no control, directly or indirectly except in certain emergency situations. Although this idea is not widely employed in electricity markets1, it is the basis for most other commodity markets.
This is the way vertically integrated utilities that are not part of a power pool or an organized market continue to operate. Utilities have directly controlled end-use devices for many years. The most notable example is air conditioning cycling programs in which the grid operator periodically switches off banks of air conditioning compressors for a portion of each hour on a rotating basis to reduce demand. Programmable communicating thermostats (PCTs) and EPRI's Vehicle to Grid (V2G) concept are other examples. In a Smart Grid context, grid operators and/or qualified intermediaries2 would exercise control over all grid-enabled devices -- both supply and demand -- where customers allow such control and comply with other applicable eligibility requirements. Two-way communications, presumably through some type of gateway that supports multiple devices, might be required in some situations while one-way communications might be acceptable in others. Consumers would have the ability to temporarily disable utility control in accordance with applicable tariffs. Customers who allow the utility to control their grid-enabled devices would receive Incentives in the form of either rebates or price discounts based on how well they perform3.
Perhaps the most compelling advantage of centralized control is that it is a known quantity. A/C cycling programs have been around for a long time. In many places, customers accept them and grid operators have learned how to utilize them. Grid operators also like the apparent certainty of being able to exercise control over the grid compared with the perception that devices not under their control are less likely to respond, or won't respond in a predictable manner. Centralized control doesn't rely on a market mechanism or dynamic pricing, both of which are controversial in some regulatory jurisdictions.
However, these advantages have to be balanced against a number of difficult challenges, including:
Customer Acceptance. At least some customers are likely to object to the idea that anyone, including their local utility, can reach inside their homes and businesses4 to control energy use.
Availability of Smart Devices. Many manufacturers of the smart devices that would interact with a smart grid are not enthusiastic about the idea of a command-and-control world. Service providers are similarly skeptical. This could limit the availability of suitable devices, limit both competition among the manufacturers that would provide them, and more importantly, discourage innovation. Moreover, some would argue that a switch controlled by the grid operator needs no real intelligence and consequently, doesn't need to be very smart.
Lack of Flexibility. By their very nature, A/C cycling and similar initiatives are inflexible and cannot easily be tailored to address variations in customer preferences or, more importantly, grid conditions that are likely to be less predictable as penetration levels of renewable generation increase. One-size-fits-all programs also can't easily deal with combinations of devices -- for example a PV array coupled with batteries at one home, and a fuel cell coupled with thermal storage at another.
Suboptimal Outcomes. Computer systems that attempt to "optimize" the use of individual devices will not be able to handle all of the possible combinations of supply and end-use devices, each with different efficiencies, duty cycle requirements and customer preferences as to their use. As thing stand today, the computer systems used by grid operators in organized markets to optimize the dispatch of generation occasionally fail to solve in the available time frames, even though they have to deal with a relatively modest number of grid-connected resources5. Adding millions of end-use and distributed generation devices means that either the amount of available computing power must be increased by many orders of magnitude at enormous cost in order to guarantee acceptably complete solutions in the required time frames, or the optimization problem must be greatly simplified which in turn affects the optimality of the solution.
Customer Privacy and Data Collection. Privacy advocacy groups have written extensively about the risks of collecting device-specific consumption data. Distributed generation exacerbates this situation to the extent the grid operator must also know how much each customer pays for fuel, the efficiency of each customer device, how much each customer pays for maintenance, and whether and how generation and storage are co-located, among other parameters, in order to incorporate distributed generation in its optimization6. Moreover, grid operators and intermediaries take on an additional burden by having to ensure that all of this data is up-to-date and correct in order to guarantee that their optimal solution does not inflict harm on individual customers even though customers as a group are better off.
Liability. Centralized control of the Smart Grid creates new legal liabilities of a type and scale that grid operators have never faced before (though intermediaries do deal with some of them today). First, they will likely have to assume a greater degree of liability for grid-enabled device under their direct control that fail to operate properly. Second, they will be held liable for unauthorized release of device-specific data, whether on the supply (e.g. distributed generation inputs) or the demand (meter data) side, no matter what the reason. Third, the scale and complexity of a central optimization means it will occasionally produce faulty outcomes. Customers as a whole may realize meaningful savings, but individual customers could see higher costs, whether it is because the optimization was unable to solve in the allotted time, the optimization algorithm was faulty, or certain customer-specific data was incorrect or out-of-date.
All RTOs and ISOs in the US and the Alberta Electric system Operator (AESO) operate in this fashion. The Centralized Market differs from Centralized Control in several respects. First, buyers and sellers must submit bids (to buy) and offers (to sell) consisting of price-quantity pairs, but the prices need not be strictly cost-based7. Second, an algorithm clears the market by matching up buyers and sellers subject to a variety of operational and power flow constraints, including transmission congestion. This typically takes place once around mid-day for the following delivery day (Day-Ahead Market, or DAM), and again periodically within each delivery hour (Real-Time Market or RTM) to deal with new information and residual changes in supply, demand, available transmission capacity and other system conditions. The Day-Ahead Market is generally financially binding whereas the Real-Time market is physical. Third, variations in amounts delivered or consumed between the financial Day-Ahead Market and the physical Real-Time Market are settled in cash. Fourth, while suppliers are discouraged from producing amounts that are inconsistent with their financial commitments unless otherwise instructed by the grid operator, typically these kinds of behavior are not prohibited or penalized as they would likely be under centralized control.
One of the important advantages of the Centralized Market approach is that it transfers some of the decision-making authority away from the grid operator and into the hands of market participants. Another is transparent prices that help form expectations about market participant behavior provide investment signals, and guide operating decisions.
The Centralized Market approach relies on optimization algorithms that are largely identical to the ones used for Centralized Control. To the extent market participants are allowed to submit multi-part cost functions rather than simple price quantity pairs, the optimization must be capable of handling them8. This means the Centralized Market approach is still prone to algorithmic flaws and it may still not reach a suitable solution quickly enough, especially as the number of participating devices and/or entities grows from a few thousand to a few million. More significantly, customers that wish to participate must understand and operate under an extensive, complex set of market rules and procedures that typically runs into the thousands of pages and imposes burdensome obligations and eligibility requirements, which makes participation difficult, time-consuming, and expensive. The high cost of participating also limits liquidity, which can lead to price volatility unrelated to market fundamentals.
Centralized Markets typically only provide limited forward price visibility, which means customers have little ability to act independently of the grid operator to address changes in grid conditions that could otherwise become apparent through changes in forward market prices. This is particularly important for integrating variable generation, which is subject to a relatively high degree of forecast uncertainty and some production variability, and for storage, which could very effectively self-manage its own state of charge given a stream of forward prices that reacts to changes in market and grid conditions, including the impacts of changes in forecast amounts of variable generation.
This idea builds on the centralized market approach by adding several elements and slightly modifying several others.
The first point of departure is providing more opportunities for market participants to make adjustments by clearing the market more frequently than once for all hours of the upcoming delivery day (Day-Ahead Market) and then again in real-time (Real-Time Market). A series of Hour-Ahead Markets (HAM) would be added to the Day-Ahead and Real-Time Markets that can include as many or as few forward-looking hourly delivery intervals as desired, though 24 would likely be the right number, and these could be cleared as frequently or infrequently as desired. Electricity retailers could take advantage of these more frequent update intervals9 to manage changes in weather and other hard-to-forecast conditions that affect their procurement decisions on behalf of retail customers. Large central station generators could better manage around unexpected outages. Variable generators could adjust their financially binding delivery commitments based on forecast updates. Grid-level storage and electric vehicles would benefit by being able to respond more flexibly without having to wait for guidance from the grid operator. In each case, providing more frequent opportunities to make adjustments reduces risks and costs.
The second point of departure is to provide a simplified method for retail customers to participate directly in the market. The availability of forward prices that adjust regularly over the upcoming 24 hours means smaller market participants could simply react to these prices (by acting as price takers) rather than having to formulate bids or offers, though they would have the option to do so and the forward prices would help them10. To the extent parties wish to submit bids or offers, they could do so via the same kinds of intermediaries that currently provide this function in many RTOs. Conceptually, this is similar to the way retail customers participate in equity markets today: larger parties make the market by submitting bids and offers directly to ECNs11, while smaller parties operate through retail intermediaries12 that aggregate bids and offers and submit them to the ECNs on a customer's behalf. Although market participants (via their intelligent, autonomous devices) could simply plan and execute their operation guided by the forward price stream but without submitting bids or offers (price/quantity or quantity-only), doing so would provide price certainty compared with the less certain and more volatile Real-Time prices that would be used to cash out deviations from forward commitments.
The third point of departure would be to simplify the algorithms that are used to clear the market, if necessary, so that they can be run more quickly and more often without requiring large investments in computing power. These algorithms would still have to consider transmission and system security constraints, but they could limit bids and offers to simple price/quantity pairs rather than allowing more complex bid curves.
If one of the key goals of a Smart Grid is to better integrate renewable resources, then one of the most important benefits of this approach is that it allows market prices to more readily reflect changes in system conditions that result from changes in variable generation forecasts and other factors, which in turn allows other elements of the grid to adjust accordingly. A forecast increase in wind production during the middle of the night would lead to a reduction in market prices, which might in turn induce the operator of a storage device to buy that low cost energy and it might also prompt other, flexible generators to reduce their output by purchasing back energy sold earlier at a higher price. This scenario is not possible today because once the Day-Ahead Market is cleared, the hourly energy markets tend to operate for no more than the upcoming delivery hour.
Another advantage is that it would facilitate the economically beneficial use of distributed generation and storage, electric vehicles and price-responsive demand. Consumption and production decisions, such as charging an electric vehicle now at a higher price rather than in four hours at a lower price, would be greatly simplified by knowing what those prices are and being able to act on them. Having opportunities to enter the market to make adjustments more frequently also simplifies decision-making and participation. Distributed generation, demand response, storage and electric vehicles would not require specific, proscriptive tariff provisions in order to be able to buy or sell electricity, or to beneficially take advantage of market information.
Unlike either the Centralized Control approach or the Centralized Market approach, the grid operator would not have to know anything about the economic details of the myriad supply and end-use technologies that participate in the market. It would not be responsible for gathering data, securing that data, and making sure it is up-to-date. Decisions taken by market participants based on forward prices would be the responsibility of the participants rather than the responsibility of the grid operator. To the extent some type of optimization algorithm is required to clear the market, it could be somewhat simpler and the required degree of computational effort to clear the market for a given hour would not have to expand to accommodate increases in the number of intelligent devices.
The Decentralized Market approach does have some drawbacks. First, the forward hourly markets may not attract enough participation to provide sufficient liquidity, which could lead to unacceptable levels of price volatility. Second, it would require changes in existing RTO and ISO market systems that could be costly and time-consuming to implement13. Third, there would likely be many prices for an individual delivery period14 rather than the single price that is inherent in today's RTO and ISO markets. Although multiple prices for a single delivery period that depend on the timing of the transaction are a common feature of most financial markets, they are still relatively rare in US electricity markets. Finally, some customer groups are uncomfortable with the idea of passing locational marginal prices directly to customers.
It's relatively easy to describe an end state. Defining a path toward that state that doesn't needlessly scare customers; disrupt their lifestyles, budgets and businesses; or raise unfounded objections is much more difficult. Following are some of the transitional issues that need to be addressed before the soul of a Smart Grid can emerge:
- Renewable resources have high fixed costs and low or no variable costs. At high penetration levels, they depress average market prices by displacing other generation with higher variable costs. Their presence also leads to increased price volatility as conventional resources are dispatched around them. Some price volatility is good for a Smart Grid, but how will volatility be perceived alongside market prices that are low on average?
- If customers pay for energy based solely on market prices that reflect marginal cost bidding, what mechanism allows resource owners to recover their fixed costs, particularly if market prices are depressed by the presence of low or zero variable cost renewable energy resources?
- How will changes in the predictability and timing of customer demand affect utilities? Does the underlying regulatory and institutional structure have to change as well?
- No RTO currently passes locational prices directly through to retail customers. Some customer groups are concerned about the impacts of doing so, yet the Smart Grid won't work well unless the coordination mechanism reflects system conditions. How should this dilemma be resolved?
- Will customers accept and be able to understand a bill that includes prices that change every hour?
- To address customer concerns over locational pricing, would it be sufficient to pass through locational prices to intelligent devices for coordination purposes while continuing to charge customers flat rates? Would it be problematic to pass through locational prices to intelligent devices while charging customers an average hourly price that has been averaged over all locations in a utility service territory? What gaming and adverse selection issues does this option raise?
Smart Meters and general concepts about what a Smart Grid might do are not enough. Regulators, policymakers, industry experts and stakeholders need to turn their attention to the operating system that will enable a Smart Grid to actually work.
References 1 A concept along these lines was proposed for California's wholesale electricity market in "Simplified Bidding for WEPEX", Cazalet, E.G. and J. Ellis, 1996. http://www.cazalet.com/images/Simplified_Bidding_for_WEPEX.pdf 2 Most RTOs and ISOs only deal with qualified intermediaries, which in turn deal with end-use customers. In California, the intermediaries are known as Scheduling Coordinators, while in ERCOT, they are known as Qualified Scheduling Entities. 3 In other words, customers would not be rewarded for simply allowing the control device to be installed. Disabling the device, even when allowed to do so, would affect the level of compensation. 4 Conspiracy theorists are already alleging that Smart Meters open to the door to government control over electricity use. For example, http://drscoundrels.com/?p=1131 5 Typically in the low thousands of individual generating units, if that many. 6 A system operator would prefer to know this information so that it can treat DG on an equal basis with central station plants. FERC has already done something similar by requiring RTOs and ISOs to ensure that demand resources can provide some of the same operational attributes so DR can be treated on an equal basis with central station generation. 7 Market participants can also be price takers by submitting quantity-only bids and offers, in which case they pay or receive the prevailing market price for their entire bid or offer volume. In most RTOs and ISOs, generators submit offer curves that are not unlike the price curves they would provide to the grid operator in the centralized control arrangement. 8 More precisely, a set of price/quantity pairs that must be monotonically increasing with respect to price, plus a value for starting and running at zero output. 9 It is likely, though, that they would. 10 Customer devices, including PEVS, could be pre-programmed to respond at certain prices, e,g, through auto-DR. 11 Electronic Communications Networks, which have largely replaced floor brokers and specialists (pit trading) in most financial markets. 12 Examples include Charles Schwab and E*Trade Financial. 13 Presumably such changes would be undertaken only if they produce benefits that outweigh the costs. 14 Potentially as many as 26 different prices -- one each for the Day-Ahead and Real-Time Markets for that hour, and an additional price every time the market for that hour is cleared between the close of the Day-Ahead Market and the time power starts to flow.
1 A concept along these lines was proposed for California's wholesale electricity market in "Simplified Bidding for WEPEX", Cazalet, E.G. and J. Ellis, 1996. http://www.cazalet.com/images/Simplified_Bidding_for_WEPEX.pdf
2 Most RTOs and ISOs only deal with qualified intermediaries, which in turn deal with end-use customers. In California, the intermediaries are known as Scheduling Coordinators, while in ERCOT, they are known as Qualified Scheduling Entities.
3 In other words, customers would not be rewarded for simply allowing the control device to be installed. Disabling the device, even when allowed to do so, would affect the level of compensation.
4 Conspiracy theorists are already alleging that Smart Meters open to the door to government control over electricity use. For example, http://drscoundrels.com/?p=1131
5 Typically in the low thousands of individual generating units, if that many.
6 A system operator would prefer to know this information so that it can treat DG on an equal basis with central station plants. FERC has already done something similar by requiring RTOs and ISOs to ensure that demand resources can provide some of the same operational attributes so DR can be treated on an equal basis with central station generation.
7 Market participants can also be price takers by submitting quantity-only bids and offers, in which case they pay or receive the prevailing market price for their entire bid or offer volume. In most RTOs and ISOs, generators submit offer curves that are not unlike the price curves they would provide to the grid operator in the centralized control arrangement.
8 More precisely, a set of price/quantity pairs that must be monotonically increasing with respect to price, plus a value for starting and running at zero output.
9 It is likely, though, that they would.
10 Customer devices, including PEVS, could be pre-programmed to respond at certain prices, e,g, through auto-DR.
11 Electronic Communications Networks, which have largely replaced floor brokers and specialists (pit trading) in most financial markets.
12 Examples include Charles Schwab and E*Trade Financial.
13 Presumably such changes would be undertaken only if they produce benefits that outweigh the costs.
14 Potentially as many as 26 different prices -- one each for the Day-Ahead and Real-Time Markets for that hour, and an additional price every time the market for that hour is cleared between the close of the Day-Ahead Market and the time power starts to flow.size="1">