Squirrel up to No Good – Part 1
- January 1, 2019
- 426 views
It was a couple of weeks before Christmas, and my wife (Nancy) and I were both working on the Bolognese Club of America (BCA) Newsletter. No, not pasta sauce, a saucy puppy (Bichon Bolognese), a rare breed that came out of Italy about 800 years ago. Our office dog (Joey) is a Bolognese, and Nancy is Treasurer and Newsletter Editor for the BCA. See http://boloclubofamerica.org/bca.htm for more details.
We were rushing to complete the Holiday Issue so our members could receive it well before Christmas,
And then the lights went out. Ditto Nancy's computer and my big display (my laptop didn't mind). Nancy used a mild expletive, and said the obvious: "Power's out." In 10 or 20 seconds the power came on, and stayed on. Thank you, PG&E Cornerstone Project (section 3 below)!
About an hour later I went to our gym at the Stanford Healthcare campus in Livermore for a workout that never happened. Yep, the power was still out. As Nancy called our friends around Livermore, we found that some (like us) lost power for a few seconds, some for a few minutes, and some for a few hours. We found out later that a squirrel had gotten into some PG&E equipment. When Nancy told me this, Joey looked at me seriously, as if to say "I knew those squirrels were up to no good."
Thus was born this article on outage management, and how it has evolved over the years to restore power as quick as the laws of physics (and mischievous rodents) allow. This paper is mainly on history, hardware and system-level solutions, and a second paper in this series will be on metrics and software solutions.
It was in the mid-1980s, I was working for a company that would later evolve into the Siemens Power TG business unit. We had introduced a small SCADA system that was much more advanced than any of its competitors, our market share was rapidly expanding, and I was on a road trip somewhere in the southeast. Most of our small system customers were small cooperatives and municipal utilities. There were lots of these in this area, and we were visiting two or three of them a day demonstrating our system. We were talking to the engineering manager in a utility's headquarters when the lights went out.
All utility employees in the HQ (maybe 15 or 20) scrambled into their long-practiced assigned positions. Some started answering the phones (first calls came in about 10 seconds after the outage), and a few grabbed the "supplies" (slips of paper and brightly-colored pins). As each employee received a call from an outaged consumer, they wrote down the customer's basic information on a slip of paper, and passed it to another employee. Behind the reception desk of the utility was a large map of their service area complete with all of the distribution circuits. As the slips were read, a pin for each outaged customer was pressed into the map. After about 20 pins were in place, the outage was defined: A single feeder, and the same one that fed the HQ. Then engineering manager got on the phone to call the linemen to "drive the line", trying to identify the source of the outage.
Over the intervening years many improvements were made, but until after the year 2000, these were incremental, not monumental. The SCADA system we were selling made a major difference, because each substation had a remote terminal unit (RTU) that monitored all of the circuits in real-time, and the dispatcher (monitoring the SCADA system) received an immediate alarm when a circuit first tripped, and another when the circuit locked out.
Another technology that has been around since before 1950 are fault indicators. These are simple devices that can set an indication (mechanical flag or battery-backed lamp) when a fault current is detected. It can also use separate flags/lamps for circuit-to-circuit faults and ground-faults. By starting at the substation and following the tripped fault detectors, the lineman can be steered to the fault. These are particularly useful for more complex branched circuits when the fault is on a branch.
The next improvement required two technologies: First in the early 1990s The FCC allocated 40-each 900-MHz radio frequency channels for use by utilities. A small industry was created producing the radios that used these licensed frequency bands, and it became possible to package them with small distribution line devices like Cooper Form 4C Recloser Control and the S&C SCADA-Mate Switch Controller. These devices used standard SCADA protocols, and thus they could communicate with a SCADA system through the radios as if they were small RTUs. They were frequently placed out on the distribution circuits, and could be used to more clearly characterize faults, then isolate and restore faulted distribution circuits.
Reclosers have been around since the 1940s. Most faults on overhead distribution circuits are momentary, so a recloser combines a medium-voltage circuit breaker with a basic protective relay. When a fault is detected, the recloser trips, then waits a short time for the fault to clear, and attempts to reclose the circuit's (recloser's) breaker. Frequently a recloser will attempt to reclose multiple times before locking out, waiting a programmed time after each trip. Reclosers that protect branch circuits frequently operate with downstream switches called sectionalizers. These devices divide circuit branches into segments. Each sectionalizer will count the number of fault-current flows followed by trips, and open its switch after a programmed number of these sequences, thus allowing segments upstream of the faulted segment to be restored by recloser cycling. Note that a recloser's trip-current and timing settings must be coordinated with the same variables of the main (substation) circuit breaker to allow the recloser to clear its faulted branch without other parts of the circuit being interrupted.
Line switches are simple, basic switches that can disconnect a circuit. These switches are rated to interrupt the load-rating of a circuit, but cannot interrupt a fault like a breaker. Thus they must operate on the dead-circuit after a trip to isolate a faulted segment.
3. Fast-Forward to the Present
I generally focus on PG&E as they are my home utility. They have aggressively implemented smart grid functions. One part of this effort was the statewide AMI/demand response systems implemented by the California investor owned utilities. These systems are described in part 3 of the series I posted earlier on AMI (link below). PG&E started this effort in 2009, and used both Aclara and Silver Springs Network AMI systems (mostly the latter, which is now owned by Itron).
Note that the linked paper was posted last spring, so the information is still current, but some links in this paper may not work.
All or virtually all AMI systems have an outage reporting function, and the Silver Springs system (as implemented in Livermore where I live) is very fast, reporting outages in seconds via last-gasp alarms that are immediately sent from each meter that has lost power via the Internet to the Silver Springs outage detection system.
A source that predates the AMI outage information is PG&E's interactive voice response system with speech recognition, which was installed in 2004. Since I've used PG&E outage reporting call-in system since well before 2000 (albeit rarely since our power is pretty reliable) I can attest to this system's effectiveness. However in the last eight to ten years it has been superfluous. Even if I called within a few minutes after the outage, the system already knew about it and had an estimate of time-to-repair. Ten years ago roughly corresponds to the time that the AMI system in Livermore came online. Also at that time that PG&E's Cornerstone Project was being implemented.
The Cornerstone project started with a proposal to the California PUC (CPUC) in May of 2008 for a $2 Billion project to improve the resiliency and reliability of its electric distribution system. To make a very long story short, after a series of hearings with all stake-holders, the CPUC rejected the original proposal, but approved a project with greatly reduced scope ($357 Million) in June of 2010.
In March of 2014 PG&E posted the accomplishments of Project Cornerstone. These included:
- Installed Intelligent Switches on More than 500 Electrical Circuits: Through December 2013, more than 230,000 customer interruptions have been avoided by utilizing this smart grid technology. In the event of an outage, instead of waiting for a crew to arrive on scene to restore circuits manually, the new devices do it automatically, often within minutes. Utility workers installed automated “intelligent” switches on almost 400 circuits last year; in total, more than 500 circuits have been enabled with this advanced “self-healing” technology.
- Upgraded Rural Circuits: PG&E installed more than 5,000 sets of fuses and 500 line reclosers on more than 440 of the worst-performing rural circuits to isolate service interruptions and minimize their impact on customers. The installation of this equipment has resulted in a 33 percent reduction in the number of customers experiencing sustained outages from 2010 performance levels.
- Enhanced Substations and Circuit Interconnectivity. Technicians replaced and upgraded substation equipment to improve operational flexibility and added circuit capacity to maintain or restore service when electricity needs to be rerouted in the event of an outage. These upgrades also had the benefit of handling an increase in demand during heavy loading conditions, such as hot summer days.
 Thomas Hoffman, Computerworld, "Speech Recognition powers Utility's Customer Service", Sep., 2005, https://www.computerworld.com/article/2559568/it-management/speech-recognition-powers-utility-s-customer-service.html
 CPUC, Decision 10-06-048 June 24, 2010 "Decision on Pacific Gas and Electric Company, Request to Implement a Program to Improve Electric Distribution System Reliability", http://docs.cpuc.ca.gov/Published/Final_Decision/119833-14.htm#P581_90189
 PG&E Currents, "Customers Have Benefitted as PG&E Completes Multi-Year Electric Reliability Project", http://www.pgecurrents.com/2014/03/06/customers-have-benefitted-as-pge-completes-multi-year-electric-reliability-project/