Or what an Avalanche Training taught me about trusting AI in Grid Operations
Disclaimer: This article was originally published on the AI-native Power Grid Channel on Substack, but adapted and refreshed for the Energy Central Community.
People who know me know I am burning for two things: backcountry skiing and data / AI for utilities. Some (or many) people would even say I am boring, because these are the only things I really love to talk about. Almost all my keynotes or presentations start with one of my skiing stories. Maybe for a good reason, because the mountains often teach great lessons for the grid, too.
Surviving a steep snowy face does not prove your decision to drop in was right. Turning around below the summit does not prove it was wrong. The grid has the same problem. But unlike the mountain, we can check.
The missing Feedback from the Mountains
The ski season is starting soon. And as always in early November, I go to some avalanche courses. So I did last week, too. I have done ski touring and mountaineering for many years. And I always thought I knew quite a few things. Still, every time I learn something new. And from Tuesday's course there was one idea that stayed with me.
The mountains almost never give us real feedback on our decisions.
Think about it. We head out. We look at the snow. The wind. We check the slope angle. We analyze the terrain. We decide to go up. We decide to ski it. We drop in. Make some nice turns. Nothing happens. We have a beer. Or two. Talk about our great line. The adventure. And feel we made a good call. But still, maybe we judged the conditions completely wrong. Maybe the face was close to release. Maybe we were close to triggering an avalanche. Maybe we were just lucky. The mountain doesn’t tell us. Nobody tells us.
Or we decide not to go. We turn around because it feels too risky. Because the slope angle is 31 degrees. Or just our gut feeling says: no. Again, nothing happens. Maybe we were right. Maybe the slope was perfectly fine. We will never know.
The clear feedback is the one nobody wants: getting caught in an avalanche. Luckily, the mountains almost never give us this one.
Trained by Experience or trained by Luck
Still, we continuously gain a lot of knowledge. We learn. We get more experienced. But the mountains do not teach us whether our judgements and decisions were right. Even worse, the mountains might teach us wrong things and wrong behaviour. Every lucky run gives us more confidence. Every lucky line gives us more courage and boldness. Every lucky day gives us more trust in our own decision-making or judgements. But we might be trained purely by luck.
And that is exactly the reason why avalanche professionals and ski guides use structured decision methods and checklists. Because they understand their decision making, their gut feeling, might be trained by luck.
The Control Room has the same Problem
Now let’s move from the mountains to the grid. Let’s move to the control room.
Think about it. An operator observes. Analyses. Reasons. Judges. And takes an action. Nothing bad happens. Was it a good decision? Was it the best decision? Maybe. Or maybe the N-1 margin just absorbed the issue. Nothing tripped. Nothing logged. Nobody ever will know how close it was to triggering something bigger.
Or an operator rejects and overrides a recommendation from an AI. Nothing happens. Was the operator right? Was the AI right? We don’t know. It somehow worked out anyway. And the other path, the path that might have been the best one, was never taken.
Or an AI recommends a conservative action. Curtailment, redispatch, a topology change. The cost of action shows up in the next operational report. But the outage it might have prevented: not visible.
When we train AI on historical operator decisions, we only train it on the paths people actually took. We train the AI to learn our habits. And our luck. But it never learns about the runs we never skied.
So when someone tells me “the AI was right 500 times” or “the face was safe the last 500 times”: how do you know? Just because nothing happened, it is not a great proof of a good decision. Not for ski guides. Not for operators. And not for AI either.
The Grid has one big Advantage
In the mountains we cannot analyze or evaluate what would have happened. We cannot replay the conditions. The climb. The run. We can’t.
Luckily, for the grid, we can do so. We have models. We can run power flow. We can run dynamic simulation. We can replay the decision. Even the one that was not taken. We can calculate how close we were to a limit. And not only whether we crossed it.
In grid operations, we can do things a mountain guide can only dream of:
Run AI in shadow mode. Log every reasoning and recommendation. Log every decision. Also the rejected ones.
Replay the rejected or not-taken actions in simulation mode. See the outcome.
Calculate the margin to failure. How close we are to limits.
Collect near misses.
Learn from them, like aviation has done for many, many years.
In a recent post “Trust in what we know, trust in what we do”, I introduced four (logical) layers: measurement, trusted state, cognition and control. To be able to replay scenarios, you must have the second layer in place. If you want to replay a decision, you must trust the state of the grid at the exact moment the decision was made: the Trusted State of the Grid.
Trust needs real Feedback
Trusting AI in the control room should not be built on “nothing bad happened the last x times”. This is the exact failure mode that gets skiers caught in avalanches. Trusting AI in grid operations must be built on decisions we can check, replay, measure and audit.
In the mountains, we cannot simulate the other scenarios. On the grid, we can. Not doing it is a choice.
See you on the mountain. Or in the control room.