Modernizing Your Data Management Approach

Written by: Gerald Gray and Michael Covarrubias

Let’s face it – we all know we have data problems. Quality issues, governance issues, transformation issues as data moves from one “cylinder of excellence” or “silo” to another. Maybe you implemented an enterprise service bus (ESB), but all that did was exacerbate the point-to-point interface issue – now everyone can make their own point-to-point interface for just “their” project. So, what do you have? Something complex and time consuming to manage! Something prone to errors and potentially even an increased vulnerability to security related incidents.

Now you are also looking to create an environment where you can enable machine learning, artificial intelligence, and you have a focus on data science overall – but if all you’re doing is piling up bad data – your machine can’t be trained nor tuned.  It is the age-old data problem of garbage in, garbage out.  Maybe you’ve already gone through the painful steps of creating an analytics platform while utilizing concepts such as data fabric or data mesh. How many of you started by mapping your data with Microsoft Excel? I’m sure the answer is – a lot of you. How did you feel about that process? Painful? Slow? Error prone? All the above?

If you’ve been involved with this kind of processes, you have probably experienced the situation where the same concept or object was used and perhaps defined in different ways depending on the system from which you pulled the data.  Consider the meter as an asset, information about the meter will be in the meter system, customer system, and the asset management system, but these three different copies can easily have conflicting, and supplementing information. Changes could have been made but they might not have been synchronized across platforms.  Let alone having a common definition of the meter and other data assets that the organization agrees to.

Why is a Semantic Model Important?

Consider a Forbes survey given to data scientists asking them how they spend their time. Overwhelmingly time was spent on what we refer to as data janitoring, cleaning, organizing, and translating data from one system to the next. Think about the amount of time spent on this instead of analytics and actual model training. Not having a common approach to data management is maintenance intensive and often results in re-work.

Semantic modeling is used to depict relationships that exist among specific data and establishes a common definition of the data. A recent article by Gartner Leverage semantics to Drive Business Value From Data advised data and analytics leaders to utilize a semantic approach to their enterprise data. If you don’t, you will be faced with the endless challenge of data silos.

Whether your approach to building your analytics platform is based on the concepts of data fabric, data mesh or a combination of both, or using knowledge graphs, semantic modeling is important because it speaks to data and governance. In the end, data must be understood and trusted. Semantic modeling allows you to exploit connected data; it also leverages common definitions that everyone across the organizations agrees upon.

Is there a good starting point?

After almost 30 years of development and continuing improvement as innovation and new technologies drive additional use cases, the utility Common Information Model (CIM) is widely recognized as the go to research and reference model from which to build an enterprise semantic model.  But despite this – we see many times utilities start their data science initiatives with “roll their own” data model. There is still the occasional person that wonders, “Is the CIM even a thing?”, or don’t even know that there is a CIM.  They decided to design their own because it seems faster at the outset. Creating an enterprise semantic model from scratch always seems faster in the beginning. Until the data concepts become more complex and one realizes that three decades of development is not something to be trivialized. In addition to the CIM as a reference model, vendor specific reference models are also needed, however, utilities who do utilize the CIM find that it provides close to 80% of what they need.

“Based on our experience, the CIM simplifies and supports utilities when capturing new data, for more analytics, intelligent automation, model training, and for outcomes benefiting all stakeholders.”

I like to describe the CIM as a starting point for the electric utility industry.  The end game is for the utility to have a well-defined data model and associations that are not tied to a specific technology or vendor.

Wait there is more – Meta what?

The data about data. The data about interfaces. Mapping and transformation rules. How to govern and catalog. How to secure. Metadata, the rich information associated with any given set of data. Metadata can inform why there are differences seen in the data – such as why there might be different identifiers in different systems for the same asset.  You must understand how you are going to connect the different metadata elements to the data, how else would you easily know what to change when a system needs to be upgraded or even replaced? Bringing together data from disparate systems is hard to develop, and hard to manage, but must be accomplished to scale and to provide accurate and trusted data to both internal and external stakeholders.

Modern data management considers different aspects of the data journey within your organization, starting with semantic modeling connecting data and associated metadata. Modern data management enables reusability and ensures a common understanding while reducing complexity and delivering value.

Xtensible Approach – Affirma

For over two decades, the Xtensible team has been working with and supported utilities in taking a semantic-based approach to managing their data, utilizing the CIM standard and supporting in the data management lifecycle.   

With Affirma, our Enterprise Semantic and Metadata Management Solution, we are taking our proven semantic modeling approach, and using the CIM as a reference model to help build your enterprise semantic model for analytics and for data science. Affirma offers a unique capability for utilities to not only establish a common definition of data—for reusability and delivery of data to support data-in-motion and data-at-rest—but to also bring together disparate technologies for data mapping, lineage, integration, profiling and more within a single solution supporting your chosen data architecture.

In Summary

Regardless of the direction you take, there are core competencies needed for modern data management. This includes understanding your data through a semantic model, linking data about data, managing change to reduce complexity leading to innovation and growth.