It really hasn't been long that utilities have been installing ever larger and more sophisticated (and expensive) software systems--CIS, ERP, Work Management, Outage Management, MDM, Call Centers, Telecom, etc. Technology has exploded in slightly more than 20 years and software systems for utilities have gotten larger and larger and more complex by the year. During that process, utilities frequently have turned to outside consulting firms to help with implementing these large systems. These outside firms help with selecting software and often with installing and implementing it, frequently supplementing efforts of the software vendors themselves.
Sometimes the cost of such "advice" equals or exceeds the cost of the software and some horror stories have emerged about these costs and the time it takes to get the project completed. A couple of these horror stories include:
- · A large public company that adopted a large new CIS and hired an advisory firm to help with the implementation. Nearly 10 years later, the system still wasn't working acceptably and management fired the advisory firm, the CIO who had championed the effort, and abandoned the failed system for a simpler, less expensive one.
· A large municipal seeking to install a new CIS experienced replacement costs of $75 to $100 million and has been through four vendors and a similar number of advisory firms to achieve a system that still doesn't work very well.
There is a lot of similar anecdotal information about these issues, much of it not very complimentary to advisory firms. Warren B. Causey, Ltd. conducted a survey recently to see how well the "trusted advisors" are doing, with some interesting findings. While the survey was not large, it did provide rather consistent findings. Some of these include (survey respondents were guaranteed anonymity):
- · Virtually all utilities of various sizes have been involved in at least one major software installation over the last 10 years.
· One utility executive wrote: "We found consulting firms to generally be worthless. Adept at repackaging information we already knew or are quite capable of discovering on our own as if it were a revelation, and providing NO value-add to any selection or implementation process. At best we hire consultants to perform job specific tasks. That way we can clearly see value from our investment."
· Another executive wrote: "We were a bit disappointed in the knowledge levels of the staff of the implementer. We have a very dedicated IT team here and it was not long before we were able to manage without them. There were no horrific errors but things could have been designed better at the outset. We are not large or complex. We chose an implementer with utility experience."
While advisory firms can, and do, roll out a lot of complimentary commentary about their services with past customers, the not-so-complimentary commentary also seems to be growing nationwide. The WBC Ltd. survey found:
- · Eighty-seven percent of respondents reported their utility did hire an advisory firm for either software selection and/or implementation.
· Nearly 50% of all respondents who did have an advisory firm still reported that the project went "over time and over budget".
· Approximately 50% indicated they were "not pleased with the work of our advisory firm".
It should be noted, again, that this survey was not large and respondents were "self-selected". Thus it cannot be considered statistically valid. However, it is another anecdotal bit of evidence that there are "rumblings" about the process out there and those rumblings seem to be growing.
There also are "rumblings" in the market that some advisory firms "misrepresent" themselves and what they can and cannot do. Sometimes such firms describe themselves as "system integrators" and at other times they are helping develop requests for proposals (RFPs) that may be tailored toward certain favored vendors. Others have on-going relationships with large software vendors, such as SAP and Oracle, and some utilities have felt themselves "steered" toward a vendor preferred by the "trusted advisor".
One of the major issues in this process is that there are no generally recognized, universal standards for advisory firms. What is a proper time-frame for the installation of a certain software system? How much should a utility pay for "outside assistance" in getting a new system acquired and implemented? Even the definitions of consulting firms or "trusted advisors" vary. For example, Wikipedia provides this definition: "A consulting firm is a firm of experts (consultants) providing professional advice to an organization or an individual for a fee. The primary purpose of a consulting firm is to provide access to industry-specific specialists, consultants and subject matter expertise."
In another article Wikipedia describes why information technology consultants are used:
- - To gain external, objective advice and recommendations
- To gain access to the consultants' specialized expertise
- Temporary help during a one-time project where the hiring of a permanent employee(s) is not required or necessary
- To outsource all or part of the IT services from a specific company
The same source, Wikipedia, however, also notes that the open-ended consultancy that has been in widespread use in the utility industry for many years now seems to be waning: "Generally, fixed-fee IT consulting is for a specific amount of work, within a defined timeframe. Many companies are now moving towards a fixed-priced IT consulting model. This trend is expected to continue as more companies now require delivery of IT consulting services within a defined time and price structure. Open ended consultancy models generally favor the consulting firm, as the consultancy firm is rewarded on a per-day basis, there is no incentive to complete assignments within a fixed time, the result often being risk of project and cost overrun."
Project and cost overrun has become increasingly widespread in the utility industry. For example, anecdotal information indicated a West Coat utility may have spent more than $200 million on a CIS implementation. Questions that arise from that type of situation include what are the metrics for success? What was the original budget?
To balance the discussion, we spoke with an industry insider that stated that all hope for finding trusted advisors and great system integrator consulting firms is not lost. There are trusted advisors and system integrator consulting firms out there that can back their track records and provide compelling data points, metrics, and facts to support the premise that their projects were indeed "successful". The source states you can find a trusted advisor and reliable consulting firm if your vendor evaluation process simply knows how to ask the right questions.
The vendor evaluation process must know how to navigate and sift through propaganda and twisted numbers you often get from many traditional firms that want you to select them, this source suggested. The questions you ask potential trusted advisors and consulting firms and the answers you seek must be fact-based .
"Perhaps the single most powerful question to ask when screening vendors is how is the vendor's track record when one does a comparison of actual project costs and schedules against what was originally agreed to and stated contractually in the original project statements of work and project charters for budget and schedule," this source said. "The answers you get to this question should be very revealing and is a great way to separate the best vendor performers from the poor performers." The source, who works in the advisory industry, goes on to say that too many vendors are winning business for the wrong reasons (e.g., "I need a bigger firm and throat to choke if something goes wrong. Everybody else is using them so they must be the best firm out there, etc.").
Obviously the major problem with utilities and large software replacements is that there appear to be no generally accepted metrics for success. "If we had those, that would take a lot of interpretation out of the discussions," said one advisory firm executive. "But none of the bigger firms want to have specific metrics."
What may be called for in the current situation, which is getting increased attention across the country, is better metrics on what constitutes reasonable costs for a large software system replacement and how do you hold both the "trusted advisors" and internal managers and executives accountable. Lack of accountability seems to be running rather rampant in these areas.