Demoting People to Promote Software: The TM Forum RA Maturity Model v2


I really have better things to do than to write a blog about everything that is wrong with the TM Forum’s new Revenue Assurance Maturity Model. I really do. Nothing I could write, or do, will influence the ‘best practice guidance’ issued by the TMF. The people running their RA team have a fixed agenda. That agenda is far more transparent than the supposedly collaborative process by which they issue their standards. In case you missed the press release, the agenda is most easily illustrated by the following:

  • Israeli software firm cVidya leads the TMF’s Enterprise Risk Management group;
  • Israeli software firm cVidya co-leads the TMF’s Fraud Management group; and
  • Israeli software firm cVidya leads the TMF’s Revenue Assurance group.

Does this suggest that TMF guidance draws upon a wide-ranging and representative sample of industry opinion about how to manage risk, fraud and assurance? You can decide for yourself. I state facts. It is also a fact that, over the last five years, the TM Forum has repeatedly issued guidelines which understate the importance of the employees of the telco, in order to push sales of technology. When I assert this, I get a lot of criticism. Naturally the criticism comes from people (however impressive machines are, they still cannot advocate for themselves). And so, the theory goes, I must be biased (after all, I have a freebie blog!) whilst the people I argue against must be honest, decent, unbiased folk (who only need to generate millions of dollars of revenue by selling software). So let me make a few, very succinct points about why the new RA Maturity Model proves that the TM Forum evades transparency wherever possible, and always seeks to undermine the value of people in order to promote sales of software.

1. Nobody told you that the new RA Maturity Model was coming out.

If you are employed by a company that is a member of the TM Forum, you have the right to comment on the draft documents they issue, prior to formal ‘approval’. Approval itself is a mystery – it is not clear who actually decides what is approved or how they reached that decision. But at least you can comment, saying if you like or dislike what they produce. Or you could, if you knew that a new document was coming out.

My point here is simple. In a few weeks, expect a press release from cVidya which plainly states that the TMF has approved the new RA Maturity Model, and that cVidya has made it available as a software add-on to their existing product suite. What the press release will definitely not say is that the TMF has approved “GB941 Revenue Assurance Solution Suite 4.5” and that cVidya have updated their products accordingly. The press release will not use that language because nobody knows that “GB941 Revenue Assurance Solution Suite 4.5” is code for the new RA Maturity model. And that is why the TM Forum sends out notifications alerting its members to the release of GB941 Revenue Assurance Solution Suite 4.5, without bothering to mention what is in it.

To illustrate my point, last week I contacted one of the four telco employees listed as authors of the new RA Maturity Model, to ask if he knew when the document was being approved. He had no idea that the deadline for comments was only days away. He admitted he had not even read the draft document. If the supposed authors of the document do not read and approve it, then who does?

2. The old model stated that people were an independent and crucial dimension in the determination of maturity. The new model deletes this dimension.

The original TMF RA Maturity Model had 5 dimensions: Organization, People, Influence, Tools and Process. In order to score as a fully mature organization, the organization had to be fully mature in every dimension, without exception. This was a straight copy from the Software Engineering Institute’s original Capability Maturity Model, and drew on their empirical evidence. The new TMF RA Maturity Model has 4 dimensions: Organization, Process, Measurement and Technology. Spot the difference? There are still a few questions about people, now included under the Organization dimension. But no explanation is given to justify this radical change, which demotes the importance of people, whilst putting even more emphasis on technology. In fact, two of the four dimensions are now dominated by technology, because the new ‘measurement’ dimension only makes sense in the context of technology to provide measurement.

But to fully understand the way in which people have been demoted in the new model, you need to appreciate the following…

3. The old model said that the assurance chain is only as strong as its weakest link. The new model just takes an average.

The old model was built on a straightforward but important principle. When many parts have to work together, the weakest performing part sets the limit on the overall performance. Good organization but weak process will deliver weak performance, good tools but weak people will deliver weak performance, and so on. The old RA Maturity Model emphasized the importance of improving maturity across all the dimensions in a coordinated way, because spending a lot of money or effort to improve one dimension would be wasteful and ineffective, if the other dimensions were left far behind. The new model just takes the aggregate score across all the questions, and translates this into the overall level of maturity.

This means that in the new model, even if no effort is put into recruiting and developing staff, a high maturity score is still possible by simply putting more money into technology and the things that senior managers tend to do.

Once again, the new RA Maturity Model deviates from a key principle in the Capability Maturity Model, which was why it was adopted in the original RA Maturity Model. No justification is given for this fundamental change of approach. The document begins by suggesting reasons why a new version of the maturity model was needed, such as the increasing popularity of digital services, and a different ‘ideal’ for revenue assurance (whatever that means). However, these reasons cannot possibly explain the much more fundamental changes that have been made in practice, without showing any reasoning or data to support those changes.

4. The new model makes it too easy to attain the highest level of maturity.

In the old model, to attain the highest level of maturity, the organization had to achieve the highest level within each of the five dimensions. It was a simple idea, which expressed how difficult it should be to achieve the ‘ideal’. In effect, an optimal organization could only exist if an optimal answer was given to every single question. Is this not obvious common sense? How can the whole organization be optimal at anything, if some crucial elements are sub-optimal?

The new model not only brings in averages, but sets low expectations. To be scored amongst the highest level of maturity, the telco needs only to achieve a score which is 80% of the maximum score possible. That means that a telco can completely fail to do some important tasks, like adequately training staff, or reviewing new products, and still be assessed as ‘optimal’ at revenue assurance.

5. There is obvious bias to the individual questions in the new model.

Consider the following questions, all taken from the new RA Maturity Model:

Is appropriate budget made available for the supply of RA technology?

Is appropriate budget made available for the deployment of RA technology?

Is appropriate budget made available for the operation of RA technology?

Is appropriate budget made available for the on-going support and maintenance of RA technology?

In contrast, there is only one question that might be interpreted as relating to another crucial aspect of a revenue assurance budget.

Is the resource profile of the RA team reviewed periodically to ensure it is staffed appropriately?

There are many other examples of how the questionnaire is slanted, but this example neatly illustrates the main problem. It was written by people obsessed by using software, and indifferent to the alternatives.

6. And all the rest…

I could go on for much longer, and in much more detail, but people complain that I rant for too long. So I will not go into a lot more detail. My main point is made: the new RA Maturity Model deliberately places less importance on people in order to focus even more attention on software and the budget to buy it. But there are very many other flaws with this work.

The new model repeatedly confuses the revenue assurance maturity of the whole organization (the very clear purpose of the original maturity model) with the maturity of a nominal RA Department. It even talks about the ‘ideal’ RA function, as if all that matters is the function, and not how the rest of the business behaves. The goal of revenue assurance is holistic, making demands all across the telco, and the original model sought to empower RA managers and staff by making this clear. Also, the business should have the right to split up work between different departments in any way that best suits them. What matters is the overall result to the organization, not the ego of some guy with the job title of ‘Head of RA’.

The new revision was supposedly needed to keep up with technology, but its understanding of technology is backward-looking. Time and again it refers to ‘RA technology’ in ways that indicate this technology must be separate to other technology. RA is a goal, not a technology. There is no reason why the same technology might not satisfy multiple goals, including the goals of RA. As such, the new model takes no account of the impact of Big Data, and other trends towards mass aggregate use of data across the enterprise. In fact, it still has a prejudice against using data from ‘secondary’ sources, even whilst Big Data is making a nonsense of the idea that data can only be trusted if it comes from ‘primary’ sources.

The new model claims to be a simplification of the old model, but it is not. The old model had five answers to every question, a simple way to express how every answer to a question maps to one of five levels of maturity. By destroying this mapping, the new model is opaque, and does not represent maturity as a stepwise improvement that must go across all dimensions.

As sadly typical of the TMF RA team leaders, the new model lacks transparency. This fits with its increasing complication, which is hidden from view and then mis-represented as simplicity. The new equations to calculate maturity are not visible to the user. The old assessment could be performed with pencil and paper, whilst the new one must be done in a Microsoft Excel spreadsheet, because of the equations hidden within. All the questions and answers in the original model were written out in full, so everybody could see them and implement them as they wished. Because the old model was transparent, telcos were free to tailor the model if they wanted to. There is some irony in this fact, because Gadi Solotorevsky often gave presentations about the ‘TM Forum RA Maturity Model’ in co-operation with Telefonica, even though Telefonica had very clearly changed the model to reflect their point of view. As the new document explicitly states, it would not be possible for a telco to change the new model, even if they wanted to, because of the way the equations have been implemented.

It should be noted that the new document claims to have improved on the old model because it has ditched the weighting scheme which was used in the original model. However, it is important to reflect on why the original model had such an inelegant weighting scheme. The reason was that the weightings were the result of many people’s contribution to the original model. If we surveyed the opinions of ten people about how important question A is, relative to question B, we might expect ten different answers. To get to an answer, the original model just totalled the weightings proposed by all the contributors, and used the average. It was not a perfect system, but it was clear and fair. The new model says it has improved upon this. However, I cannot work out how it would be possible to do this, unless just one or two people decided to impose their will on the work. As such, the new model must be much less of a collaborative team effort than the old model was.

Which leads me to my final point. When I was at WeDo’s user group event, I saw an excellent presentation by Daniele Gulinatti, VP of Fraud Management & Revenue Assurance at Telecom Italia. His presentation struck a chord with me, because it was all about real people, and getting the best from his team. They delivered great results by using imagination and good processes, irrespective of the limits on their technology budget. And some of his team were in the audience, and I can vouch that I could feel their enthusiasm from the other side of the hall. So I find it hard to reconcile Daniele’s effervescent humanity with the fact he is listed as one of the authors of this stilted, cold TMF document. On the one hand I see a manager who clearly understands that superior results can only come from a motivated team. On the other hand, I see a TMF document that treats people as inferior to, and more disposable than machines. What can I do, but shrug my shoulders, and wonder how this is possible?

Perhaps this divergence is natural in human affairs. Many managers want official-sounding documents to show to their bosses, arguing they should have a higher budget. I was always conscious of this potential pitfall with the original RA Maturity Model. Even though it explicitly presented a strategic overview, there was always the prospect that it might be manipulated to give quick budget wins. That is why so many vendors and consultants copied the idea (but not the content) in the hopes of boosting their sales. Their versions of the RA Maturity Model soon disappeared. The original TMF RA Maturity Model has thrived, because it really was long-term, strategic, and built on solid foundations. And that means curbing bias (like the need to maximize this year’s software budget) in order to present a more balanced model that genuinely considers what is needed in the long-run (like a motivated team, which receives proper rewards for its successes).

But like barbarians, the ‘leaders’ of the TMF team are determined to wreck anything that does not immediately gratify them. Maybe I am in the minority. Perhaps the majority agrees with their approach. If so, I would accept the will of the majority. But we will never know, because whilst the original RA Maturity Model was written in 2006 with the involvement of just three telcos, the new RA maturity model has been written in 2014 with the involvement of just three telcos. Getting three telcos to contribute to an RA document in 2006 was a minor miracle. In 2014, it is a sign of apathy, or worse. After all, people have had 8 years to get used to the idea of an RA Maturity Model. Only a few of us understood the idea in the beginning. The TMF claims that half of the respondents to its RA surveys use the model. But despite that, they could only get MTN, Telecom Italia, and Telefonica Chile to contribute their conception of the new ‘ideal’ for revenue assurance. With the greatest respect to the people working in those telcos, why do they know the new ‘ideal’ for revenue assurance, more than all the other people who now work in telco revenue assurance? And based on the person I spoke to, what confidence is there that anybody currently working for a telco has actually read the whole document?

The team who wrote the original RA Maturity Model produced the questionnaire using a voting process. Questions were proposed, answers proposed, people voted on which ones made the cut, and which were rejected. And then, there was a vote on the weighting of the questions. If the TMF really wanted the opinion of telcos, why did it not run a survey on the content of the new RA Maturity Model? Such a thing would have been impossible in 2006. In 2014, the same task is incredibly easy. I believe it is because the whole point of their ‘collaborative’ process is to exclude the involvement of telcos, whilst making it appear that they invite their input. Everything is done to make it hard to participate or respond, from requiring people to fly around the world to attend meetings in person, to hiding equations in spreadsheets, to sending out notifications about “GB941 Revenue Assurance Solution Suite 4.5”. The TMF does lots of surveys about lots of things. Why not decide the new ‘ideal’ for revenue assurance by doing a survey? The only possible reason is that the answers might not support the leaders’ agenda.

The new RA Maturity Model is a broken product. But that is no surprise: it is the output of a broken process. The TMF has no interest in fixing one. It is beyond my abilities to fix the other. The only good thing about the new model is that it will die in a year or two, victim of its own failings. It says too little about people – and people often last longer than technology. It is too easy to reach the top level of maturity, meaning there will soon be calls for an upgrade. It does not promote the kind of balanced approach needed for long-run improvement. The equations are too complicated to understand, and have been hidden from view, meaning they cannot be fixed if they do not work. These fundamental flaws have doomed it to an implausibly short life for a supposedly ‘strategic’ model. But then, we should not be surprised. The real authors of this revised model are worried about this quarter’s sales figures, not about the next evolution of a mature strategy for business improvement.

Eric Priezkalns
Eric Priezkalns
Eric is the Editor of Commsrisk. Look here for more about the history of Commsrisk and the role played by Eric.

Eric is also the Director of the Risk & Assurance Group (RAG), a global association of professionals working in risk management and business assurance for communications providers.

Previously Eric was Director of Risk Management for Qatar Telecom and he has worked with Cable & Wireless, T‑Mobile, Sky, Worldcom and other telcos. He was lead author of Revenue Assurance: Expert Opinions for Communications Providers, published by CRC Press. He is a qualified chartered accountant, with degrees in information systems, and in mathematics and philosophy.