Taking a Robust Approach to Accenture’s Research

On the surface, Accenture’s 2011 report into risk managment for comms and tech firms tells us a generally positive story. Businesses are taking risk management more seriously, comms and technology firms are most likely to have a centralized risk management function, although they lag other industries when it comes to appointing Chief Risk Officers. The survey says a mere 24% have appointed a CRO. Interesting stuff… but something feels wrong about this report. Why do I say that?

  • The comms and tech industry report is the product of a much wider survey of 400 businesses across many industries. Accenture tells us that about 10% of the surveyed businesses fitted into the comms and tech category. But the report rather confusingly skips backwards and forwards between results that come just from the comms and tech subset, and results that came from all industries combined. Why not just consistently present how comms and tech firms compared to the overall averages, for every question asked? One possible explanation is that the surveyed firms did not answer all questions, leaving lots of holes in the data and making straightforward comparisons impossible.
  • Although the report tells us that comms and tech lagged in other industries because only 24% of them had appointed CROs, I find this stat suspicious because of how high it is. There are other measures of how many senior positions have been appointed to risk management. Take, for example, the number of magazine, journal and web interviews there are with comms and tech CROs. There are hardly any. Trust me, I look for them. Take a look at conferences and other events aimed at comms and tech CROs – very few again. I find it hard to believe that there are lots and lots of events I do not know about. In an industry where I am plagued by salesmen trying to sell me all sorts of irrelevant events, if there were more risk conferences, the salesmen would be rifling through LinkedIn and doing a simple string match on my job title. Finally, look through the agenda of risk conferences that straddle all industry types. You will not find many speakers or working groups or agenda items that relate to comms and tech. One straightforward explanation of why a survey overstates interest in a subject is that the survey respondents are self-selecting, and hence biased. In other words, if your business has a CRO, you may have someone who will respond to a risk survey. If there is no CRO, and not much interest in risk management, then probably nobody will respond to a risk survey either.
  • As somebody working in industry, I scan through surveys for the real meat. I want data that will help me in my job. Hearing about the importance of continuous improvement is very nice, but figures on how much people spend on risk management is far more interesting. The global survey, across all industries, says that more than half of the surveyed firms spent more than USD25M on risk management during the last two years. Wow. I could do with that slice of budget pie. How did the comms and tech firms compare to the pan-industry average? The comms and tech report does not say. How disappointing… and suspicious too. If Accenture asked the question of all respondents, then why suppress the results for the comms and tech firms? I can think of three possible reasons: most comms and tech firms gave no answer, most of them gave low answers, or most of them answered that they did not know how much was spent. Contrast that with Accenture’s eagerness to tell us that 21% of firms (across all industries) expected to increase their spend on risk management by more than 20%. Relative stats like that are of little help unless we first have some absolute baselines. After all, 20% of very little = not much more.

Forgive me for being sceptical – I would not be a very good risk manager if I did not question the information supplied to me. The three observations I make above are enough to discourage me from placing any reliance on Accenture’s report. However, there is one Accenture finding that put me on red alert when I read it. This is what Accenture wrote about the ‘Risk Masters’, the firms that supposedly manage risk better than the norm:

This mindset of seeing risk management as a proactive driver of business value is a distinctive feature of Risk Masters. Almost two-thirds of Risk Masters (64 percent) indicate that their risk management capabilities provide competitive advantage to “a great extent,” compared with only 42 percent of the peer set.

Sounds great. Until you remember that those firms that suffered a great risk collapse were not the type to do modestly and underperform their peers. The great risk collapses came from a slew of firms that generated great returns, were very profitable, and then went ‘pop’ the first time that the market asked pointed questions about the sustainability of their business model. Wall Street was full of clever people who could explain, with the benefit of wonderfully exotic equations, how well they managed risk to get a competitive advantage. And then their firms collapsed and poor old taxpayers had to do some much harder sums about bailing out the Wall Street whizzes. But if you do not like my take on risk management, at least listen to what Nassim Teleb has to say on the importance of robustness:

Most people think that they can predict the black swan, that with quantitative sophistication they can get answers. They don’t get the idea that because we can’t predict black swans, then we need to restructure institutions and rethink strategies to be more robust in the face of uncertainty.

…. I have always been very skeptical of any form of optimization. In the black swan world, optimization isn’t possible. The best you can achieve is a reduction in fragility and greater robustness. You may have heuristics, but not an optimization rule. I hope the message will finally get across because I haven’t succeeded yet. People talk about black swans but they don’t talk about robustness, which is the real lesson of the black swans.

You can enjoy the full Nassim Teleb interview here. I suggest you do; it is more enlightening than a report by Accenture.

Eric Priezkalns
Eric Priezkalns
Eric is the Editor of Commsrisk. Look here for more about the history of Commsrisk and the role played by Eric.

Eric is also the Chief Executive of the Risk & Assurance Group (RAG), a global association of professionals working in risk management and business assurance for communications providers.

Previously Eric was Director of Risk Management for Qatar Telecom and he has worked with Cable & Wireless, T‑Mobile, Sky, Worldcom and other telcos. He was lead author of Revenue Assurance: Expert Opinions for Communications Providers, published by CRC Press. He is a qualified chartered accountant, with degrees in information systems, and in mathematics and philosophy.