Seeing the Big Picture for Benefits Analysis

Most of you will have heard of the US Government. As an organization, they are responsible for decisions like how much to spend on stopping other countries from invading the USA, deciding how much to spend on preventing deaths from industrial pollution, and deciding how much to spend on public infrastructure that will be used by private sector businesses and private citizens. As such, the US Government is intimately concerned with a topic that also interests the typical reader of this website: how to assess risk, and hence how to compare the costs of various current and potential activities to the uncertain benefits that will flow from them. Put like this, the US Government is the biggest risk management organization in the world. When considering the theory of how to do risk management, I like to keep the US government in mind as a practical example of how risk management works in practice. I do this for four reasons:

  • The US Government is big. They have a lot of money, spend a lot of money, and can do work to a degree of sophistication that is unrealistic for other, smaller, organizations.
  • The US Government is transparent, at least when compared to private sector businesses and many other governments.
  • The US Government is the culmination of democratic processes that permit its primary stakeholders (American citizens) to provide very extensive and open feedback about risk priorities.
  • It makes lots of mistakes.

Do not get me wrong. Every organization makes lots of mistakes, so I have no intention of criticizing how well the US Government is managing risk in practice. To do that fairly, I would need as much data as the US Government has, and whilst I admit to being opinionated, and to having quite a good memory, I will not claim to have as much data as the entire US Government. Also, I could make only a negligible contribution to the total sum of criticism that the US Government receives all the time. I review their mistakes so I can learn from them, based on the principle that it is cheaper to learn from somebody else’s mistakes, rather than from your own. And being so extensive in scope, the US Government deals with a much more extensive ‘universe’ of risk than any other organization, meaning that when their risk management suffers from a general and systematic error, it occurs at a level of generality that most closely approximates to the level of generality found in international risk management standards like ISO31000 and COSO. After all, not many organizations have divisions as diverse as the Federal Reserve, which deals with mortgage underwriting risk (oops), like NASA, which aims to send people into space and bring them safely back again (err…) and the Environmental Protection Agency, which many Americans accuse of doing too much.

Seen from this perspective, the US democratic process is the greatest human process for deliberating and deciding risk appetite. The resulting appetite for risk has global influence on the climate, world trade and security. So, given that I referred to mistakes, what are they doing wrong? Well, they suffer the same problem as everyone else. They suffer from bias. Or at least, they suffer from bias at recurring intervals; I will avoid commenting on whether the decisions reached today are more or less biased than decisions reached in the past or which may be reached in future. It is sufficient to note that bias is evident in how the US Government assesses risk, just because risks that seemed severe when one kind of politician is in charge are considered less severe when another kind of politician is in charge. Perhaps that is not surprising, but bear in mind that, at a fundamental level, the US Government follows the same ‘objective’ approach to compiling a cost-benefit analysis of risk as the paradigms presented in ISO31000 and COSO. The conclusion to be drawn is that following an ‘objective’ process is no guarantee of an objective decision. Even processes designed to promote objective thinking about risk will be prey to (conscious or unconscious) manipulation because of subjective judgements about what data is considered relevant and how to calculate its significance.

The point is illustrated by a recent article in The Economist. It gives examples of how, since Barack Obama became President, government agencies perform the same kind of calculation about risks, but have tended to include more benefits, and fewer costs, when evaluating when to take action. Similar observations were made in a 2011 article for the New York Times. The latter focuses on the rise in the ‘value of a statistical life’, which is effectively the dollar value that US Government agencies use in their equations when deciding how much to spend on preserving a human life. Evidently your cost-benefit conclusions will be different depending on whether you value a life at $2M or $10M. So whilst the process can be considered objective – you gather data on costs, you gather data on benefits, then do a subtraction and see whether you are left with a plus or a minus – this superficial objectivity can be easily undermined by subjectivity in what data is used and how it is used. A cynic would observe that Republicans always reach objective conclusions on the value of a human life that lead to fewer regulations, whilst Democrats always reach objective conclusions on the value of a human life that lead to more regulations. I will not pick a side in a political debate, but merely want to observe that whenever people work backwards from the amount of regulation desired to the ‘objective’ data they select for the cost-benefit calculation, then they have already biased that calculation and hence undermined the purpose of performing it.

If objectivity is not guaranteed by mechanically following a process, then we need to go beyond processes in order to deliver risk management that is fit for purpose. Honesty, transparency, and consensus all play a part in performing a reliable cost-benefit analysis. That is why I am so dismissive of cVidya’s ProactiV tool and the TM Forum model it was based upon. Put simply, it always concludes that the benefits of more software will outweigh the costs. Always. And they have so obscured this essential truth that Dr. Solotorevsky, of both cVidya and the TM Forum, actually recommends the tool will help telcos to reduce risk. That is true if you conveniently forget that businesses have multiple objectives and face multiple risks. No business has a primary and overriding objective to spend money buying software no matter how little benefit it delivers. People can kid themselves that using this tool helps them to objectively analyse risk, but I do not want to work for a company that will spend a million dollars for certain just to eliminate one cent of possible risk. There is no job security and little job satisfaction in working for a company that is wasteful, even if I work in the department that selfishly benefits from the waste. cVidya’s ProactiV tool is the metaphorical equivalent of a government department that says a human life is worth a million billion gazillion dollars and then demands tax rises to pay for all the ‘benefits’ it will deliver.

Just following a process or adhering to a standard is never sufficient to deliver objective risk management. To manage risks correctly, we must have a collective sense of priorities that allow us to address competing goals. The government weighs up the risk of inadequate infrastructure investment with the risk of inadequate defence with the risk of stifling the private sector through excessive tax… and so on. The US Government is chosen by a democratic process that helps it arrive at the right balance; if a government’s risk appetite is badly out of line with the majority of voters, then the voters can and will replace it. An equivalent process occurs in business, though we must remember that an employee is not a metaphorical equivalent of a voter. The investor is our ‘voter’, and they vote with their money. The consensus we arrive inside business is like the consensus reached inside government. We should not confuse votes by political representatives with votes for political representatives, and nor should we confuse the interests of employees with the interests of the business itself. Employees have a stake, and that needs to be aligned to other stakeholders. Good alignment leads to mutual benefits. For example, employee safety is generally going to be of benefit to longer-term investors. To be brutal, it costs money to replace and train people. Only a business that over-prioritizes quick returns to short-term investors will cut costs and corners at the expense of employee safety. Hence, it is possible to align the goals of different stakeholders to achieve mutual benefit. And the vehicle for this is the risk appetite statement, a public and transparent articulation of how the business weighs up and prioritizes the potential for variance between its various objectives and its actual performance.

To clarify the importance of understanding and setting priorities for variance, let me use a simple hypothetical example. A business may have two objectives: it will generate $XM in profits; and no employees will die as a result of accidents. What, then, if we find the business is under pressure and is struggling to meet its profit objective? Cutting expenditure on safety may be a way to attain their goal, but this may come at a human cost. So a moral business would more readily accept a variance to its profit objective than to its safety objective.

The risk appetite statement acts like a manifesto, giving guidance to investors and to employees, so investors back the businesses that match their goals, and employees make decisions aligned to those goals. Like good government policy, the formulation of this statement will need to be supported by both an internal and an external discussion that both contribute to the formulation of an agreed mission with clear priorities.

And by now… most people would tell me this blog is too long. In a way it is, just like democratic processes can be exhausting in how long they take. Many voters will only take an interest in the weeks immediately before an election. Some government employees take a sporadic interest in serving the public. Sporadic interest is also a problem insufficiently addressed by the textbook methods advocated in ISO31000 and COSO. A system that looks perfect on paper is of limited use if only occasionally followed in practice. Consider this: a company says it follows the standards and performs a cost-benefit analysis when deciding how to respond to risk. Fine. Then it must know the value it places on human life. How else could it calculate what to spend on safety? Of course, what really happens in some businesses is that it puts in place some mechanics for calculating risk, but never finishes the job. We end up with a peculiar, partial and superficial compliance to a risk standard, but not a genuine attempt to objectively analyse the costs and benefits of risk treatments across the full spread of properly prioritized risks. We end up with something that looks like an objective calculation by a government department, but which actually reverse-engineered its subjective selection of data from the conclusion it wanted to reach.

For all its failures, the US Government gets a lot right. At least we know what value is placed on a human life by US Government agencies. At least there is sufficient transparency that journalists can report on, and voters can learn about, how risk perceptions have changed and how this might be influenced by subjective factors. These decisions can be analysed for what they are, and are not lost because they are hidden from view or twisted into unrecognizable forms. And, thanks to transparency, I have a lot of data which helps me to improve the objectivity of decisions that people otherwise try to evade because they find it uncomfortable to do so. Remember, not making a decision is still making a decision: the decision not to act now. Not making a reasoned decision about safety risks does not eradicate those risks and nor does it mean there is no sense of risk priorities being manifest in practice. The same can be said of any type of risk. Mechanics, compliance, standards and process are not sufficient for good risk management, as demonstrated by arguments about the risk priorities of the US Government and its failures in practice. Honesty, transparency and a genuine desire to reach consensus are also important, especially if failings are to be identified and improvements made. Whilst the execution may be imperfect, the US Government illuminates the challenges when turning good risk management theory into good practice, and illustrates the moral characteristics that are essential for success.

Eric Priezkalns
Eric Priezkalns
Eric is the Editor of Commsrisk. Look here for more about the history of Commsrisk and the role played by Eric.

Eric is also the Chief Executive of the Risk & Assurance Group (RAG), a global association of professionals working in risk management and business assurance for communications providers.

Previously Eric was Director of Risk Management for Qatar Telecom and he has worked with Cable & Wireless, T‑Mobile, Sky, Worldcom and other telcos. He was lead author of Revenue Assurance: Expert Opinions for Communications Providers, published by CRC Press. He is a qualified chartered accountant, with degrees in information systems, and in mathematics and philosophy.

3 Comments on "Seeing the Big Picture for Benefits Analysis"

  1. Eric,

    Your article illuminated a serious telecom and political problem. I found the New York Times story particularly alarming.

    I’ve long known that the politicians of my United States are skilled at lying with statistics. But even if you can’t find a clever way to present your statistics, isn’t it nice that politicians can employ the fallback tactic of magnifying the costs or benefits?

    When I was in college back in the 1970s, we heard a lot of talk about the “military/industrial complex”. And the effort to correct that problem led, in part, to the rise of the regulatory czar.

    But forty years later, the military/industrial complex never really went away. In fact, we have added an energy/environmental/government complex and the banking/government complex. The sad result is that referees today are no longer just regulating. The effect of their policies is to penalize some teams in support of their opponents.

    General Motors can now compete well with Chrysler, Ford and Toyota because the regulator contributed several hundred billion dollars to GM. Alas, even the best intentions of our wise politicians sometimes goes for naught. The heavily subsidized hybrid car, GM’s Chevy Volt, is no longer being produced because not enough people want to pay $40,000 for it.

    Telecoms have never been as politically connected as the banks, automakers, and green energy producers. Witness the recent slap in the face that AT&T got when it tried to merge with T-Mobile USA. Now a smart company like AT&T doesn’t make a huge bid to acquire another mobile operator unless it knows its back is covered by the regulators. So apparently, AT&T got bad intelligence or was double crossed.

    Still, I need to temper my criticism of regulators somewhat, because sadly, my wife and I were once ruthless regulators.

    You see, we have 5 cats in our house, and in their early years of growing up, we regulated our cats severely by not allowing them to roam outside the house. And why did we do this? Because we couldn’t bear the thought of seeing one of them killed by a car running over them on the street.

    But notice something. We had put value on our cats’ <u><b>existence</b></u>, not on their everyday <u><b>living</b></u>. The cats themselves were begging and pleading to be set free to roam in the outdoors, to catch mice, climb trees, and live the natural life of a cat in the wilderness.

    Fortunately, we live in a rural area with plenty of woods behind the house, so we eventually did set our cats free, installing a cat door on the house so they could go in and out anytime. Now it all went fine till the day I awoke to find a raccoon in our kitchen – luckily, an Auto-ID cat collar solved even that issue!

    My point is by allowing our cats to take some risks, it enriched their lives immeasurably. Yes, they’ve had to compete for territory with other cats in the neighborhood, and a couple times they came home injured from cat fights, but on the whole, they learned to survive and prosper once my wife and I gave up the regulatory game and opened the door for them.

  2. Eric,
    You remind me of a naive child with fingers crossed praying "everything will work out".
    Put simply, you concludes that benefits from software will not always outweigh the costs.
    While you are right – you make a grave error always trusting the good nature of progress.
    Why don’t you disable both antivirus & firewall software on computers while you’re at it? Surely, you can’t always promise that they will justify the investment.
    Let one or two hackers get inside and have their fun.

  3. @ Elad, you make some interesting comments. You’re right. I am naive, in a way. I still expect people in business, and government, to retain an ethical baseline. I expect them to tell the truth. No, that’s not right. I demand they tell the truth. Your problem is that I am far less naive than you would like. I demand the truth, but I don’t get fooled by lies from people like you. That is why I call out, over and over and over, when people like you manipulate and cheat, just so you can put a few more shekels in your miserable pocket. And the best part is that, because you’re naive about how the internet works, you just called me names on my website, after I pointed out the truth about your company’s tactics. Thanks! Seriously, I love it when people do that. You know why? If you were a grown-up who understood progress, then you’d know that this kind of exchange will bring more traffic to my website, meaning more people will find out the truth about your company. So let me reiterate the facts. (1) cVidya’s ProactiV tool is skewed to always reach the same moronic cost-benefit conclusion: no matter how much you spend on software, you supposedly would be better off if you spent even more. (2) I want people like you to leave comments like yours.

    Now I know what you’re thinking: “did he just dare me to leave another comment?” Well, to tell you the truth, in all this excitement I kind of lost track of all the ammunition I’ve got against your company. But being as this is the most powerful RA website in the world, and could blow a hole clean through your reputation, you’ve got to ask yourself one question: “do I feel lucky?” Well, do ya, punk?

Comments are closed.