The economist Ronald Coase observed that if you torture data long enough it will eventually confess to anything. Governments and big businesses are prone to torturing data because the people working for them want to show their decisions were good, their predictions were sound, results are moving in a positive direction, and if anything went wrong it must be somebody else’s fault. This leads to a simple observation about the rollout of STIR/SHAKEN in the USA, which is supposed to reduce nuisance robocalls by authenticating the origin of calls, has cost approximately half a billion dollars, and was mandated by government. Should we anticipate impartiality when asking for the opinions of people who predicted STIR/SHAKEN would be a success? Or should we expect them to torture data for evidence that the results will eventually turn good if the world just keeps spending more money on the technology they recommend? The December data from TransNexus, a supplier of STIR/SHAKEN, tells us the following:
- The rollout of STIR/SHAKEN in the USA has currently plateaued with just one quarter of all US calls being authenticated using the technology
- 4 percent of all US calls are robocalls. 4 percent of unauthenticated US calls are robocalls.
- There are three grades of authenticated call: A, B and C, with A being the highest level of authentication.
- 1 percent of A-grade calls are robocalls.
- 13 percent of B-grade calls are robocalls.
- 13 percent of C-grade calls are robocalls.
The data scientists amongst you will have drawn some straightforward conclusions. Calls that receive the A-grade authentication are much less likely to be robocalls. However, calls that receive the B-grade and C-grade authentication are more than three times more likely to be robocalls than calls that have not been authenticated. Instead of authentication being a sign that you can trust a call, those calls which have the B and C grades of authentication are less trustworthy than calls which have received no authentication at all.
Simple maths also tells us something else, though TransNexus does not make this plain in the data they shared. If 4 percent of all calls are robocalls, and 4 percent of unauthenticated calls are robocalls, then 4 percent of authenticated calls must be robocalls. So whilst A-grade calls are much less likely to be robocalls, there are so many robocalls authenticated at the B and C grade that the STIR/SHAKEN authentication program has had negligible impact on the overall likelihood of receiving a robocall.
The way I would respond to this data is to step back, realize there may be strategic flaws in a half-billion-dollar program that has so far delivered no consumer benefit, and give serious thought to what changes need to be made to the strategy. TransNexus interprets the data differently. They say more telcos must be forced to authenticate more calls using STIR/SHAKEN, more telcos should be forced to spend more on STIR/SHAKEN, and more should be done to enforce STIR/SHAKEN rules. In other words, they argue the strategy is fine but it has not been pursued with enough vigor.
TransNexus may be able to provide more specific statistics that might bolster their case, though there are some minor problems with the quality of the data they share publicly. They choose to share graphs rather than tables of precise figures and two of their graphs present slightly different but clearly inconsistent answers for the number of unauthenticated calls which were robocalls: one graph shows this to be slightly below 4 percent, the other to be slightly above 4 percent. Nevertheless, we can assume their raw statistics are generally reliable because they show STIR/SHAKEN has failed to accomplish anything in the 6 months since the technology became mandatory for all IP networks owned by larger US telcos. The last thing we need is to torture data for evidence the current strategy is the correct strategy because a lot of people are already motivated to do that. What is really required is an impartial understanding of why the results have been so much poorer than expected.
The strategy for STIR/SHAKEN has been underpinned, and undermined, by naivety about how bad actors behave in real life. A sophisticated con artist does not stop conniving because of the implementation of simple checks they can easily work around. Effective con artists incorporate the pretense of being authentic into their scams. STIR/SHAKEN has been co-opted by bad actors. This is now proven by the data, but it should have been predicted too.
If the person knocking at your door is supposed to show ID then a career criminal will volunteer to show you his fake ID before you ask for it. An online scammer that wants your money sends you a message that looks like they work for the security department of your bank and says they need your urgent help to prevent your money being stolen. Law enforcement agencies also understand how to use trust to catch criminals: instead of taking down an encrypted international criminal comms network, the FBI let real criminals unwittingly enroll other criminals on to a network secretly run by the FBI so they could snoop on all the messages exchanged by the criminals. The authentication provided by STIR/SHAKEN was always going to be abused by bad actors to make bad calls look more trustworthy. This is corroborated by the data: it is relatively easy for bad actors to get B and C-grade authentication for calls, so they choose to obtain that level of authentication instead of leaving their calls unauthenticated.
The naivety of proponents of STIR/SHAKEN was illustrated when TransNexus remarked they had…
…found an amazing and unexpected correlation between robocalls and SHAKEN attestation level.
Their choice of words was revealing. TransNexus was commenting on September data that showed B and C-grade calls were twice as likely to be robocalls as unauthenticated calls. Experienced fraud professionals would not have found this correlation to be unexpected. Nor would fraud professionals be surprised that the problem has grown worse in the three months since, with B and C-grade calls now three times more likely to be robocalls than unauthenticated calls.
It is troubling that a business like TransNexus, which has been intimately involved in the development of STIR/SHAKEN, could be so naïve. Their surprise suggests that neither regulators nor standards bodies have taken advice from professionals with a sound understanding of how corruption occurs, even though enormous volumes of illegal robocalls are a symptom of the pervasive corruption of telecoms networks. It should have been obvious that bad actors would seek to have spam calls authenticated at the B and C grades because bad actors are generally in league with other bad actors, or seek to work with complacent telcos that do not care about preventing the abuse of telecoms networks.
The problem with the strategy pursued for STIR/SHAKEN is that it is easily abused unless the authentication runs from end-to-end, from the origin of a call to its destination, as occurs with calls that receive the A-grade authentication. Neither B nor C-grade authentication is end-to-end. In these cases, an authentication signature is applied to the call by a telco that does not really know the origin of the call. That telco is trusting someone else in the absence of robust authentication. Why would it be surprising to discover this trust can be abused, either because bad actors will lie to honest telcos, or because complacent telcos impose insufficient checks on their customers? It is only surprising because it would be even more damning to admit that the expensive technology for STIR/SHAKEN was rolled out despite this gaping weakness in the strategy for reducing robocalls.
The data suggests that STIR/SHAKEN could work well if everybody used it properly. However, if every telco was good and honest then we would not need STIR/SHAKEN. Authentication is required because some people should not be trusted. STIR/SHAKEN was rolled out in the knowledge it could only be applied to IP networks, not TDM networks. It was rolled out in the knowledge it would be mandatory in the USA and Canada, but not anywhere else. A lot of effort is now being put into forcing the same technology on to more networks, and into pressuring more telcos and more countries to follow the lead of the USA. This is like trying to remedy a major strategic weakness as if it were an unforeseen operational hiccup. The sense of growing desperation that surrounds STIR/SHAKEN vindicates any decision to delay spending on STIR/SHAKEN because of doubts about its efficacy.
Perhaps streets would be free of litter if everybody carried their rubbish home. Perhaps the police could be disbanded if criminals were persuaded to abandon crime. Perhaps wars would end if nations stopped buying weapons. But we still spend money on street cleaners and door locks and armies because we cannot afford to rely on dewy-eyed idealism. In the case of STIR/SHAKEN, the USA collectively spent half a billion dollars on technology in the belief that everybody else would voluntarily spend large amounts on the same technology because they all have the same goals. Little has been spent on rooting out corrupt business people in the USA who have the right connections to get their calls authenticated by US telcos. This sequence is back-to-front. STIR/SHAKEN is the expensive technology that might have been implemented later on, after demonstrating to the world what significant improvements had already been realized by taking simpler, cheaper steps to drive bad actors out of the telecoms ecosystem.
STIR/SHAKEN is a technology that is well suited to the task of turning a good situation into a perfect situation, but terribly unsuited to the task of turning a corrupt situation into a good situation. On the contrary, the resources poured into STIR/SHAKEN makes overall failure more likely, because some telcos will treat the enormous cost as justification to only do what they have been forced to do. If you need the whole world to collectively tackle a problem, it is best to start by proposing cheap and easy solutions, and then to build upon them after they have become the norm. I find it hard to understand why anyone thought the US STIR/SHAKEN strategy would succeed, and assume it was because they focused so much on the way the technology works that they ignored what we all know about human flaws.
