Follow the science!
Popular 21st century proverb
If you have been paying attention to debates about the extent to which scammers use comms networks then you already know where this article is headed. Many readers will sincerely believe in the importance of using scientific analysis to understand and shape our world. They acknowledge the importance of science for measuring and tackling climate change, or for preventing the spread of disease. Meanwhile, we have almost no objective research into the use of telco services for crime. Only a few academics have explored the subject. Some of those academics have told me of their frustration with telcos who are unwilling to give advice or share data. To make matters worse, their findings are typically ignored by comms providers and industry associations. This is wrong. Electronic communications is now a bedrock of society. To protect the public, professionals working in the private sector need to be willing to listen to other voices than their own. This is what Sathvik Prasad, Aleksandr Nahapetyan and Bradley Reaves of North Carolina State University recently wrote on the topic of monitoring robocalls.
Industry sources publish regular reports on call trends, but using proprietary and opaque methodologies… Academic research uses transparent methods and evaluations, but comprise “one-shot” efforts describing a single moment in time. Collectively, the technical community and the public at large need a reassessment of the state of the network.
I agree. We do need a reassessment of the state of the network, although decades of following this topic prompt me to word the demand more strongly. I would say that society is flying blindly into a storm of networked crime with no map, no compass, and no sense of direction. Do you disagree? Do you think opinion polls of less than a hundred fraud managers can provide a robust way to measure the global scale of sophisticated crimes that even hardened professionals sometimes find difficult to understand? Do you trust companies with a commercial interest in promoting their products and their services to give unbiased guidance about the scale of the problems they claim to solve? Or should we be using the fundamentals of science — impartial analysis, hard data, robust mathematics, representative samples, peer review — to determine the scale of crime and whether our response has been successful?
What’s measured improves
Peter Drucker
The world’s leading management consultants have also emphasized the need to gather reliable data in order to inform decisions. They reiterate the same themes as scientists because empirical observation is vital to expanding human knowledge. However, the peculiar realm of communications crimes seems to have concluded that we only need to record the number of complaints submitted by customers, ask the opinions of practitioners about the scale of crime generally, and recycle the marketing produced by technology vendors. We do this despite simultaneously acknowledging that the public is poor at differentiating between scam and legitimate communications. We express confidence in the surveys produce by industry insiders even though practitioners say they lack the tools to measure the crimes afflicting the businesses that employ them, never mind the businesses which do not. We pretend there is no reason to doubt claims presented by marketeers, although every competent auditor and risk manager is trained to be skeptical about information from sources that are likely to be biased.
At the Mobile Ecosystem Forum’s anti-fraud conference last year, the two keynote speakers each presented a slew of statistics that were completely irreconcilable. This kind of discrepancy does not provide a sound basis for setting priorities or determining policies, even though the talks were presented to people who are supposed to be decision-makers. One speaker was discussing the cost of crime to the general public, whilst the other was primarily talking about the cost of crime to communications providers. The distinction was not made clear to the audience. I doubt many were conscious of the discrepancy. There is a small area of overlap between the two groups of statistics that were discussed, but neither group produces figures that are granular enough to determine if they are consistent with each other. This did not discourage people in the room from acting as if they had learned something. Many held their phones aloft in order to photograph the slides showing these statistics. If you treat misinformation like information then you have not really learned anything. We know from our experience of the web and social media that only a bumpkin still trusts everything they are told, yet professionals in the comms sector are content to repeat hearsay about the scale of crime because they have no better sources of information.
I can continue to enumerate examples of bad information that received more respect than they deserved. There was the occasion when my colleague Rob Chapman used a freedom of information request to demonstrate that the UK police had fed SIM swap statistics to the national press that were so utterly unreliable that the police were incapable of reproducing them. I have dissected the methods used by the widely-quoted survey of fraud run by the Communications Fraud Control Association (CFCA) and shown how they are flawed. We even trolled the CFCA by running rival surveys and making all the answer data available to anyone who wanted it. Transparently sharing raw data is a basic academic discipline that the CFCA refuses to adopt. However, the bitter truth is that nobody examined the raw data that we published. Even when the anti-fraud community has an opportunity to query the science, there is not enough curiosity to do so. This conclusion should be jarring given the much more scientific methods used by allied professionals who work in the domain of cybersecurity.
There are many ways to manipulate statistics that have gone unchallenged in the realm of communications scams. One simple trick is to inaccurately describe a trend in words even though the raw data reveals the opposite trend. Another involves an imprecise definition of the phenomenon that is being measured, as has often occurred when commentators have treated scam calls as synonymous with robocalls; some robocalls are legal and some scam calls are live. Frauds sometimes get lumped together because of a superficial similarity in the methods used, even if the victim and the harm differs greatly; for example, a simbox can be used to avoid termination fees or it can be used as a relay station for calls from a live scammer engaged in impersonation fraud. The worst trick almost always goes unchallenged. Many sources only choose to report the statistics that suit them, whilst suppressing any statistics that do not. Tomorrow’s Commsrisk article will highlight how American professionals are claiming there has been an increase in enforcement action against illegal robocallers although the Federal Communications Commission has dodged their obligation to report to the US government on the number of fines they have issued to robocallers.
However, nothing is accomplished by solely being negative. Somebody has to do something positive if we want more robust monitoring of scam communications. The simple solution is to emulate and expand upon the work being done by academic researchers like the team from North Carolina State University that was quoted above. More phone honeypots could be implemented for a tiny fraction of the amounts spent on other kinds of anti-fraud controls. India already mandates comms providers to implement honeypots, and is considering whether to expand their use. I encourage all national regulators to ask themselves why they are not contemplating using honeypots to monitor for scams, measure their scale and automatically extract data that could be used to rapidly shut down scams, such as the callback numbers that are often included in scam messages.
The limitation of a single-nation honeypot is that it cannot provide the kind of comparative analysis which would best guide policymaking. If two countries pursue two distinct anti-scam policies, then we could draw especially useful conclusions about the effectiveness of those policies by comparing trends using consistent measures over time. The honeypot technology to do this already exists; we have just not been using it to perform this task. That is why I have partnered with BluGem, experts in the manufacture and use of network test devices, on the development of an international honeypot that will produce comparative measures of SMS scam activity in various countries. BluGem has committed resources to sustain the honeypot program for one year. I am hoping that national regulators and academic researchers will pay attention to the results and then help us to obtain funding for an expansion of the honeypot program to more countries and more communication channels.
Using phone honeypots and other empirical measures to monitor scam activity is the third of the three missions that prompted the continuation of Commsrisk.
