When British vendor Revector announced the latest version of their test call generator, they focused on its USP:
Uniquely, Revector’s Test Event Generator is available as an Android application, which can be fully customised for the specific needs of network engineers.
That struck me as odd. Why would using open source software on common hardware make Revector’s tool unique? Why is it unique to use the hardware and software that is so commonly available? Lots of things are made better, and cheaper, if they run open source software on standard hardware. After all, those are the trends which have driven game-changing technologies like Big Data and network virtualization.
Some may say you need specialist equipment to do ultra-precise tests for detecting leakages and frauds, but that is hogwash. A lot of useful tests could be performed by slightly modifying standard consumer smartphones. Most fraud and assurance tests involve making a call and recording what happened – and your average smartphone is capable of doing that. Smartphones are powerful computers, and they are cheap because they are made in bulk. So anyone who wants high technology at low prices should think about how they could use smartphones to replace bespoke equipment. After all, NASA has sent Android smartphones into space and used them as satellites. Yes, you read that correctly. NASA sent Android phones into space because they provide a cheap alternative to the custom-made technology which typically gets deployed in their satellites. If it makes sense for NASA to save millions by sending phones like the HTC Nexus One and the Samsung Nexus S into space, then why do telcos struggle with the idea of adapting the same kinds of phones they already sell to customers, and using them to make phone calls?
Revector announced that their test call generator runs Android, but that is not strictly true. Technically savvy individuals know there is a difference between Android and other operating systems; I struggle to understand why Revector has chosen to misinform the market. Presumably they wish to hint that the operating system is Android-like, without giving away the ‘secret’ of which operating system they really used. I consider this to be a flawed policy; their best opportunity is to sell the test phones in large quantities at a competitive price, but if they do that, it is inevitable competitors will get their hands on one of the devices and soon discover what operating system it runs. Also, speaking as a former customer, I dislike it when vendors present misleading technical information. They must assume that customers are too ignorant to spot the differences between the marketing pitch and the product actually offered, or too indifferent to care.
Whilst engineers who are willfully ignorant of statistics may insist telcos need precision to do tests for leakages and frauds, the truth is that a higher number of tests would compensate for imprecision because variances will be identifiable by looking at the distribution of results. And the main obstacle to doing more tests is the cost of the equipment. More importantly, most things that go wrong can be identified without the need for a precise test. Put simply: if you make a call that is never billed, or is billed at the wrong rate, you do not need spurious engineering precision to spot that error. Customers spot those errors all the time, and they do not need fancy test machines!
It seems to me that having test devices which are just like customer phones also serves another benefit, because they better replicate all aspects of the customer’s experience. And that leads me to a speculation: perhaps the same development work is also behind Revector’s new OTT bypass detection product. Typical test call generators are too specialized to be used for detecting OTT bypass; networks will not terminate calls on an OTT app if the terminating device cannot run OTT apps! Most test call generators are designed to precisely record boring-boring-waffle-tech-stuff, instead of doing things that ordinary consumer phones do, like running apps. But just as I could test for OTT bypass by installing an OTT app on my phone, and getting somebody else to call it, a smartphone running the right software could do exactly the same test, without needing manual intervention.
Test call generators are an underutilized technology because they are unnecessarily expensive and restrictive. As a consequence, telecoms assurance has become synonymous with passive data collection and analysis, with little consideration given to proactive test techniques. But OTT bypass and the use of smartphones is a great illustration of how proactive testing that emulates the user experience would empower risk and assurance functions, without needing massive investments in new technology. Somebody – perhaps Revector – needs to find the right business model and marketing spin in order to justify modest modification of consumer devices, before selling them in large enough bulk to recoup their investment and generate a worthwhile profit. And if telcos know what is good for them, they will want to follow NASA’s lead, by sending smartphones to boldly test what no person has tested before…
Update 18.30 UTC, 21st December 2015: This article was amended following a complaint by Revector that commercially sensitive information had been divulged about the operating system used by its test call generators. The complaint is contrary to our understanding of the basis on which the information was originally supplied, but the article has been altered as a courtesy.
An interesting article and to emphasise a number of the points being made.
There are other reasons why I am a fan of the ‘test event generator in a phone’ solution. The ability to actually use the TCG while moving means that you can create calls established on 4G and dropping to 3G, or loss of signal conditions as they would be experienced by a customer by setting up calls while travelling on a train, walking around or driving. Not having to physically connect to a network infrastructure means that I can test calls to and from competitors networks simply by inserting their SIM card.
I suspect the reference to Android is simply a means of saying that the hardware for probes are available from your stores and local high street – I bought mine second-hand. There is about half an hours work to get them attached to the web based management system, and you are up and running. This speed of deployment I have found very useful when landing on foreign shores for a short consulting piece of work.
The thing that Eric has not mentioned is that there are many apps on the market that offer to provide call durations – however, when reading the small print, they do not claim to be accurate. If you call someone and watch your screen, you will find that the call duration starts while the far end starts to ring. So you need to be careful what you are measuring. For my work, I like the thought that my timings are repeatable and accurate, others may have different requirements.
Mark Yelland
Consultant, raaiim ltd.
Hi Mark, thanks for another insightful comment.
You’re right to emphasize the importance of timing, because people can wrongly assume that test devices measure durations more accurately than they really do. No test device will perfectly match the timings taken by network elements, because they’re not located in the same place and they don’t work the same way. However, professionals who perform tests relating to charging accuracy need to have a detailed understanding of how their test device works, and how it compares to the network elements that provide the timing which is used for calculating charges.
I know you have a keen understanding of this particular topic. It’s an area of expertise that separates you from many other consultants. It takes skill to provide a competent analysis of data, but further knowledge is needed to critique and interpret how the real-life performance of networks and test devices can influence the data that has been collected.
Hi Mark, hi Eric,
You are completely right in your comments. Probes that are used by service providers display data that may not be the same than the one used by operators to perform their billing and rating. Neverthless, these meterings can be modified with the probes provider who often changes the triggers for the start of the metering. During my experience, I remind having had this problem of accuracy only once (with a north american operator).
By the way, this metering issues are not only present for voice transactions but also for DATA transactions. But here, the problem is neglected by operators. Indeed, when you ask if they base their billing on volume up or down? or wheter if they meter in bytes or Kilobytes? or whether if they start metering when the PDP context is set up or before?…Most of the time the answer is “We don’t know”!