Telcos make test calls for lots of reasons. Just as software always needs to be tested, so does every aspect of a comms service, including the software, hardware, configuration and the human operations that surround them. Revenue assurance is one of the possible goals of test calls; some telcos have been using test call generators (TCGs) to independently verify their metering and billing for well over a decade. However, there is a split between telcos that use test calls as part of their revenue assurance strategy, and others that see no need to perform test calls. And amongst the telcos that execute test calls there is a second split, between those which perform test calls manually, and those which use automated systems to make the calls and reconcile them to data output from billing and other systems. These differences in thinking beg many questions. Why does the world of revenue assurance exhibit such contrasting attitudes to test calls? Who has the best approach? Would investing in TCGs deliver good returns for RA teams? Should RA teams treat test calls as an essential control, or can they live without them? These are the questions we explore in episode 25 of the Commsrisk podcast.
Our guests for the podcast were from SIGOS, a leading global manufacturer and provider of TCGs and test call services. Though SIGOS has a solid reputation, they would admit that revenue assurance was not a customer segment they targeted in the past. They have since overhauled their marketing strategy and give more priority to revenue assurance than previously. As such, SIGOS’ internal discussions represent a microcosm of the wider debate about the extent to which TCGs can benefit revenue assurance. From SIGOS we were joined by Steffen Öftring who oversees their RA division, and Paul Lia, who handles business development for both revenue Assurance and fraud. Steffen and Paul were forthright and articulate in arguing the case for test calls, and they presented a distinct vision of test call services which contrasts with some of the arguments made by vendors in the past. They were also generous and patient when fielding all our tricky questions!
Dan Baker was my co-host once again. Dan had the difficult decision of calling time on a conversation that was entertaining whilst also being deep and informative. Our interview lasted just over an hour, but given Steffen and Paul’s willingness to freely answer our challenging questions, we could have continued much longer.
You can listen to the interview via your internet browser, by pressing the ‘play’ button on this page. Alternatively, feel free to download the mp3 file from here. A good way to avoid missing future episodes of the Commsrisk podcast is to subscribe to one of the free services that will download episodes automatically. For example, visit the Commsrisk page on the iTunes Store, or follow our link on the Blubrry Android service.
I learned a lot from this discussion. Thank you for sharing it.
Thanks for the interesting podcast – it raised a few questions for me and a few comments as well. In no particular order:
– I agreed with Eric’s observation on defining the “role” for TCGs in an overall revenue assurance strategy. It seems that the focus of Sigos is headed toward tariff validation activities which makes some sense but there are existing tools in this space. I don’t think you need an externally validated xDR to do this as you are really checking the rating or billing calculation engine is applying the right pricing to the input data it receives. Now TCGs have always had a role to play in that second case.
– It would have been interesting to understand any trade offs in some of the larger test call programs beyond next capacity. If you are testing off network (domestic or roaming) then are there interconnect costs that get incurred with real cash out.
– Similar to the above, and with many plans now being “all included”, I would be interested if Sigos customers are testing calls to international destinations. Are there interconnect costs incurred for that program, and does Sigos support TCGs in other countries as end points so carriers do not need to place TCGs all around the world?
– Eric said he couldn’t think of a reason why a regulator would not run a test call program on behalf of all operators in that country. I can think of a few – the focus for the regulator may be consumer protection and overcharge so in any analysis that is likely to be the focus but which telco wants the regulator telling them they have an overcharge before they know it? Similarly, regulators are often criticised for being slow to publish results – a telco wants more immediate feedback if they have an undercharge error so that it can be corrected sooner. Lastly, what level of interaction will be needed to inform regulators of new tariffs and be confident they can configure these correctly for subsequent testing? They are complex enough already and how protracted might the process be of exception validation between the regulator and the telco be, before agreement is reached on what is “really” an overcharge and what is not. Efficiency gains from this? I am not convinced. Lastly, regulators may well be subject to FOI requests – would telcos be comfortable that all this data and analysis be made public for possible interpretation and mis-interpretation by others, including possible tariff comparisons between plans (assuming similar call schedules are set up)?
– little mention was made of signalling as an alternate to switch verification. A long standing “competitor” to TCGs from a validation of event creation.
Hi Mike, thanks for all your thoughtful points. There is enough here to have justified a full-blown article expressing your opinion!
I’ll address the point which most specifically relates to my position: why regulators should do test calls to validate bill accuracy on behalf of customers.
Firstly, it doesn’t matter if telcos would like the idea or not: the beauty of test calls is that you, I, a regulator or anyone could make test calls, with or without the permission of telcos. If the reason regulators don’t do them is because they want to protect the sensitive feelings of telcos then that, in my opinion, is lousy and flawed reasoning. Regulators don’t exist to appease telcos, whilst consumer protection is a vital aspect of their mission.
Secondly, if telcos don’t want to hear about overcharges from a regulator then they shouldn’t want to hear about overcharges from their customers… can you see where this point is headed? I would have more sympathy with this argument if telcos did a better job in the first place. Many telcos think it is normal to mischarge customers, then issue credits when customers complain. We don’t accept this kind of business behaviour in other industries, so why should telcos be so slack and self-indulgent? And if the point is that telcos are happy to be caught out by (individual, powerless) customers but not by the regulator, then that is another good reason for the regulator to act on behalf of all customers.
Thirdly, I see no reason why a testing programme couldn’t work more efficiently than you suppose, and provide a speedy alert of any errors detected. Telcos are expected to be able to port numbers and provide other key services within days. What stops a regulator’s test call system from being set up with the right contact details so an alert is sent the relevant telco contact as soon as a mismatch is detected? That approach would make sense – perhaps the telco will identify reasons why the apparent mismatch isn’t an error after all. In the end this problem can be solved by technology, so if bureaucracy is getting in the way it’s because regulators are bureaucratic and incompetent with technology – which is all the more reason to replace their backwards consumer protection methods with proper testing using straightforward and efficient technology.
Finally, I don’t think the test program needs to check tariffs. As you made clear in an earlier point, rating accuracy can be handled separately from tests of metering accuracy. In fact, customers are far better placed to check rates than they can check if a call record was valid and accurate. So I think including rating accuracy would only muddy the waters and confuse the consumer protection issues. Regulators could alleviate a burden for customers where it’s hard for them to protect themselves. Instead, they overly rely on customers complaining to protect themselves. It follows that you don’t need to implement tests in areas where customers are capable of checking prices themselves, but you should implement tests to protect customers in ways they can’t replicate at home.
As for FOI requests, I consider that to be a trivial objection (though perhaps typical of the way data protection does get thrown up as an excuse to avoid doing sensible things even when it’s not really a genuine concern). A lot of government data is now available for public consumption and the downside risk has been minimal, because generally people don’t spend a lot of time “misinterpreting” government data in a way that causes any harm. (They may, however, correctly interpret data in ways that harm people and organizations that deserve some harm!) As for tariff comparisons, even if the regulator was testing tariffs I can’t see the logic which allows consumers to draw an inference about who offers the best price as we’ve not discussed the extent to which the call patterns match real customer behaviour. More importantly, you could do the same analysis without making a single test call – you could just shove the tariff data into a database and then do a monte carlo analysis using hypothetical ‘calls’. Nobody is doing this monte carlo analysis, so what is the chance somebody will try to pervert the results of a test call program to draw an invalid inference that is so abstract in nature the average consumer will struggle to understand why they should believe it?
Thank you Mike for the points you raise and thank you Eric for your responses regarding Regulators.
Mikes first point is totally correct, in theory. IE Test calls can be used for tariff validation but most telcos already have other tools that could do this. In reality approximately half of the issues we detect could have been identified by passive CDR analysis. But they were not! There can be many reasons for this, but ultimately they don’t matter. The truth is they could have been, even should have been detected by other processes but they weren’t. So having an independent process of externally testing and validating tariffs and business rules is essential if it is important to detect tariff issues as soon as possible. Given the volumes of revenue involved this would seem to be a prudent course of action. SIGOS carried out a large number of audits last year in most regions of the world including Europe and North America. All of these telcos had passive tools in place to re rate cdrs and check the tariffs were being billed properly. Some of the telcos went further and also used active test calls themselves or through managed service providers. SIGOS still found tariff errors in every single audit.
For Mikes other point I can confirm that SIGOS supports TCG in virtually every country so that operators can make test calls to and from any country without needing to place their own TCG all around the world. Sim multiplexing makes it even easier for operators as their sims don’t need to leave our office in order to make a call from anywhere to anywhere. Our customers test offnet calls as well as IDD and roaming calls. As Mike points out, this does generate real interconnect charges. So thought is given to the volume of these test events in order to target the revenue risk points and keep charges to a minimum. Ultimately the size of the charges is a very small fraction of the revenue generated through these channels. The value of the issues detected by this testing more than justifies the charges. A positive side of these interconnect charges is that it allows us to test their accuracy. In theory we should never find any errors in these charges from foreign networks as they are checked by a Data Clearing House before our customers receive them. We still find errors. This is another example of testing that should not be necessary in theory but is essential in reality.