The telecom world is quickly moving towards networks built on SDN and NFV infrastructure. These two technologies are creating new business opportunities for telcos to quickly develop and deploy new and useful cloud services.
Another key attraction of NFV and SDN is its ability to abstract the operation and monitoring of services across a broad range of devices, PCs, and servers. At the same time, the virtual machines running these technologies can live everywhere from servers in a data center to CPE devices at the customer’s site.
And while NFV and SDN deployments have just started and will take years to evolve, many of the telco giants are committed to migrating many of their dedicated network equipment to full software control.
This brave new virtual services world is very exciting, although there are many serious security concerns with moving to NFV infrastructure or NFVi.
And here to help us sort out this highly complex topic is Gal Ofel, Director of Software Solutions at Telco Systems, a pioneering company in the SDN and NFV sector.
A few months back, Gal and I had a great discussion on the many revenue-generating services that SDN and NFV will enable, making this conversation about the security concerns of these technologies quite timely.
Dan Baker, Research Director, TRI: Gal, what’s more fundamental than security? I can’t imagine NFV/SDN seriously taking off until security issues are nailed down.
I think there’s a parallel here to telecom’s move from circuit-based to IP-based voice services.
Circuit-based phone networks were very secure and had high quality. The reason for that was the network equipment vendors maintained strong end-to-end control of either the A or B side of the call.
Now the telecom world moved to voice over IP service because it had other advantages: it was a much cheaper service to deliver, while IP eased the path to enhanced services, like voice mail and “follow-me” routing of calls.
But, let’s not forget that tradeoffs were made. In the circuit based voice world, the quality of calls was great. However, voice quality issues and dropped calls in the IP world are a regular occurrence. Since so many service providers are in the IP chain, greater QoS monitoring and protection against malware is required.
Now, we know that engineering innovation may cause IP’s quality and security issue to go away, but it remains a work in progress, just like NFV/SDN.
Gal Ofel: It’s true, Dan. Anytime you introduce more flexible and dynamic network architecture, you tend to pay a price in terms of greater security and quality monitoring.
So, if I may, I’d like to walk you through what I consider the four key security vulnerability areas in NFVi:
1. Migration from Secure Hardware to Less Secure Software
The first issue is the security risk that comes from migrating hardware-based data packet processing equipment into something that is software-based.
For example, routers and firewalls are traditionally run on dedicated hardware with either ASICs developed for that purpose or FPGAs. These hardware-based solutions are very stable and run at full wire speed. When they are overloaded, they know how to handle the situation and do bypass. What’s more, the hardware is difficult to crack from a security point of view.
But now, as you move away from hardware, you become much more vulnerable to the Distributed Denial of Service (DDOS) attacks. For instance, it’s much easier to crash software as it is much more sensitive to peaks in traffic or other issues, such as opening many new connections at the same time.
And since the control plane sits on a PC or server, it’s easier to attack it and crash it by creating a high load.
2. Open & Exposed NFV Infrastructure
Another key security issue is that NFVi is quite open and exposed.
Unless security measures are added on, the control planes within NFVi can be openly accessed by a management tool. In fact, the NFVi can span any host or any server in the data center of the carrier – and even the CPE devices a carrier provides to business customers.
Not only can software be added, removed or re-configured at any time, in some cases the end user accesses the system via a self-service portal, which could expose the NFVi to external malicious control. So the inherent openness of NFVi becomes a magnet for trouble.
Now, if NFV is strictly controlled inside the data center, security may be less problematic because security is centralized there. However, carriers are talking about deploying NFV widely across the network, such as in applications like heavy mobile edge computing that employ NFV-enabled CPE devices.
3. Third Party Software
Yet another NFVi vulnerability is all the third party software installed on the machines and servers where the NFVi lives. Some of that third party software could be infected or carry malicious code. The added value of NFV is that we could choose and execute VNFs from different vendors and choose the best fit.
Here’s the issue: it’s difficult to detect and control malware that is already sitting somewhere on a host and inside the firewall. Even worse, that malware can also propagate itself across the network. Some call this East West traffic propagation – and it’s a huge issue.
4. The Complexity of Virtual Machine Environments
And one of the final key security concerns is the virtual machines themselves. A server will often run many different VNFs, so if a server is experiencing problems, many VMs may be affected.
So the weaving of all these VMs across the servers makes it hard to isolate root causes. And one bad virtual machine might knock out several others.
Yet another complication is all the updates to the VNF across numerous brands of third party software. So you add it up: the inherent complexity of VMs operating across many different machines and across many different vendors, and there are lots of security vulnerabilities that complexity brings in the door.
It’s a nice backgrounder, Gal. I understand Telco Systems had developed a solution to these NFVi security problems.
Yes, our new solution is called NFV CyberGuard. It was released towards the end of last year and it is attracting lots of attention.
It’s made up of two elements. The first element is a network probe — software that can be deployed in different locations in the network.
The probe operation is analogous to the surveillance cameras you have at a manufacturing plant. The plant will have gates and door to protects its perimeter, like a firewall. But the surveillance camera is essential for visualizing what occurs inside the premises.
The probe basically identifies traffic and reconstructs sessions. It then applies rules to pinpoint malicious traffic or suspicious events. The idea is to either capture the suspicious content or run it through a big data engine for further analysis.
For example, let’s say we suddenly see someone is trying to mange 200 instances of the same VNF in different machines. And perhaps the IP address is constantly changing, indicating someone is trying to hide their activities.
That sort of problem would be identified by our surveillance probe.
Here’s a another example. The bad guys are trying to harvest passwords using a robot that runs a brute force attack to test thousands of potential passwords. Now password harvesting is difficult to detect if you are only scanning a few host computers. But if you can look across the entire network, you can detect a pattern as the hacker tries to break the password from multiple addresses. So having a network view of multiple hosts and devices allows you to find suspicious patterns you would not normally catch.
The second element of the solution is an NFVi agent that monitors what is happening inside the NFV infrastructure. It looks at the hypervisor, the virtual switch and other elements — basically looking at what a virtual machine is doing with its hosts or how one VM is trying to access another VM.
An example of a security breach here might be some malware that’s penetrated many machines and is now trying to capture the log files of another one. That’s not a normal situation, so the NFVi agent would raise an alert.
I’m curious to know what’s actually happening in the marketplace. What’s been the experience with security in virtual environments so far?
There have been some major problems. For example, in May 2015, a new attack was identified called VENOM: Virtualized Environment Neglected Operations Manipulation. It was a virtual environment manipulation where the hackers penetrate the hypervisors. And it exploited open source code through the floppy disk on a machine.
Now nobody is using floppy disks anymore, but through that interface it was possible for an attacker to get access to a VM and also escape from the host and from there it was possible to access the whole network.
In this case, a leading cloud provider had to restart all their cloud servers. It was a huge operation that they did successfully, however the risk was there.
So this is quite troubling since the whole world uses open source and that code is created by thousands of developers. That means the number of potential attack vectors is enormous.
Gal, thanks for this interesting briefing. Sounds like NFV will be adopted cautiously till these big security issues are under control.
Yes, it’s true. There’s a big list of security scenarios that we are pushing on the algorithm creation side. But I think the industry will get there. Over time, our ability to close NFV vulnerabilities will improve and that will give the industry the confidence it needs to make progress. I’m very optimistic.
