Making sense of big data

In the era of Big Data businesses across the world are gathering huge quantities of information on their customers’ activity and their own operations every day. Mobile operators are no exception and are experimenting with new ways to collect and analyse as much data as possible to enhance their customers’ experience and gain a competitive edge.

However, operators face new layers of complexity as networks move from 3G to LTE. Historically the protocols used in telecom networks generated a lot of management information and networks could be polled regularly to get a full picture of what was going on. In 3G environments, as traffic passed through the network, the network engineer was always in full control.

However, Ethernet and IP have changed all that. IP networks are dynamic by nature and network and traffic flows can change in a matter of nanoseconds. But these transfer protocols do not yield a lot of management information so, in order to build a complete picture of what is going on in a network, the whole network needs to be monitored in real-time, says Dan Joe Barry, VP marketing at intelligent network adapter provider Napatech. “The problem with Ethernet and IP is that, once you’ve got it up and running it’s very hard to manage,” he explains. “The network is figuring things out for itself; it is dynamically changing all of the time, based on the situation. Algorithms are always working to optimise to adapt to what is happening in the network. That’s where solutions had to be created to provide insight into what is happening in the network.”

Barry explains that probes built with Napatech network adapters tap into the network at different locations to give operators a clearer view of whatever part of the network they want to look at. By using these probes, operators are able to view all the packets going back and forth. Such solutions are important in monitoring various scenarios throughout the telecoms network. Performance monitors at network and application level are the most common use cases for monitoring solutions, in order to view how the network is performing in general, or in different geographic areas. Probes are also useful in monitoring the network for certain vertical use cases, he says. For example, latency measurement solutions, which are often used in financial networks, try to determine how long it would take for data packets to get from one part of the network to another. This is vital knowledge for operators that have networks that are used for trading. Another key area is in security, where the operator has to protect against attacks and manage firewalls, intrusion prevention systems, event management systems and many other appliances and solutions.

“We see a lot of new and innovative products coming along every day that are designed to provide insight into what’s happening in the network for a specific purpose,” explains Barry. “The one challenge that all of those products create for operators is how to get all of the data from the network and have that available to me so that I can analyse it for the purpose I need to analyse it. That was the problem that Napatech was founded to solve. It turns out that that’s a very difficult challenge to solve.”

The reason it is so difficult, according to Barry, is that “in a normal communication situation”, an operator is only interested in the packet that goes between the user and the person they are communicating with. All the other packets on the network can be filtered out. However, when an operator wants to analyse a session, they are interested in every data packet in the network, and this can be difficult to manage.

“The basic proposition we have is that we guarantee delivery for all the data for analysis; zero packet loss. Not only at the adapter level but also through the system, through the server, and right up to the application that needs the data, we guarantee they get all that data. That’s the proposition and that’s what we’ve built a name on.”

And while one part of Napatech’s challenge is in designing the product, Barry explains that another is in enabling customers to use standard servers with reliability and ease of use.

“It is simply a case of plugging in the adapter, installing the software and off you go,” says Barry. “It’s reliable and we’re trying to make it as easy as possible for those guys to do what they need to do by using standard servers, and still not compromise on performance.”

He claims that one of Napatech’s key attributes is that the firm does not work directly with mobile operators, but exclusively with appliance vendors. The proposition for such vendors is that they are not lumbered with the task of building proprietary hardware and programming chips themselves.

“We say you focus on the software; get a server from HP, Dell or whoever you like, and you put our card in the slot and the hardware part is done. If you don’t even want to touch the hardware, we have system integrator partners and they can put the solution together.”

This means that appliance vendors can focus exclusively on the software development because the real value in those solutions is in the software, Barry argues.

In this video, Dan Joe Barry, VP Marketing at network analysis solutions provider Napatech, explains how operators can collect and analyse data from their networks in real time.

“We focus on our job, which is data handling. So that leaves these guys who are security experts, who know about hackers, tricks, malware and algorithms, to handle that. That’s a very specific knowledge base to have, especially at that level. It’s not fair to expect them to know about the security details at that level and to also understand Ethernet technology and servers. Focus on what you’re good at. You do what you’re good at and we’ll do what we’re good at, and that’s the secret of success. Our customers who have gone down that path haven’t looked back.”

Barry also argues that the possibilities for the operator are endless once they have this information to hand. Firstly, by collecting and analysing data, operators can understand what is happening in their network. They can then make decisions based on that data.

Certain operators will use the data to focus on network optimisation and optimise the services they are offering. Others will be interested in using that information to create new types of offerings, and services, based on the behaviour in the network. “When you have a stream of real time data, you can store that and use it as historical data,” says Barry, “It can be a very reliable source of information about what has happened at any time in your network. We collect data on every single frame, every packet and you can recreate precisely what happened in the network at any point in time.”

He adds that this is useful for trouble shooting, but it is also useful for analysis. Operators can see how the network changes at different times of the day and how users behave throughout the day and on weekends and holidays. It can also monitor what kind of applications they are using. With that kind of information, operators can optimise their billing strategies, innovate with new pricing model and put together promotional offers.

According to Barry, real-time data enables real-time reaction. So, instead of engineers waiting hours or minutes to resolve issues, they can be addressed as they happen. And that’s only one side of the equation‑realtime data acquisition means that the data is complete and reliable for whatever post-event or real-time analysis an operator wants to perform. This, combined with real-time reaction, enables a number of exciting opportunities, such as the ability to predict problems before they occur and react immediately when a change or an anomaly occurs. It also allows an operator to plan more effectively based on reliable historical data. In short, it opens the possibilities of what can be done with networks and OSS/BSS systems.

“The key thing is having the insight. Once you have that, there are lots of possibilities. I think in general, carriers, like any other end user—are interested in finding out how they can use analysis and big data to run their business better. How can I use data to run my business better? This is one trend of all businesses at the moment—the feeling that if you can do this well, you will stay ahead of your competition. It enables you to base decisions on hard data, rather than intuition or feeling.”