Rising security and privacy concerns amid remotely accessible networks and complex multi-cloud and hybrid architectures have led to the widespread adoption of encryption. By using cipher suites and special keys to encode plaintext into ciphertext, encryption provides a simplified means to ensuring that data is transported securely and is accessible only by authorized parties. Applications and networks see encryption as a necessity due to the complex web of communication links and computing nodes involved in delivering today’s traffic. Without encryption, these links and nodes can be easily exploited by malicious third parties to gain access to an enterprise’s or end user’s data.

There is however continuous evolution in attack tactics that aim to break into encryption barriers. In retaliation to these malicious maneuvers, new and more complex encryption protocols made their debut in recent years. Among these are protocols like TLS 1.3, DNS-over-HTTPs, DNS-over-TLS and IPsec tunneling, which provide robust and rigorous traffic obfuscation. With increasing pressure from end users and regulators, a number of security frameworks such as NIST and ISO 27001 now require the incorporation of these new protocols.

Has deep packet inspection lost its relevance?

On one hand, encryption enhances data security. On the other, it challenges a core element of networking, which is traffic visibility. Traffic visibility is vital for the efficient steering and management of traffic flows. Partial or limited visibility into the underlying traffic flows greatly compromises networking functions which are tasked with handling and processing IP packets as they move through the network. One of such functions is deep packet inspection (DPI), a real-time traffic identification technology that enables network administrators to ascertain the identity, behavior and performance of packets and traffic flows navigating the network.

Traditional DPI techniques rely on packet payload readability to ascertain the underlying applications and services. This allows DPI to deliver real-time traffic analysis that enables application-aware, intelligent networking. Traditional DPI tools also gather data from packet headers, where protocols and basic packet information such as packet size and source / destination addresses are captured and analyzed alongside application information to establish various traffic performance attributes such as latency and speed.

Encryption, has, over the years, eroded parts of information that used to be available to DPI tools, raising questions on DPI’s ability to deliver comprehensive traffic insights in years to come. As more layers of information are taken off the inspection radar, conventional DPI tools will be left with very little information that makes granular traffic analysis impossible. Moreover, these visibility gaps will only worsen as tougher and more stringent encryption protocols such as TLS 1.3 and ESNI continue to proliferate.

Gauging industry views on DPI and encryption

As a leading provider of DPI technology, concerns on DPI’s effectiveness in analyzing encrypted traffic prompted ipoque, a Rohde & Schwarz company, to research the role and relevancy of DPI for modern networks and to assess the evolution of DPI techniques against new and emerging encryption protocols. The research took place last year, from October 2022 and December 2022, via a survey of 34 leading networking vendors with product portfolios spanning five different networking subsegments – security/fraud/revenue assurance, traffic management, analytics, policy control and network performance management/service assurance solutions. Results of the survey were produced in the report ‘Deep Packet Inspection and Encrypted Traffic Visibility for IP Networks’, released by ipoque on the 15^th of March.

As hypothesized, a large majority of networking vendors (85.3%) find new encryption protocols to be a major concern for modern networks as they interfere with real-time network insights which are critical to network functions. For example, network performance management becomes crippled when vendors fail to identify the applications or services contributing to packet loss or jitter on the network. Loss of traffic visibility also makes it nearly impossible to make sound network decisions and design and execute SLA-based plans.

Loss of crucial traffic insights due to encryption

When it comes to the degree of information loss from encryption, the report finds malware and threat information to be among the most affected layers. This is not surprising as it requires all associated packets of an application or service to be collectively analyzed before traffic anomalies can be identified. Without application-level behavioral baselines and network analytics, traffic irregularities arising from DDoS attacks and fraudulent log-in attempts, for example, can go unnoticed for a long time. Other affected layers include packet payload data which is necessary in establishing data breaches and data loss, and information on the degree of regulatory and policy compliance which is crucial in monitoring and protecting the movement of sensitive data.

Quest for the ultimate tool

The research report also explores tools and techniques adopted by networking vendors to overcome visibility gaps from encryption. The most straightforward and obvious solution seems to be decryption or SSL/TLS inspection. SSL/TLS inspection decrypts data packets, reads them, and re-encrypts them. The survey finds that despite its prevalence, SSL / TLS inspection is continuously challenged by various security, regulatory and practicality issues. For example, routing of traffic to a forward proxy can potentially expose plain-text data within the network, creating new security vulnerabilities in the network. Decrypting and accessing personal and sensitive data can also raise serious compliance concerns, and is already illegal in many states and countries. The report outlines many other issues associated with SSL / TLS inspection, including latency and configuration complexities.

These constraints leave most vendors in need of a better tool that can analyze and monitor encrypted traffic without breaching any operational, regulatory or security requirements. Vendors in the survey agree that such tools must be scalable and easily deployed, and must ensure accuracy, granularity and be future-proof. Ultimately, these point the shift within the industry towards non-intrusive, yet highly reliable techniques for inspecting encrypted traffic.

Emerging techniques for analyzing encrypted traffic

The report notes a strong preference for two advanced techniques for analyzing encrypted traffic. The first is the use of behavioural and statistical / heuristic analysis. The second category relates to AI-based techniques such as machine learning (ML) and deep learning (DL) techniques. Behavioral and statistical/heuristic analysis includes analysis of metadata and other available statistical information. ML and DL based tools leverage advanced ML and DL algorithms such as k-nearest neighbors (k-NN), LSTM, decision tree learning, CNN and RNN. For instance, a DL model could be trained to identify patterns in encrypted traffic that are typically associated with malware, such as abnormal traffic volume or communication with malicious domains.

Introducing encrypted traffic intelligence (ETI)

Pivoting back to DPI’s own response to encrypted traffic, the survey uncovers continuous reliance on DPI, with close to four-fifths of networking vendors using or planning to use DPI. The prevalence of its use across today’s IP networks puts the onus on DPI to continue delivering highly accurate and reliable traffic analysis, and this has obviously led to a few significant strides in the industry. Among these is the introduction of encrypted traffic intelligence (ETI) by ipoque across our suite of market-leading DPI solutions, R&S®PACE 2 and R&S®vPACE. R&S®PACE 2, which uses scalar packet processing, caters for standard filtering needs across traditional environments, while R&S®vPACE, our vector packet processing (VPP) based DPI engine, is optimized for intensive processing in cloud environments. By incorporating ETI into our DPI software we bring to market next-gen DPI engines capable of addressing not only encrypted traffic, but also any other traffic that uses obfuscation or anonymization.

Our ETI combines a mix of ML and DL algorithms with high dimensional data analysis for traffic detection and classification of encrypted traffic flows. The algorithms include k-nearest neighbors (k-NN), decision tree learning models, convolutional neural networks (CNN), recurrent neural networks (RNN) and long short-term memory (LSTM) networks. These employ thousands of features, including statistical, time series and packet-level elements, with additional features being automatically identified by DL algorithms. ETI combines these with behavioral and statistical/heuristic analysis and advanced caching techniques such as service and DNS caching to enable R&S®PACE 2 and R&S®vPACE to deliver deep traffic insights at the protocol-, application- and service-level.

Nine in ten vendors require ETI

Equipped with ETI, ipoque’s DPI technology allows highly accurate and reliable classification of underlying applications and services, enabling network administrators to implement dynamic traffic steering and application-based usage control, while keeping tabs on malicious activities. For proactive and automated management of networks, networking vendors can leverage comprehensive analytics by DPI to create usage and behavioral thresholds for anomaly detection, optimize network performance based on SLAs and performance KPIs, identify and prioritize critical traffic flows, and ensure optimal resource utilization.

The market demand for DPI with encrypted traffic intelligence is evident from the research, which finds 92.6% of networking vendors to already have or planning to incorporate ETI in their existing or future DPI solutions. This validates ipoque’s continuous effort in this space and marks the increasing awareness of the importance of next-gen DPI in addressing the complexities introduced by the latest encryption protocols.

Timely debut

Despite boasting a combination of advanced, computing-intensive ETI techniques, R&S®PACE 2 and R&S®vPACE are easily deployable, highly scalable, and are designed to cater for both traditional scalar packet processing and new vector packet processing (VPP) environments. This enables the modules to be implemented in all types of networks – traditional, virtualized and cloud-native. Vendors deploying R&S®PACE 2 and R&S®vPACE have access to ipoque’s comprehensive signature library that is updated weekly, incorporating the latest applications and threats patterns. Users can also take advantage of ipoque’s additional features and plug-ins to create a tailored DPI solution that meets their unique needs. Our customers also benefit from our long-standing leadership and expertise in DPI, with extensive ongoing research on latest classification methods for encrypted and non-encrypted traffic.

Our latest report will be a great read for networking vendors challenged by encryption and who are contemplating the role of DPI in delivering traffic insights in the wake of newer and tougher encryption protocols. In addition to the above findings, the report provides in-depth analysis of various other key topics including vendors’ awareness of encryption protocols, the prevalence of different encryption technologies, benefits from encryption, destabilizing effects of encryption on network management and security, demand for ETI in general and shifts in DPI procurement choices. Find out how you can leverage DPI to address encryption challenges by downloading a copy of our report which is available for download here.