Data Plane Development Kit (DPDK)

A Software Optimization Guide to the User Space-Based Network Applications

Heqing Zhu, Heqing Zhu

• Publisher: CRC Press
• Year: 2020
• ISBN:
  9781000194425
• Edition: 1
• Topic: Informatik
• Subtopic: Softwareentwicklung

This book brings together the insights and practical experience of some of the most experienced Data Plane Development Kit (DPDK) technical experts, detailing the trend of DPDK, data packet processing, hardware acceleration, packet processing and virtualization, as well as the practical application of DPDK in the fields of SDN, NFV, and network storage. The book also devotes many chunks to exploring various core software algorithms, the advanced optimization methods adopted in DPDK, detailed practical experience, and the guides on how to use DPDK.

Section 1

DPDK Foundation

There are ten chapters in this section, focusing on the basic concepts and software libraries, including CPU scheduler, multicore usage, cache/memory management, data synchronization, PMD (poll mode driver) and NIC-related features, and software API (application programming interface). PMD is very important concept and the new user space software driver for NIC. Understanding the DPDK (Data Plane Development Kit) basics will build a solid foundation before starting on the actual projects.

In the first five chapters, we will introduce the server platform basics such as cache use, parallel computing, data synchronization, data movement, and packet forwarding models and algorithms. Chapter 1 will introduce the networking technology evolution, and the network function appliance trend, going from hardware purpose-built box to software-defined infrastructure, which is present in the cloud. Essentially, it is the silicon advancement that drives the birth of DPDK, and a few basic examples are given here. Chapter 2 will introduce memory and cache in the performance optimization context, how to use cache and memory wisely, the concept of the HugePage and NUMA (non-uniform memory access), and the cache alignment data structure. Chapters 3 and 4 will focus on multicore and multi-thread, the effective model for the data sharing for high parallelism, and the lock-free mechanism. Chapter 5 will move to the packet forwarding models and algorithms, and the decision is required to choose the run-to-completion and/or pipeline model, or both.

The next five chapters will focus on I/O optimization, and we will talk about PCIe, NIC, and PMD design and optimization details, which enables DPDK PMD to deliver the high-speed forwarding rate to meet demands at 10 Gbe, 25 Gbe, and 40 Gbe up to 100 Gbe in a server platform. Chapter 6 will have a deep dive on PCIe transaction details for the packet movement. Chapter 7 will focus on NIC performance tuning, and the platform and NIC configuration. Chapters 8 and 9 will go further into NIC common features and software usage, the multi-queue, flow classification, core assignment, and load balancing methods to enable a highly scalable I/O throughput, and will introduce the NIC offload feature on L2/L3/L4 packet processing. Chapter 10 is about packet security and crypto processing; securing the data in transit is an essential part of Internet security. How can DPDK add value to “encrypt everywhere”?

1 Introduction

Heqing Zhu and Cunming Liang

Intel^®

Contents

1.1 What’s Packet Processing?

1.2 The Hardware Landscape

1.2.1 Hardware Accelerator

1.2.2 Network Processor Unit

1.2.3 Multicore Processor

1.3 The Software Landscape

1.3.1 Before DPDK

1.3.2 DPDK Way

1.3.3 DPDK Scope

1.4 Performance Limit

1.4.1 The Performance Metric

1.4.2 The Processing Budget

1.5 DPDK Use Case

1.5.1 Accelerated Network

1.5.2 Accelerated Computing

1.5.3 Accelerated Storage

1.6 Optimization Principles

1.7 DPDK Samples

1.7.1 HelloWorld

1.7.1.1 Initialize the Runtime Environment

1.7.1.2 Multicore Initialization

1.7.2 Skeleton

1.7.2.1 Ethernet Port Initialization

1.7.3 L3fwd

1.8 Conclusion

Further Reading

1.1 What’s Packet Processing?

Depending on whether the system is a network endpoint or middlebox, packet processing (networking) may have different scope. In general, it consists of packet reception and transmission, packet header parsing, packet modification, and forwarding. It occurs at multiple protocol layers.

• In the endpoint system, the packet is sent to the local application for further processing. Packet encryption and decryption, or tunnel overlay, may be part of the packet processing, session establishment, and termination.
• In the middlebox system, the packet is forwarded to the next hop in the network. Usually, this system handles a large number of packets in/out of the system, packet lookup, access control, and quality of service (QoS).

The packet may go through the hardware components such as I/O (NIC) interface, bus interconnect (PCIe), memory, and processor; sometimes, it may go through the hardware accelerator in the system. Most of the packet movements and modifications can be categorized as follows:

• Data movement is, like packet I/O, from PCIe-based NIC device to cache/memory, so that CPU can process further.
• Table lookup/update, this is memory access (read/write), this is used for packet-based access control, or routing decision (which interface to be sent out).
• Packet modification involves network protocols that are defined at many different layers, just like peeling the onion layers; each protocol layer has its data format, and usually, it is defined by the International Standard like Internet Engineering Task Force/ Request for Comments (IETF/RFC). Packet processing often involves the packet change, header removal, or addition.

1.2 The Hardware Landscape

Traditionally, the network system is highly complicated and consists of the control, data, signal, and application plane; each plane can be realized with the different subsystems; and these systems are known as the embedded systems with low power consumption, low memory footprint, but real-time characteristics. Such systems require the hardware and software talents to work together.

In early 2000, CPU only had a single core with high frequency; the first dual-core processor for general computing emerged in 2004. Prior to that, the multicore, multi-thread architecture is available in the networking silicon, but not in the general-purpose processors. In the early years, x86 was not the preferred choice for packet processing. As of today, the below silicon can be used for packet processing system. From the programmer skills, they can be split into the different category.

• Hardware accelerator (FPGA (field-programmable gate array), ASIC (application-specific integrated circuit));
• Network processor unit (NPU);
• Multicore general-purpose processor (x86).

These systems are used for different scenarios; each hardware has certain advantages and disadvantages. For large-scale and fixed function systems, the hardware accelerator is preferred due to its high performance and low cost. The network processor provides the programmable packet processing, thereby striking a balance between flexibility and high performance, but the programming language is vendor specific. In the recent years, P4 has emerged as a new programming language for packet processing, and it gained the support from Barefoot Switch and/or FPGA silicon, but not common for NPU.

The multicore general-purpose processor has the traditional advantages such as supporting all generic workloads and the server platform that is commonly equipped with high-speed Ethernet adapters. The server has quickly evolved as the preferred platform for packet processing. It can support the complex packet processing together with the application and service; the application and service software can be written with many different programming languages (C, Java, Go, Python). Over the years, there are lots of high-quality open-source projects that emerged for packet processing, such as DPDK, FD.io, OPNFV, and Tungsten.io. The cloud infrastructure has gone down a path known as NetDevOps approach, ta...