DPDK Support | PcapPlusPlus (2023)

DPDK Support | PcapPlusPlus (1)

The Data Plane Development Kit (DPDK) is a set of data plane libraries and network interface controller drivers for fast packet processing, currently managed as an open-source project under the Linux Foundation. The DPDK provides a programming framework for x86, ARM, and PowerPC processors and enables faster development of high speed data packet networking applications (taken from Wikipedia).

DPDK provides packet processing in line rate using kernel bypass for a large range of network interface cards. Notice that not every NIC supports DPDK as the NIC needs to support the kernel bypass feature. You can read more about DPDK in DPDK's web-site and get a list of supported NICs here.

As DPDK API is written in C, PcapPlusPlus wraps its main functionality in easy-to-use C++ wrappers which have minimum impact on performance and packet processing rate. In addition it brings DPDK to the PcapPlusPlus framework and APIs so you can use DPDK together with other PcapPlusPlus features such as packet analysis, etc.

What does PcapPlusPlus offer for DPDK?

PcapPlusPlus tries to cover the main functionality of DPDK and its most important and popular features. Here is what PcapPlusPlus offers for DPDK:

  • Encapsulation of DPDK's initialization process - both outside and inside of the application - using simple scripts and methods
  • A C++ class wrapper for DPDK's packet structure (mbuf) which offers the most commonly used functionality
  • A C++ class wrapper that encapsulates a DPDK-controlled NIC (A.K.A DPDK port) for receiving, processing and sending packets using DPDK as well as an interface to retrieve NIC info, status, etc.
  • Offload packet capture to multiple worker threads running on different cores + control the load balancing configuration between these workers
  • Multi RX/TX queue support
  • Detailed statistics about packets being captured and processed
  • Seamless integration to other PcapPlusPlus capabilities including: parsing of packets received with DPDK using the various protocol parsers offered in PcapPlusPlus, saving packets to a pcap file, sending crafted/edited packets through a DPDK-controlled interface, packet reassembly, and more.
  • An easy-to-use C++ wrapper for DPDK KNI (Kernel NIC Interface)

Set up PcapPlusPlus with DPDK

Supported DPDK versions

The following DPDK versions are currently supported:

  • DPDK 21.11 (LTS)
  • DPDK 20.11 (LTS)
  • DPDK 19.11 (LTS)
  • DPDK 18.11 (LTS)
  • DPDK 17.11 (LTS) (not regularly tested)

Older and newer versions may not work.The non-LTS versions may also work, but are not regularly tested.

In addition, not all poll-mode drivers (PMDs) were tested, but the majority of them should work. Please report an issue if the PMD you're using isn't working.

(Video) DPDK support for new hardware offloads

The following operating systems and configurations were tested:

  • Ubuntu 20.04, 18.04 LTS 64bit with kernel version > 3 and gcc version >= 4.8
  • CentOS 7.1 64bit with kernel 3.x and gcc 4.8.x (not regularly tested)

Download and install DPDK - prior to DPDK 20.11

Building and installing DPDK is pretty straight-forward and in a nutshell goes like this:

$ cd /dpdk/source/location
$ make config T=[platform_type_string] && make

Download and install DPDK - DPDK 20.11 forward

DPDK changed their whole build system in version 20.11 and is now using meson and ninja. The full build instructions can be found here, but here are the important steps in short:

$ meson build
$ cd build
$ ninja
$ ninja install
$ ldconfig

It is important to build and install DPDK and also run ldconfig as mentioned in the documentation, otherwise PcapPlusPlus won't be able to find the DPDK binaries during the link process.

(Video) Introduction to DPDK

PcapPlusPlus configuration for DPDK

Once the DPDK build is completed successfully, run PcapPlusPlus configuration script (configure-linux.sh) to build PcapPlusPlus with DPDK. Please refer to the configuration instructions to get more details.

DPDK initialization with PcapPlusPlus

DPDK has two steps of initialization: one that sets up Linux to run DPDK applications and the other at the application level which initializes DPDK data and memory structures. PcapPlusPlus wraps both of them with easy-to-use interfaces. Please see the following two sections to get more information.

Initialization before application is run

Several Linux configuration steps are needed to run DPDK applications:

  • DPDK uses Linux huge-pages for faster virtual to physical page conversion resulting in better performance. Huge-pages must be set before a DPDK application is run
  • DPDK uses a designated kernel module for kernel bypass (there are 3 supported options: igb_uio, uio_pci_generic, vfio-pci). This module needs to be loaded into the kernel if not already loaded
  • One or more NICs should move from Linux control to DPDK control
  • For DPDK KNI there is one more kernel to be loaded into the kernel (rte_kni.ko)

PcapPlusPlus offers a python script that automatically configures all of the above. The script is located in PcapPlusPlus root directory and is named setup_dpdk.py. It is based on the dpdk-devbind tool that ships with DPDK and extends it with more functionality. The script supports Python 3+.

This script has 3 modes of operation: setup to configure the steps mentioned above, status to view the status of all known network interfaces, and restore to go back to the original Linux configuration.


Note: this script uses another file named setup_dpdk_settings.dat to keep information needed for restoring the Linux configuration. This file is also located in PcapPlusPlus root directory. Please do not remove this file

(Video) Setup DPDK Development Environment (on PC)

Setup - takes the following parameters:

-g, --huge-pages AMOUNTThe amount of huge pages to allocate. By default each huge-page size is 2048KB
-i, --interface NIC_NAME [NIC_NAME ...]A space-separated list of all NICs that should move from Linux to DPDK control. Only these NICs can be used by DPDK, the others will stay under Linux control
-m, --dpdk-module {igb_uio,uio_pci_generic,vfio-pci}The DPDK module to load. If not specified the default is igb_uio. NOTE: in DPDK 20.11 igb_uio was moved outside of the DPDK repo into a separate repo which means it doesn't come out of the box and needs to be built separately. However the recommendation is to use either vfio-pci or uio_pci_generic which come as part of most Linux distros. You can read more about it here
-k, --load-kniInstall the KNI kernel module (not loaded by default)
-p, --kni-params KNI_PARAMSOptional parameters for installing the KNI kernel module
-r, --rte-sdk RTE_SDKDPDK home directory. If this argument is used, the script will use it as the DPDK directory, otherwise it will look for the RTE_SDK environment variable. If both don't exist it will look for RTE_SDK in setup_dpdk_settings.dat. If none of them exist an error will be thrown
-v, --verbosePrint more verbose output

If everything goes well the system should be ready to run a DPDK applications and the output should look something like this:

pcapplusplus@ubunu:~/PcapPlusPlus$ sudo python setup_dpdk.py setup -g 512 -i enp0s3
[INFO] set up hugepages to 512
[INFO] loaded kernel module 'uio'
[INFO] loaded DPDK kernel module 'igb_uio'
[INFO] set interface 'enp0s3' down
[INFO] bound interface 'enp0s3' ['0000:00:03.0'] to 'igb_uio'

Status - shows the interfaces under DPDK and Linux control. In the example below one interface is under DPDK control and the other (enp0s8) is under Linux control:

pcapplusplus@ubunu:~/PcapPlusPlus$ sudo python setup_dpdk.py status

Network devices using DPDK-compatible driver
0000:00:03.0 '82540EM Gigabit Ethernet Controller 100e' drv=igb_uio unused=e1000,vfio-pci,uio_pci_generic

Network devices using kernel driver
0000:00:08.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s8 drv=e1000 unused=igb_uio,vfio-pci,uio_pci_generic *Active*

No 'Baseband' devices detected

No 'Crypto' devices detected

No 'Eventdev' devices detected

No 'Mempool' devices detected

No 'Compress' devices detected

No 'Misc (rawdev)' devices detected

This command take the following parameters:

(Video) DPDK deep-dive

-v, --verbosePrint more verbose output

Restore - restores the Linux configuration to its previous state before setup_dpdk.py setup was run. Please note that this command uses the setup_dpdk_settings.dat file to identify the previous state. If this file was deleted or moved the restore process will most likely fail. Please also note that a machine restart will probably restore most of this configuration, but this command enables restore without a machine restart.

If everything goes well the output should look something like this:

pcapplusplus@ubunu:~/PcapPlusPlus$ sudo python setup_dpdk.py restore
[INFO] removed hugepages
[INFO] bound device '0000:00:03.0' back to 'e1000'
[INFO] restored interface 'enp0s3'
[INFO] removed 'igb_uio' kernel module

This command take the following parameters:

-v, --verbosePrint more verbose output

Initialization at application startup

When using DPDK in your application it should be initialized on application startup. This configuration includes:

  • Verify that huge-pages, kernel module(s) and NICs are all set
  • Initialize DPDK internal structures and memory, poll-mode drivers etc.
  • Prepare CPU cores that will be used by the application
  • Initialize packet memory pool
  • Configure each NIC controlled by DPDK

These steps are wrapped in one static method that should be called once in the application startup:

(Video) Debunking the Top 10 Myths of DPDK

If this methods succeeds DPDK is ready to be used in your application.

Next steps

If you're curious to learn more, please refer to the following resources:

  • DPDK tutorial
  • DPDK example applications: DpdkExample-FilterTraffic, DpdkBridge, KniPong
  • DPDK API reference. A good starting points would be DpdkDevice.h file description and DpdkDevice class description
  • You can also find all the unit-tests in the Pcap++Test project (search for tests that contain "dpdk" or "kni")


1. DPDK Support for New Hardware Offloads
2. Build and Launch Docker* Containers with DPDK Support | Intel Software
(Intel Software)
3. Run DPDK testpmd application to send tens millions of packet 1
4. Data Plane Development Kit (DPDK )
(Tech Upskill)
5. OVS-DPDK vhostclient setup
(Naftaly Avadiaev)
6. DPDK Support for Vhost Acceleration
(DPDK Project)
Top Articles
Latest Posts
Article information

Author: Prof. An Powlowski

Last Updated: 04/02/2023

Views: 5826

Rating: 4.3 / 5 (64 voted)

Reviews: 95% of readers found this page helpful

Author information

Name: Prof. An Powlowski

Birthday: 1992-09-29

Address: Apt. 994 8891 Orval Hill, Brittnyburgh, AZ 41023-0398

Phone: +26417467956738

Job: District Marketing Strategist

Hobby: Embroidery, Bodybuilding, Motor sports, Amateur radio, Wood carving, Whittling, Air sports

Introduction: My name is Prof. An Powlowski, I am a charming, helpful, attractive, good, graceful, thoughtful, vast person who loves writing and wants to share my knowledge and understanding with you.