4. How to install DPDK for Lagopus software router

This section describes how to install DPDK [1] for Lagopus software router.

[1]For more information about DPDK, check http://dpdk.org/

4.1. Minimum hardware requirement

Lagopus software router requires DPDK which uses multiple CPU cores to achieve high-performance network I/O and processing performance. Therefore, you have to prepare a host machine that meets the following minimum hardware requirements.

  • # of CPU cores: 4 or more
  • Memory size: 4 GB or more
  • NIC: DPDK-enabled NICs

4.2. Software Versions

4.3. Installation steps

Lagopus software router requires to build DPDK in shared lib mode. Follow below steps to install DPDK in shared lib mode.

  • Install Ubuntu Server.

  • Download and extract DPDK source code

    $ wget http://fast.dpdk.org/rel/dpdk-17.11.1.tar.xz
$ tar xvf dpdk-17.11.1.tar.xz

  • Install necessary packages.

    $ sudo apt update
$ sudo apt install gcc make python libnuma-dev libelf-dev

  • Modify config file to enable CONFIG_RTE_BUILD_SHARED_LIB

    $ cd dpdk-stable-17.11.1
~/dpdk-stable-17.11.1$ cp config/common_base config/common_base.original
~/dpdk-stable-17.11.1$ vi config/common_base
< CONFIG_RTE_BUILD_SHARED_LIB=n
---
> CONFIG_RTE_BUILD_SHARED_LIB=y

  • Build and install DPDK

    ~/dpdk-stable-17.11.1$ make T=x86_64-native-linuxapp-gcc config
~/dpdk-stable-17.11.1$ make
~/dpdk-stable-17.11.1$ sudo make install

4.4. Setup DPDK

To enable DPDK on your environment, you have to enable hugepage.

  • Setup Hugepages

Make hugepages available to DPDK. You can setup hugepage in two ways:

  1. Manual configuration: Repeate steps after reboot if you select this.
  2. Script configuration: Select this to keep it permanent after reboot.

Note

When configured manually, you need to repeat the same steps after rebooting OS.

  1. Manual confguration (undone after reboot)

    Perform the following commands

    $ sudo sh -c "echo 1024 >  /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages"
$ sudo mkdir -p /mnt/huge
$ sudo mount -t hugetlbfs nodev /mnt/huge

  2. Script configuration (permanent after reboot)

    • Reserve 1024 pages of 2 MB hugepages in linux by adding the following line in /etc/sysctl.conf.

      $ sudo vi /etc/sysctl.conf
vm.nr_hugepages = 1024

    • Add a directory for hugepages and the following line to /etc/fstab so that mount point can be made permanent across reboots.

      $ sudo mkdir -p /mnt/huge
$ sudo vi /etc/fstab
nodev /mnt/huge hugetlbfs defaults 0 0

    • Reboot

      $ sudo reboot

    • Confirm HugePages are configured correctly by the below commands.

      $ grep -i "HugePages" /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
$ mount | grep huge
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
nodev on /mnt/huge type hugetlbfs (rw,relatime,pagesize=2M)

4.5. NIC (Network Interface Card) assignment

The following steps detatch the control and management of NICs from Linux kernel and attach drivers used by DPDK application.

In this example, the host has 4 NICs where we are going to use 3 oth them (enp0s8, enp0s9, enp0s10) for Lagopus software router.

Note

  • You need to unbound NIC from kernel (ixgbe driver) before using it with DPDK.
  • You will lose connection to the OS if you unbound NIC used for management plane (ex: ssh).

  • Confirm NIC you are planning to use for DPDK are NOT marked *Active*.

    $ cd ~/dpdk-stable-17.11.1
~/dpdk-stable-17.11.1$ ./usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s3 drv=e1000 unused= *Active*
0000:00:08.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s8 drv=e1000 unused=
0000:00:09.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s9 drv=e1000 unused=
0000:00:0a.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s10 drv=e1000 unused=

  • Load kernel modules

  • Disable NICs

  • Bind NICs to DPDK driver

    ~/dpdk-stable-17.11.1$ sudo -s
~/dpdk-stable-17.11.1# modprobe uio
~/dpdk-stable-17.11.1# insmod build/kmod/igb_uio.ko
~/dpdk-stable-17.11.1# ip link set enp0s8 down
~/dpdk-stable-17.11.1# ip link set enp0s9 down
~/dpdk-stable-17.11.1# ip link set enp0s10 down
~/dpdk-stable-17.11.1# ./usertools/dpdk-devbind.py --bind=igb_uio enp0s8 enp0s9 enp0s10

  • Confirm (only) NICs for DPDK are listed under “Network devices using DPDK-compatible driver”

    ~/dpdk-stable-17.11.1# ./usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:00:08.0 '82540EM Gigabit Ethernet Controller 100e' drv=igb_uio unused=e1000
0000:00:09.0 '82540EM Gigabit Ethernet Controller 100e' drv=igb_uio unused=e1000
0000:00:0a.0 '82540EM Gigabit Ethernet Controller 100e' drv=igb_uio unused=e1000

Network devices using kernel driver
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller 100e' if=enp0s3 drv=e1000 unused=igb_uio *Active*

4.6. Next Steps

Refer to How to install Lagopus software router for steps to install Lagopus software router.