.. _getting_started: Getting Started with Pktgen =========================== This section contains instructions on how to get up and running with `DPDK `_ and the ``pktgen`` traffic generator application. These instructions relate to setting up DPDK and ``pktgen`` on an Ubuntu desktop system. However, the should work on any recent Linux system with kernel support for hugeTLB/hugepages. System requirements ------------------- The main system requirement is that the DPDK packet processing framework is supported. The `DPDK Linux Getting Started Guide `_ has a section on the `System Requirements `_ that explains the BIOS, System and Toolchain requirements to compile and run a DPDK based application such as ``pktgen``. Ensure that your system meets those requirements before proceeding. You will also need a `DPDK supported NIC `_. The current version of ``pktgen`` was developed and tested using Ubuntu 13.10 x86_64, kernel version 3.5.0-25, on a Westmere Dual socket board running at 2.4GHz with 12GB of ram 6GB per socket. Setting up hugeTLB/hugepage support ----------------------------------- To get hugeTLB/hugepage support your Linux kernel must be at least 2.6.33 and the ``HUGETLBFS`` kernel option must be enabled. The DPDK Linux Getting Started Guide has a section on the `Use of Hugepages in the Linux Environment `_. Once you have made the required changed make sure you have HUGE TLB support in the kernel with the following commands:: $ grep -i huge /boot/config-2.6.35-24-generic CONFIG_HUGETLBFS=y CONFIG_HUGETLB_PAGE=y $ grep -i huge /proc/meminfo HugePages_Total: 128 HugePages_Free: 128 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB The values in Total and Free may be different depending on your system. You will need to edit the ``/etc/sysctl.conf`` file to setup the hugepages size:: $ sudo vi /etc/sysctl.conf Add to the bottom of the file: vm.nr_hugepages=256 You can configure the ``vm.nr_hugepages=256`` as required. In some cases making it too small will effect the performance of pktgen or cause it to terminate on startup. You will also need to edit the ``/etc/fstab`` file to mount the hugepages at startup:: $ sudo vi /etc/fstab Add to the bottom of the file: huge /mnt/huge hugetlbfs defaults 0 0 $ sudo mkdir /mnt/huge $ sudo chmod 777 /mnt/huge You should also reboot your machine as the huge pages must be setup just after boot to make sure there is enough contiguous memory for the 2MB pages. .. Note:: If you start an application that makes extensive use of hugepages, such as Eclipse or WR Workbench, before starting ``pktgen`` for the first time after reboot, ``pktgen`` may fail to load. In this case you should close the other application that is using hugepages. BIOS settings ------------- In the BIOS make sure that the HPET High Precision Event Timer is enabled. Also make sure hyper-threading is enabled. See the DPDK documentation on `enabling additional BIOS functionality `_ for more details. Terminal display ---------------- The ``pktgen`` output display requires 132 columns and about 42 lines to display correctly. The author uses an xterm of 132x42, but you can also have a larger display and maybe a bit smaller. If you are displaying more then 4-6 ports then you will need a wider display. Pktgen allows you to view a set ports via the ``page`` runtime command if they do not all fit on the screen at one time, see :ref:`commands`. Pktgen uses VT100 control codes display its output screens, which means your terminal must support VT100. It is also best to set your terminal background to black when working with the default ``pktgen`` color scheme. Get the source code ------------------- Pktgen requires the DPDK source code to build. The main ``dpdk`` and ``pktgen`` git repositories are hosted on `dpdk.org `_. The ``dpdk`` code can be cloned as follows:: git clone git://dpdk.org/dpdk # or: git clone http://dpdk.org/git/dpdk The ``pktgen`` code can be cloned as follows:: git clone git://dpdk.org/apps/pktgen-dpdk # or: git clone http://dpdk.org/git/apps/pktgen-dpdk In the instructions below the repository close directories are referred to as ``DPDKInstallDir`` and ``PktgenInstallDir``. You will also require the Linux kernel headers to allow DPDK to build its kernel modules. On Ubuntu you can install them as follows (where the version matches the kernel version):: $ sudo apt-get install linux-headers-3.5.0-32-generic DPDK can also work with a ``libpcap`` driver which is sometimes useful for testing without a real NIC or for low speed packet capture. Install the ``libpcap`` development libs using your package manage. For example:: $ sudo apt-get install libpcap-dev Build DPDK and Pktgen --------------------- Set up the environmental variables required by DPDK:: export RTE_SDK= export RTE_TARGET=x86_64-native-linux-gcc or export RTE_TARGET=x86_64-native-linuxapp-gcc # or use clang if you have it installed: export RTE_TARGET=x86_64-native-linux-clang or export RTE_TARGET=x86_64-native-linuxapp-clang Create the DPDK build tree:: $ cd $RTE_SDK $ make install T=x86_64-native-linux-gcc or $ make install T=x86_64-native-linuxapp-gcc This above command will create the `x86_64-pktgen-linux-gcc` directory in the top level of the ``$RTE_SDK`` directory. It will also build the basic DPDK libraries, kernel modules and build tree. Pktgen can then be built as follows:: $ cd $ make Setting up your environment --------------------------- In the ``PktgenInstallDir``/tools level directory there is ``run.py`` script, which should be run once per boot with the -s option to setup the ports. The same configuration file is also used to run pktgen by removing the -s option. .. Note:: The run.py script will do the sudo to root internally, which means the ``sudo`` is not required. The script contains the commands required to set up the environment:: $ cd /tools $ ./run.py -s default # setup system using the cfg/default.cfg file The run.py script is a python script and tries to configure the system to run a DPDK application. You will probably have to change the configuration files to match your system. To run pktgen with the default.cfg configuration:: $ cd /tools $ run.py default The ``run.py`` command use python data files to configure setup and run pktgen. The configuration files are located in the ``PktgenInstallDir``/cfg directory. These files allow for setup and running pktgen and can be configured to match you system or new configuration files can be created. Here is the default.cfg file:: # Setup configuration setup = { 'devices': [ '81:00.0 81:00.1 81:00.2 81:00.3', '85:00.0 85:00.1 85:00.2 85:00.3', '83:00.0' ], 'opts': [ '-b igb_uio' ] } # Run command and options run = { 'dpdk': [ '-l 1,1-5,10-13', '-n 4', '--proc-type auto', '--log-level 7', '--socket-mem 2048,2048', '--file-prefix pg' ], 'blocklist': [ #'-b 81:00.0 -b 81:00.1 -b 81:00.2 -b 81:00.3', #'-b 85:00.0 -b 85:00.1 -b 85:00.2 -b 85:00.3', '-b 81:00.0 -b 81:00.1', '-b 85:00.0 -b 85:00.1', '-b 83:00.0' ], 'pktgen': [ '-T', '-P', '--crc-strip', '-m [2:3].0', '-m [4:5].1', '-m [10:11].2', '-m [12:13].3', ], 'misc': [ '-f themes/black-yellow.theme' ] } We have two sections one for setup and the other for running pktgen. The ``modprobe uio`` command, in the setup script, loads the UIO support module into the kernel as well as loafing the igb-uio.ko module. The two echo commands, in the setup script, set up the huge pages for a two socket system. If you only have a single socket system then remove the second echo command. The last command in the script is used to display the hugepage setup. You may also wish to edit your ``.bashrc``, ``.profile`` or ``.cshrc`` files to permanently add the environment variables that you set up above:: export RTE_SDK= export RTE_TARGET=x86_64-native-linux-gcc or export RTE_TARGET=x86_64-native-linux-appgcc Running the application ----------------------- Once the above steps have been completed and the ``pktgen`` application has been compiled you can run it using the commands shown in the :ref:`running` section.