Changes in Pktgen

This section shows changes and bug fixes in the Pktgen application.

Pktgen-DPDK - Traffic Generator powered by DPDK

Pktgen-DPDK is a traffic generator powered by DPDK at wire rate traffic with 64 byte frames.

** (Pktgen) Sounds like ‘Packet-Gen’**

=== Modifications ===

• 23.06.1 - cleanup the code with clang-format.
• 23.06.0 - Fix up problem using “all” with portlist routines.
• 23.03.1 - In latency mode avg_cycles was only calculated when not using Lua.

• 23.03.0 - Update and fix a number of issues with latency. Some values were not updated

  or displayed on screen or in a Lua script when getting port stats. Latency packets are not able to be sent in any mode, instead of only being able to send and report latencies in latency mode. When using a portlist in lua it can be a single port in a string, a string range or a single port using a Lua Number or Integer

• 22.07.2 - Changed the number of queues from 16 to 64.
• 22.07.1 - Update copyright dates and other minor fixes

• 22.07.0 - Add new rx/tx burst options.

  Fix GUI build and stats. Make build with DPDK more compatible. Add max latency reporting. Allow building on RISC-V platforms. Add new rfc2455 script Increase the number of mbufs allocated to support better performance. Add restart command to try and cleanup the DPDK PMD or mbufs a start RX/TX. Update documentation contribution guidlines. Remove the -j option. Add relative delay to compute rate stats and support multple queue stats.

• 22.04.1 - Fix Lua build issues around new stats structure changes.

• 22.02.0 - Remove the backward compatability builds with older versions of DPDK.

  It has become too difficult to maintain building with older versions and I believe the older DPDK version builds did not work as expected. Please only build DPDK with the lastest version of DPDK or at least the previous version, but I do not guarantee it will work. I will try to keep up with the latest release version of DPDK.

• 21.11.0 - Fix DPDK defines being renamed of deprecated. Also rte_eth_devices was made static.

• 21.10.0 - Fix up ARP processing, change pcap files to rw-rw-rw mode, pcap capture on the correct

  numa node, increase the TX burst size, update to work with 21.11 DPDK, update the tx_failed counter, add TCP flags/seq/ack, pcap use max packet size for mempool, pcap is a multiple of pcap length, fixup AVX/AVX2 support, ipv6 updates.

• 21.03.1 - Fixup AVX and AVX2 compiler support, if enabled in meson_options.txt file.
• 21.03.0 - Fixup DPDK name changes and other minor problems.

• 21.02.0 - Cleanup display to be more compact and fix up colors

  Update the copyright notice years Fix the pcap issues with allocating memory on the wrong socket. Add thousand separators to see rates and values easier.

• 21.01.2 - Add updated IPv6 support.
• 21.01.1 - Update the version number with fix for sending special mbufs for input processing.
• 21.01.0 - fix warning message spit out by DPDK and add lua-5.4 support in build.
• 20.12.0 - merge in latency test support with jitter and a number of minor cleanups
• 20.11.3 - fix changes in DPDK 20.11-rc4
• 20.11.2 - The dynfield0 was removed from rte_mbuf header
• 20.11.1 - Fix net_bond library name change for older DPDK versions.

• 20.11.0 - Fixup build to DPDK PMD name changes (bonding driver)

  Fix the macro and function API name changes in DPDK 20.11

• 20.10.1 - Add option to disable avx2 support.
• 20.10.0 - fix ENA memory allocation and add setting rx/tx descriptor rings from cmdline.

• 20.09.0 - Fix the clearing of the port stats.

  Fix the lua range commands to be a table only. Use pktgen.range.XYZ() to set range values. Removed the use of pktgen.XYZ() type commands and require the range table to be used.

• 20.08.0 - Add better latency support with saving the results to a file.

  Updated the INSTALL.md to help building DPDK and Pktgen. Number of minor bugs or cleanup.

• 20.07.0 - Fix lua/gui builds to find the lua lib/headers on ubuntu 20.04

  Update the INSTALL text to locate DPDK pkg-config, this also fixes Lua (I hope in all cases)

• 20.06.0 - Convert to using meson/ninja only builds.

  Fix the default mbuf size. Fix sending of special or ARP packets. Add Makefile to use tools/pktgen-build.sh script for meson/ninja Fix up building the docs using meson/ninja. New target directory for install is $PKTGEN_DIR/usr/local, need to build and install DPDK using meson/ninja.

• 20.04.0 - Disable Lua by default in meson_options.tst file for meson/ninja builds.

  Lua support is disabled by default now as we move to meson/ninja builds only. When building with meson/ninja DPDK needs to be built meson/ninja and installed.

• 20.03.0 - Fix a typo in run.py and add af_xdp config file.

• 20.02.0 - Fix link variable to be lk to not conflect with global link struct.

  fix generating sequences with vlan header fix sequence help message

• 19.12.0 - Add per queue stats when the hardware does not support it. Use

  the ‘page stats’ command to see data and use ‘port X’ to switch ports.

• 19.10.0 - Fix up the save/load commands to get correct as in GRE key

  command Fix up the problem with pcap files not getting sent if the number of packets was small fix up a crash if the scrn pointer was invalid on startup fixed the log output in startup Fixed the save/load scripts to use the correct commands. Fix the IPv4/6 checksum calculations

• 19.08.0 - Fix linking of Lua library when no pkg-config file is found and

  linking liblua.a Fixed and issue with packet rate not being changed when the packet size is changed Change version numering to year.month.patch format Change lua pktgen.set_mac() to have three args set_mac(<portlist>, ‘src|dst’, mac_addr) instead of two args add support for setting via lua src and dst mac addresses fix up the RSS port configuration options updated code to adjust the packet tx interval based on command changes on the packet size and other areas.

• 3.7.1 - Add TTL support to single and range modes. Better docs for dump packets

  • Pktgen 3.7.1 will build with DPDK 18.02 to 19.08-rc2, but 18.08 has a problem with vhost.h VRING_EVENT_F_AVAIL not defined and appears the Ubuntu 19.04 version I am running does not define it correctly.
  • fixed the RX side using TX count for rx_burst command.
  • General cleanup

• 3.7.0 - Fixed build issues with DPDK 19.08 as DPDK renamed a lot of defines

  Fixed up the meson files to build pktgen with meson and ninja Minor cleanup

• 3.6.6 - Add portInfo() function in lua to return most of the port info in

  one structure.

• 3.6.5 - Fix ldflags order for RHEL, CentOS and other systems to include lua in build

• 3.6.4 - Add a new page stats screen to dump out info

  Updated the readme to give better info on .cfg files Fixed the port <N> to include only valid ports fix port command functions to use uint16_t for ports. fix detection of Lua libs testing LUA_PKT_LIBS for Ubuntu 18.10

• 3.6.3 - Fix race condition in start command

  Add support of having a different PCAP file per queue of a port Replace constant with macro for PCAP buffer size Expose number missed frames to Lua stats. Fix cleanup of unsent pkts, this fixes duplicate packets being sent.

• 3.6.2 - Fix GRE header pointer being NULL and fixed IPPROTO_UDP to use PG_IPPROTO_UDP
• 3.6.1 - Fix string truncation warnings for latest GCC

• 3.6.0 - Fixed config files.

  • Fixed tx stopping on some platforms.
  • Updated the copyrights to 2019
  • Add verbose flag to cleanup startup text.
  • Adding support for VxLAN

• 3.5.9 - Fixed run.py to find config files better.

  • Limit the pcap packets to the number of mbufs used.
  • Added new ‘page xstats’ screen to view extended stats for ports.
  • other minor patches

• 3.5.8 - Add code to check for using initial lcore for ports
• 3.5.7 - Fix CentOS building with Lua, fixed cli building as was not linking cli
• 3.5.6 - Minor fixes to cleanup builds sync cli,vec,utils and lua code bases.
• 3.5.5 - Minor fixes to run pktgen without libs in DPDK.
• 3.5.4 - Fix possible mbuf leak and clean up colors for pages.

• 3.5.3 - Fix performance problem on FVL NIC with losing mbufs.

  Number of little cleanups and fixes.

• 3.5.2 - Fix 18.08-rc0 fixed some fields that were removed txq_flags and .ignore_offload_fields.

  Tested backward compat to 18.02 at least. Pktgen should work with at least 18.02 release.

• 3.5.1 - Fix 18.05 related problems and minor cleanup in port configuration.

• 3.5.0 - fix compiler warning and range size increment command.

  fixes for working with dpdk 18.04 release.

• 3.4.9 - fix PRNG random number code to work correctly.

  Add more validation of -m commands Fix the rte_pci_bus.h header include problems.

• 3.4.8 - Increase the TX retry count for VMs and reset mbuf as virtio modifies the data.
• 3.4.7 - Command file can now use ‘quit’ to exit and Lua scripts use pktgen.quit()
• 3.4.6 - Fixed low performance in a VM with virtio and a TX hang.
• 3.4.5 - Fixed a few backward compact issues and synced CLI code
• 3.4.4 - Fixed the long line problem in CLI and improved the performance.
• 3.4.3 - Fix compile problem with DPKD 17.11 and a number of bugs.
• 3.4.2 - Fix a build problem with older DPDK versions.
• 3.4.1 - Fix ARP packet processing and a fe minor cfg file changes.
• 3.4.0 - Fix bonding polling and other minor changes.

• 3.3.9 - PCI Whitelist and Blacklist should be the original defines and

  does not require a ifdef for the version. Convert the run.py to dpdk-run.py script and convert the scripts to /bin/sh.

• 3.3.8 - reverse change causing TX to stop sending traffic.
• 3.3.7 - Fix BLOCKLIST and WHITELIST macro change for 17.08 DPDK

• 3.3.6 - Add flag to enable bonding PMD to do TX calls with zero packets.

  New command ‘enable|disable bonding’ Cleanup some of the copyright dates to use 2018

• 3.3.5 - fix sequeuce command be truncated and fix run.py to fix strings in cfg file.
• 3.3.4 - Minor fixes for help and run.py scipt.
• 3.3.3 - Make sure the mbuf data size is at least 2K in size
• 3.3.2 - Fix set pkt types and fix setting the pkt size for IPv6
• 3.3.1 - Update the help for range and add compact commands for range
• 3.3.0 - Add new run configs and run.py script to setup and run pktgen
• 3.2.12- Rework the src/dst IP address again and fix the reset command
• 3.2.11- Fixed location of libs for per v17.05 releases

• 3.2.10- Fixed the problem with set <portlist> src/dst ip <addr> needing /XX for subnet

  mask values. Add a test for using the initial lcore for a port and error out.

• 3.2.9 - fixed a number of problems and some code cleanup

  Fixed the problem when running a lua script the ports stats were not updated. The problem is a rte_timer_manage() call was remove my mistake.

• 3.2.8 - fixed ‘set <portlist> type ipv4|ipv6’ used ip4|ip6 instead.
• 3.2.7 - Fix pktgen.seq() lua function Ethertype and protocol were swapped doc is correct.

• 3.2.6 - Fix the pcap page not displaying.

  Fix standalone builds to include Lua headers.

• 3.2.5 - Fix setting of the seq_cnt in lua. Now you can use seqCnt or seq_cnt strings

  for setting the sequeuce count value.

• 3.2.4 - Fix setting of MAC and IP addresses in single mode.

• 3.2.3 - Allow pktgen to build on DPDK 14.04

  fix the new commands to use the correct syntax

• 3.2.2 - fix the clear stats command.

  fix range commands to match help text.

• 3.2.1 - Update the readme file.

• 3.2.0 - Add support for decimal point rate values, like 10.1, 20.54, 90.6, …

  Convert over to use CLI interface instead of cmdline interface. CLI is a directory like command line tool and please read the .rst file in the lib/cli directory Many bug fixes.

• 3.1.2 - Convert spaces to tabs and add pktgen-cfg.[ch] page

  Converting the spaces to tabs to allow for editing the code with tabs set to 4 or 8 columns per tab. Changed to allow people who are stuck on a tab=8 columns. We have modern computers tabs=4 columns seems reasonable.

• 3.1.1 - Minor cleanup of top level directory and code.

• 3.1.0 - Rename functions and files.

  When files are written change the file modes to 0666 as they are owned by root. More general cleanup of the display refresh. loading command or lua files is faster because of the screen updates fixed.

• 3.0.17- Fixed a formating issue on sequence page for port numbers.

  Save of lua code wrong for pktgen.range() should be pktgen.set_range().

• 3.0.16- Add command line option to strip CRC in hardware one RX.

  Option is ‘–crc-strip’ which strips the CRC on RX for all ports.

• 3.0.15- Update Lua to 5.3.3 version187

  Change lua pktgen.range() to pktgen.set_range() plus added the range commands from pktgen.dst_mac() to pktgen.range.dst_mac(). Still support the old commands for now. Now polls the RX and TX queues to support eth_bond interfaces using mode 4 or LACP.

• 3.0.14- Fix crash in saving configuration and random is not setup.
• 3.0.13- Fix seq only sending the first sequence packet and some cleanup.
• 3.0.12- Map port/queue pair stats to the correct lcore.
• 3.0.11- Fix compile problem with 16.04

• 3.0.10- Added the ‘pdump <portlist>’ command to hex dump the first packet to be

  send on the given ports. Only the single packet mode is supported.

• 3.0.09- Add Fix for PCAP corruption.

• 3.0.08- Add Lua support for rnd and latency commands.

  Now if latency is enable on a port then getting the stats will get the latency values via lua table.

• 3.0.07- Fixed crash on exit when using more then one core per port.

• 3.0.06- Fix PCI null pointer when using virtual devices.

  Removed the C11 constructs to compile on C99 compilers. Fix the bug of old packets after changes for new run. The problem is DPDK holds on to the mbufs in the TX done queue and those can not be changed. With 16.07 we can find all of the mbufs and changed them to correct format/sizes.

• 3.0.05- New Latency/Jitter page ‘page latency’

  Need to adjust the packet size to 96 to allow for latency timestamp. type: page latency latency 0 on set 0 size 96 start 0

• 3.0.04- Display reported the wrong rate in Mbits because the counters were not

  including the FCS bytes in the value as it was before. Minor cleanup of the code for formating.

• 3.0.03- General clean up of scripts

  Add support for share library builds Clean up formating Add PCI info screen GUI 1.0 support

• 3.0.02- Fix up the IPv6 address macros for musl libc support

  Fix the missing pthread.h include in lua socket header. Add the rnd lua support APIs Fix the checksum issue with rnd changes.

• 3.0.01- Fixed the Range sequence and VLAN problem.
• 3.0.00- Fixed code to ifdef the imcasts counters that were deprecated.
• 2.9.18- Fix the range command to set IP proto value and be able to save that value.
• 2.9.17- Fix PCAP crash when using multiple tx queues.

• 2.9.16- Fix include problems with cmdline_parse.h file.

  missing cmdline_parse_token_hdr_t define and looks like the header was not included in the pktgen-seq.h file for DPDK v2.2.0 and pktgen-2.9.15

• 2.9.15- Update Lua to version 5.3.2
• 2.9.14- Fix compiler error for gcc-4.9 and inet_ntop() routine
• 2.9.13- Add max last seen RX/TX packets to display.

• 2.9.12- Was not able to set IP protocol type for range packets.

  New command range.proto <portlist> udp|tcp|icmp Lua command is pktgen.ip_proto(“all”, “udp”)

• 2.9.11- Fix version string for new version style in DPDK.
• 2.9.10- Reformat the code and get ready for the next release of DPDK.
• 2.9.9 - Update the init screen output to not screw up DPDK screen output.
• 2.9.8 - Fixed the crash when using the sequence packets.

• 2.9.7 - Changed all rte_zmalloc to rte_zmalloc_socket calls and change seq_pkt support.

  The fix for multiple cores accessing seq_pkts was to allocate memory and copy into a private area. This sometimes caused memory allocation problems, so I removed the allocation and used spinlocks around the code. Most likely slower in some areas but better then allocating memory.

• 2.9.6 - Add support for different pattern types and a user patten string.

  New commands are ‘pattern <portlist> type’. Types are abc, none, zero or user New command ‘user.pattern <portlist> <string>’ The string can not contain a space which is a limitation of the rte_cmdline code. Added new Lua command for the above ‘pattern(<portlist>, <type>)’ and ‘userPatten(<portlist>, <string>)’

• 2.9.5 - Fixed sequence packet race condition for multiple senders.
• 2.9.4 - Fixed the ARP sends were not being flushed

• 2.9.3 - Remove change log and comment out the eth stop when done sending.

  This will most likely screw up the pcap and others, but stopping the port is not good.

• 2.9.1 - Fix up the sequeue help to remove vlan option with ipv4/ipv6
• 2.9.0 - Update to DPDK 2.0.0 and Lua 5.3.0 with README update.
• 2.8.6 - Fix argument for rte_mempool_create, which caused a crash.
• 2.8.5 - Fix compat problem with latest Pktgen and DPDK 1.8.0
• 2.8.4 - Minor updates for comments.

• 2.8.3 - Updated the Makefiles to use rte.extXYZ.mk files.

  Updated the code to build with DPDK 2.0.0-rc1 as some function prototype changed.

• 2.8.2 - Fix bug in pktgen_main_receive routine not using the correct port number.
• 2.8.1 - Add a new docs directory using Sphinx format and update version numbers.
• 2.8.0 - Update to release 1.8.0 of DPDK.

• 2.7.7 - Update Lua to 5.2.3 and fixed setting vlan ID on single ports plus added new Lua functions

  New Lua functions are pktgen.portCount() and pktgen.totalPorts() portCount() is the number of port used by Pktgen and totalPorts() is the total number seen by DPDK.

• 2.7.6 - Update code from dpdk.org version of Pktgen, which hopefull fixes the send foreve problem.
• 2.7.5 - Update to latest dpdk.org and move scrn to lib directory with name changes.
• 2.7.4 - Removed old printf_info() calls for printf_status() calls.
• 2.7.3 - Fixed race condition with updating the TX count value with a small count.
• 2.7.1 - Add a command line option ‘-T’ to enable themes and set themes off by default.

• 2.7.0 - Update to DPDK 1.7.0, Note: DPDK 1.7.0 changed how ports are detected and blocklisted,

  which means the port index is now different. You will need to blocklist or allowlist ports with the DPDK ‘-b’ or ‘–pci-blocklist or –pci-allowlist’ options. Pktgen does not blocklist ports anymore. - Moved pktgen to the examples directory plus removed the libwr_* from the lib directory - Pktgen now supports ANSI color terminals only the main screen ATM, but more later. - Best viewed on a black background display, unless you want to change it with the new theme commands. - More supported generator types, checkout the help screens.

• 2.6.8 - Fixed a transmit problem when count is set to one. Plus increase the link down delays.
• 2.6.7 - Add more support for GRE packets, log support and more testing code.

• 2.6.6 - Fix compile problem when not SSE4.2 instructions are not supported. Allowing QEMU and other

  systems to build and run. Also added a patch to take into account huge reserved pages.

• 2.6.5 - Added support for logging packet information.

• 2.6.4 - It consists of 3 commits: improvements to the pktgen-random.c unit tests,

  the real CentOS compilation fixes and a small update to tap.{c,h} so they are identical to those from zorgnax/libtap on github.

• 2.6.3 - Add a delay when traffic stops to make sure all packets are sent.

  Remove the rte_hash_crc.h include in wr_pcap.c file.

• 2.6.2 - Fixup GRE and ARP problems
• 2.6.1 - Add random bits support and more cleanup
• 2.6.0 - Split up the code for testing to be added later
• 2.5.2 - Remove extra ethertypes.h file.

• 2.5.1 - Added the following updates.

  • Implement-Rx-packet-dump-functionality
  • Add-packet-capture-functionality
  • Add-MPLS-functionality
  • Add-Q-in-Q-802.11ad-functionality
  • Add-GRE-header-generation
  • Fix-UDP-TCP-ICMP-protocol-selection
  • Add-ARP-protocol

• 2.5.0 - Update to DPDK 1.6.0 plus a few bug fixes.
• 2.4.1 - Fixed a bug in range packets when ‘inc’ value is zero use start values.
• 2.4.0 - Add support for TX tap packets. Change ‘tap’ command to rxtap and txtap.
• 2.3.4 - Minor update to help eliminate RX errors and be able to receive at wire rate.
• 2.3.3 - Update to minor release 1.5.2
• 2.3.2 - Fixed VLAN detection problem in ARP and special GARP support.
• 2.3.1 - Getting closer to line rate tx speed.

• 2.3.0 - Now supports the VLAN encapsulated packets for ARP replies

  Also added a special GARP processing to update the destination MAC address to help support a special request for failover support. Added support for DPDK 1.5.1

• 2.2.7 - Updated the code to handle multiple TX queues per port.
• 2.2.6 - Fixed a crash if the port is not up with link status
• 2.2.5 - Remove the flow control code as some systems it does not work.
• 2.2.4 - Fix the inet_h64tom and inet_mtoh64 functions to account for endianness
• 2.2.3 - range packet fixes for packet size and source/destination mac
• 2.2.2 - Minor performance changes for receive packet performance.

• 2.2.1 - Change MAC address from XXXX:XXXX:XXXX to XX:XX:XX:XX:XX:XX format

  Fixed Pktgen to allow packet changes without having to restart the tool.

• 2.2.0 - Update to DPDK 1.5.0

=====================