libxdp/IXDP - comments and resources

March 4, 2025

development

IXDP, libxdp

Overview #

This is currently just a brief collection.

libxdp #

https://github.com/xdp-project/xdp-tools/tree/main/lib/libxdp

The xsk_def_prog default eBPF program #

https://github.com/xdp-project/xdp-tools/blob/main/lib/libxdp/xsk_def_xdp_prog.c

xdp-loader status #

root@u32:~# xdp-loader status
CURRENT XDP PROGRAM STATUS:

Interface        Prio  Program name      Mode     ID   Tag               Chain actions
--------------------------------------------------------------------------------------
lo                     <No XDP program loaded!>
enp3s0                 <No XDP program loaded!>
enp1s0f0               xdp_dispatcher    native   22   90f686eb86991928 
 =>              20     xsk_def_prog              31   8f9c40757cb0a6a2  XDP_PASS
enp1s0f1               xdp_dispatcher    native   37   90f686eb86991928 
 =>              20     xsk_def_prog              40   8f9c40757cb0a6a2  XDP_PASS

root@u32:~#

AF_XDP socket multi-threaded forwarding sample application #

https://github.com/xdp-project/bpf-examples/tree/main/AF_XDP-forwarding

This example program is really worth to study, which may take some considerable time. At several places it may look deliberately cryptic and even misleading, but rather with the spirit of an humourous brain-teasing puzzle. Eventually, after having mastered this, you will probably end up implementing your own ring and UMEM management.

Normal warning messages that can be ignored #

Whe using the default program, the following logs are shown and can be ignored (see https://doc.dpdk.org/guides/nics/af_xdp.html - DPDK XDP poll mode driver):

libbpf: elf: skipping unrecognized data section(8) .xdp_run_config
libbpf: elf: skipping unrecognized data section(9) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(8) .xdp_run_config
libbpf: elf: skipping unrecognized data section(9) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata

XDP_STATISTICS getsockopt #

https://www.kernel.org/doc/html/v5.18/networking/af_xdp.html documents the following retrievable XDP statistics:

struct xdp_statistics {
    __u64 rx_dropped; /* Dropped for reasons other than invalid desc */
    __u64 rx_invalid_descs; /* Dropped due to invalid descriptor */
    __u64 tx_invalid_descs; /* Dropped due to invalid descriptor */
};

OTOH, a recent /usr/include/linux/if_xdp.h contains this extended structure:

struct xdp_statistics {
  __u64 rx_dropped; /* Dropped for other reasons */
  __u64 rx_invalid_descs; /* Dropped due to invalid descriptor */
  __u64 tx_invalid_descs; /* Dropped due to invalid descriptor */
  __u64 rx_ring_full; /* Dropped due to rx ring being full */
  __u64 rx_fill_ring_empty_descs; /* Failed to retrieve item from fill ring */
  __u64 tx_ring_empty_descs; /* Failed to retrieve item from tx ring */
};

The two entries rx_fill_ring_empty_descs and tx_ring_empty_descs do not provide any useful information and are wildly counting upwards (despite the management of the rings is correct). For the time being, we just ignore these values. rx_ring_full stays 0 in our testing environment.