IXDP - A libxdp wrapper API for zero-copy high speed packet processing applications

January 29, 2025

Motivation #

Linux XDP together with eBPF is an unique API which allows to process networking packets in direct communication with the kernel. But there’s some kind of a problem: Its usage, setup and correct handling of the 4 rings per queue and interface is very cumbersome and error-prone.

IXDP is an abstraction layer on top of libxdp which makes implementing networking applications very easy and clear. Its properties are:

• No specific eBPF programming (the default libxdp eBPF program xsk_def_xdp_prog.c is automatically in place by libxdp).
• interfaces are setup in promiscuous mode and taken completely out of their connection to the Linux IP stack.
• zero-copy operation only (suitable devices and drivers required).
• additional ability to copy packets or to inject arbitrary packets for delivery to the cable on a specific interface.

AF_XDP #

AF_XDP is a raw socket optimized for high performance packet processing and allows zero-copy between kernel and applications. As the socket can be used for both receiving and transmitting, it supports high performance network applications purely in user space.

libxdp #

libxdp is a lightweight library that allows attaching multiple XDP programs to network interfaces and using AF_XDP sockets. It basically exists to faciliate usage and control of the underlying AF_XDP sockets.

IXDP #

IXDP is an API wrapper from Inlab Networks on top of libxdp encapsulating all initialisation and ring processing.

The IXDP packet processing core functions are:

IXDPSetup() #

This function prepares one or multiple NICs for zero-copy packet processing.

The pramaters are:

• ifc interface count
• ifv an array of ifc interface/NIC names
• nqueues the number of queues to prepare (this equals the number of threads started by IXDPStart(a)
• ringsize the individual size of the RX and TX ring (recommended to use the pre-set maximum as the driver reports)

The function declaration is this:

1

void IXDPSetup(int ifc, char** ifv, int nqueues, int ringsize);

IXDPStart() #

This function starts the processing theads, exactly one for each configured queue as specified at IXDPSetup(). This has been separated from IXDPSetup() to allow additional actions in between (like daemonizing or other initialisation tasks).

1

void IXDPStart();

IXDPCallback() #

IXDPCallback() implements the packet processing application (as an unresolved external symbol/function within ixdp.c).

It needs to be fully reentrant since it is executed in parallel by multiple queue processing threads..

During its execution the executing thread is in possession of the packet contents, can modify the contents (located at address) and can extend or shrink the length of the packet by modifying it directly through the provided length_ptr.

The interface parameter contains the interface index where the packet has been received.

If IXDPCallback() returns -1, the packet is dropped - returning a valid interface index results in passing it to the TX ring of this interface (and queue).

The function declaration looks like this:

1

int IXDPCallback(uint8_t* address, uint32_t* length_ptr, int interface);

IXDPEnqueue() #

This function enqueues an arbitrary packet for transmission on the specified interface (index) as a copy. Calling it from within IXDPCallback() is perfectly valid.

1

void IXDPEnqueue(uint8_t* address, uint32_t length, int interface);

A simple Bridge #

The following example implements a trivial packet forwarding bridge with one single processing thread:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <ixdp.h>

int IXDPCallback(uint8_t* address, uint32_t* length_ptr, int interface) {
  if(interface == 0)
    return(1);
  else if(interface == 1)
    return(0);
  else
    return(-1);
}

int main(int argc, char** argv) {

  static int number_of_interfaces = 2;
  static char* interfaces[] = { "enp1s0f0", "enp1s0f1" };
  static int number_of_queues = 1;
  static int ringsize = 4096;

  IXDPSetup(number_of_interfaces, interfaces, number_of_queues, ringsize);
  IXDPStart();

  for(;;)
    sleep(10);

  exit(0);
}