IXDP Packet Flow explained
 
 

IXDP Packet Flow explained

April 19, 2025
development, configuration, projects
EtherTAP, bngx, IXDP, libxdp, RPCL

IXDP is a layer above libxdp/AF_XDP which is responsible for XDP ring management operating with a set of interfaces in promiscuous mode. IXDP registers its specific set of RPCL functions (words) which allow to control and configure the IXDP packet processing layer.

The IXDP Packet Flow Diagram #

The following diagram gives an overview how a single Ethernet packet is processed within IXDP. The whole chain is executed my multiple threads in parallel (if configured), each thread operating on its own queue.

flowchart TD a("1. packet from interface received") --> b("2. update MAC and Ethertype statistics") --> c("3. insert packet into TAP queue (if qualified)") --> d("4. user callback") d --> |drop by callback or unknown interface| d1("10. discard packet") --> d2[/"done"/] d --> |valid output interface| d3("5. check drop machash") d3 --> |drop by drop machash| d1("10. discard packet") d3 --> d4("6. fault injection (if qualified)") d4 --> |drop by fault injection| d1("10. discard packet") d4 --> d5("7. schedule packet for TX (with TAP queue insert, if qualified)") --> d2(["done (continue busy-polling from top)"])

The particular Steps explained #

1. Packet from Interface received #

This is triggered by an available packet in the XDP RX ring.

  • if.stats

    show summary statistics of all interfaces ( - )

  • if.qstats

    show detailed queue/interface statistics ( queue interface - )

2. Update MAC and Ethertype Statistics #

At this stage the MAC and Ethertype statistics are updated, this operates lockless with atomic uint64_t counters and lockless data structures. The Ethertype statistics also record the last Ethernet MAC address that has been seen sending a packet with each specific Ethertype and VLAN level.

  • if.etype.show

    show rx Ethertype and VLAN statistics for interface ( i - )

  • if.mac.show

    show rx mac statistics for interface ( i - )

3. Insert Packet into TAP Queue (if qualified) #

The TAP packet queue is a multi-producer/multi-consumer type of queue held in shared memory with the name ixdp.pqueue.tap.

A packet qualifies to be sent to the TAP packet queue if the following conditions are met:

  • RX TAP is enabled on the receiving interface
  • The TAP machash is empty (all packets qualify)
  • The source or destination address is in the set defined by the TAP machash

  • tap.hash.insert

    add specific MAC address to TAP ( mac - )

  • tap.hash.reset

    reset TAP machash to 0 elements ( - )

  • tap.hash.show

    show TAP machash ( - )

  • tap.rx.disable

    disable RX TAP for interface ( interface - )

  • tap.rx.enable

    enable RX TAP for interface ( interface - )

  • tap.status

    show TAP packet queue status ( - )

  • tap.tx.disable

    disable TX TAP for interface ( interface - )

  • tap.tx.enable

    enable TX TAP for interface ( interface - )

4. User Callback #

The user callback function may perform its own packet modifications and eventually either determines the outgoing interface or instructs the packet to be dropped. For an example see IXDP - A libxdp wrapper API for zero-copy high speed packet processing applications.

5. Check Drop Machash #

A general drop machash allows to drop speficic packets. A packet where the source or destination mac address is contained in this drop machash is discarded.

  • drop.hash.insert

    add specific MAC address to drop ( mac - )

  • drop.hash.reset

    reset drop machash to 0 elements ( - )

  • drop.hash.show

    show user defined drop machash ( - )

6. Fault Injection (if qualified) #

A packet qualifies for fault injection if:

  • The interface is enabled for fault injection (on RX)
  • The SRC MAC machash is empty or the SRC address is member of the SRC machash
  • The DST MAC machash is empty or the DST address is member of the DST machash
All four fault injection probabilities are specified in 1/1000 units, their initial value is 0 (0.000 equals “impossible” or “never”). It may happen, that a packet is corrupted differently multiple times (however, drop corruption is evaluated first).

  • fault.disable

    disable fault injection for interface ( interface - )

  • fault.enable

    enable fault injection for interface ( interface - )

  • fault.hash.dst.insert

    add specific MAC address to fault injection DST machash ( mac - )

  • fault.hash.dst.reset

    reset fault injection DST machash to 0 elements ( - )

  • fault.hash.dst.show

    show fault injection DST machash ( - )

  • fault.hash.src.insert

    add specific MAC address to fault injection SRC machash ( mac - )

  • fault.hash.src.reset

    reset fault injection SRC machash to 0 elements ( - )

  • fault.hash.src.show

    show fault injection SRC machash ( - )

  • fault.drop.propability

    set the fault injection drop propability ( p - )

  • fault.corrupt.amount

    specify the number of bytes to invert by XOR 0xff ( p - )

    The byte inversion occurs at ramdom positions starting just after the first Ethertype.

  • fault.corrupt.propability

    set the fault injection packet corruption propability ( p - )

  • fault.grow.amount

    specify the absolute number of bytes to grow ( n - )

    The bytes added here contain random data.

  • fault.grow.propability

    set the fault injection packet growth propability ( p - )

  • fault.shrink.amount

    specify the absolute number of bytes to be cut off ( n - )

  • fault.shrink.propability

    set the fault injection packet shortening propability ( p - )

  • fault.status

    show fault injection status and counters ( - )

7. Schedule Packet for TX #

The packet is scheduled for TX on the interface the user callback has determined.

The final TX stage inserts a copy of the packet into the TAP queue ixdp.pqueue.tap if:

  • TX TAP is enabled for the output interface
  • The TAP machash is empty (all packets qualify)
  • The source or destination address is in the set defined by the TAP machash
This TAP queue insertion here therefore also allows to inspect packets that have been corrupted by fault injection.

10. Discard Packet #

The packet is discarded and the XDP descriptors are scheduled for later reuse.