EtherTAP - Basic Usage

April 28, 2025

EtherTAP, IXDP, AF_XDP, libxdp, RPCL

EtherTAP #

EtherTAP is a software based Ethernet analysis and monitoring security tool which allows to obtain insight and to record selected (or all) traffic for further analysis and evaluation. EtherTAP operates with IXDP, which is an internal API at Inlab Networks on top of libxdp/AF_XDP.

XDP/AF_XDP in zero-copy mode allows packet processing at wire-speed up to 100GbE full duplex.

The Layers #

The EtherTAP application consists - somewhat simplified - of three layers:

flowchart TD a("EtherTAP") --- b("IXDP") --- c("libxdp/eBPF/NIC drivers")

EtherTAP #

The top layer, EtherTAP consists basically of command line option parsing, some RPCL initial processing (as loading /etc/ethertap.rpcl with load-if-present and inserting some variables like version.ethertap).

The functionality of EtherTAP itself is represented by the IXDP callback function IXDPCallback() and is simple and short:

1
2
3
4
5
6
7
8


int IXDPCallback(uint8_t* address, uint32_t* length_ptr, int interface) {
  if (interface == 0)
    return (1);
  else if (interface == 1)
    return (0);
  else
    return (-1);
}

IXDP #

The IXDP layer is responsible for XDP ring management operating with a set of interfaces in promiscuous mode. IXDP registers its specific set of RPCL functions (words) which allow to control and configure the IXDP packet processing layer.

All RPCL control mechanisms that are available with EtherTAP are in fact part of the IXDP layer.

Further information:

libxdp/eBPF/NIC drivers #

libxdp is a lightweight library that allows attaching multiple XDP programs to network interfaces and using AF_XDP sockets. It relies on libbpf to process eBPF object files and supports freplace and kernel features for the XDP dispatcher.

The `ethertap` Command #

The general usage information of the EtherTAP command is this:

$ ethertap -h 
usage: ethertap [options] interface0 [interface1]
       options:
       -q|--queues <q>         specify number of queues (default 1)
       -r|--ringsize <r>       specify ringsize (default 4096)
       -i|--initfile <file>    load specific initial RPCL file
       -d|--daemon             detach and run in background
       -h|--help               show usage
       -v|--version            show version
       -c|--copyright          show copyright
$ ▂

EtherTAP Options #

-q|–queues #

This option controls the number of queues that the IXDP layer operates withi over all interfaces. This parameter is equivalent to the number of threads, since IXDP runs with one thread per queue. The default is 1, which is fully sufficient to process packets bridging between two 10GbE interfaces.

-r|–ringsize #

The ringsize defaults to 4096 and can be changed with this option to another value. The NIC ring parameters are set to this value for RX and TX, also this value influences the IXDP ring and FIFO buffer dimensioning. There’s currently no NIC driver known which requires a different value, but it might be important for 40GbE and 100GbE NICs.

-i|–initfile #

Per default EtherTAP loads /etc/ethertap.rpcl with load-if-present. A different RPCL init file can be specified with this option if desired.

-d|–daemon #

With this option specified, EtherTAP detaches itself and runs in background, registering with ethertap as identifier for the control utility. When this option is not specified, the ethertap command ends in its RPCL main loop (with no line editing capabilities).

EtherTAP Modes #

Bridging Mode #

When EtherTap is invoked with two interfaces, it runs in bridging mode, acting as a promiscuous mode bridge between those two interfaces.

Example:

$ sudo ethertap enp1s0f0 enp1s0f1 --daemon 
ethertap going to background ...
$ ▂

Mirroring Mode #

When EtherTAP is invoked with just one interface, it runs in mirroring mode (sometimes this is also called “SPAN” by other software and vendors). When the provided interface is connected to a switch port, that has been configured for port mirroring it allows to inspect the traffic visible this way. EtherTAP’s fault injection is not able to inject faults in this mode.

Example:

$ sudo ethertap enp1s0f0 --daemon 
ethertap going to background ...
$ ▂

Control #

$ control -h 
usage: control <identifier>           control background process
       control <identifier> status    show background process status
       control <identifier> stop      terminate background process
       control -h|--help              show usage
       control -v|--version           show version
       control -c|--copyright         show copyright
$ ▂

Control Tasks #

control background process #

Invoking control with an identifier connects to a running background daemon which has registered itself with that identifier name. As soon as connected, stdin/stdout/stderr of the background process are connected to the control frontend which offers some line editing capabilities.

Any output that has been sent to stdout and stderr by the background process on startup is shown first as it would have appeared on screen.

Since EtherTAP runs RPCL as its main thread, this connects to the RPCL main loop indicated by the ok prompt.

Ctrl-D and Ctrl-C exit the control frontend, additionally there’s and inactivity timeout of 10 minutes.

Example:

$ sudo ethertap enp1s0f0 enp1s0f1 --daemon 
ethertap going to background ...
$ sudo control ethertap
connected to ethertap PID 6617
IXDPSetup() SUCCESS ifc:2 nqueues:1 ringsize:4096
IXDPStart() SUCCESS
ok ▂

show background process status #

This shows the background status and PID (if there’s a background process running with the specified identifier).

Example:

$ sudo control ethertap status
ethertap PID 6617 running
$ sudo control unknown status
unknown not running
$ ▂

terminate background process #

This allows to shutdown a running background process with increasing “force” in terms of signals.

Example:

$ sudo control ethertap stop 
shutdown of ethertap PID 6617 complete
$ sudo control unknown stop 
unknown not running
$ ▂

EtherTAP and RPCL #

Show Uptime #

The RPCL built-in word uptime-hr allows to display the uptime of the RPCL interpreter in a “human readable” format. With EtherTAP running in background, this looks like this (effectively displaying the uptime of the EtherTAP background process):

$ sudo control ethertap 
ok uptime-hr .
1d 22h 24m 52s
ok ▂

Show RPCL Variables #

Example: EtherTAP in Bridging Mode #

$ sudo control ethertap 
ok .d 
ixdp.configured_interfaces       2
ixdp.configured_queues           1
ixdp.configured_ringsize         4096
ixdp.interface.0.driver          ixgbe
ixdp.interface.0.name            enp1s0f0
ixdp.interface.1.driver          ixgbe
ixdp.interface.1.name            enp1s0f1
platform                         Debian12-amd64
version.ethertap                 1.39
version.ixdp                     1.64
version.rpcl                     1.75
ok ▂

Example: EtherTAP in Mirroring Mode #

$ sudo control ethertap 
ok .d 
ixdp.configured_interfaces       1
ixdp.configured_queues           1
ixdp.configured_ringsize         4096
ixdp.interface.0.driver          ixgbe
ixdp.interface.0.name            enp1s0f0
platform                         Debian12-amd64
version.ethertap                 1.39
version.ixdp                     1.64
version.rpcl                     1.75
ok ▂

The Drop Machash #

IXDP supports lockless hashtables of MAC addresses, defining just a set of MAC addresses (a specific MAC address is either a member of such a set - or not). One instance of this is the “Drop Machash”. which defines a set of MAC addresses that cause a packet to be dropped when either the source MAC address or the destination MAC address of an Ethernet packet is a member of the Drop Machash.

This question (member of the Drop Machash or not) is asked when the IXDP callback function decides to forward the packet to another or the same interface it has been received from. With EtherTAP, this functionality makes therefore only sense in bridging mode, where EtherTAP acts as a bridge between two interfaces.

`drop.hash.insert` #

add specific MAC address to drop ( mac - )

Example:

$ sudo control ethertap 
ok a0:36:9f:21:4b:90 drop.hash.insert 
ok f0:b2:b9:15:07:ca drop.hash.insert 
ok drop.hash.show
a0:36:9f:21:4b:90 Intel Corporate
f0:b2:b9:15:07:ca Intel Corporate
ok ▂

To specify addresses always to be dropped, just include the desired drop.hash.insert commands in /etc/ethertap.rpcl.

`drop.hash.reset` #

reset drop machash to 0 elements ( - )

This clears the drop machash.

Example:

$ sudo control ethertap 
ok drop.hash.show
a0:36:9f:21:4b:90 Intel Corporate
f0:b2:b9:15:07:ca Intel Corporate
ok drop.hash.reset 
ok drop.hash.show
ok ▂

`drop.hash.show` #

show user defined drop machash ( - )

Example:

$ sudo control ethertap 
ok drop.hash.show
a0:36:9f:21:4b:90 Intel Corporate
f0:b2:b9:15:07:ca Intel Corporate
ok ▂

Interface Statistics and Counters #

`if.drivers` #

show interfaces and drivers ( - )

Example:

ok if.drivers 
0 enp1s0f0         ixgbe
1 enp1s0f1         ixgbe
ok ▂

Here EtherTAP is operating in bridging mode since it has been started with two interfaces.

`if.etype.show` #

show rx Ethertype and VLAN statistics for interface ( i - )

Example:

ok if.drivers
0 enp2s0           igc
1 enp3s0           igc
ok 0 if.etype.show 
0 0x0008 802.3 size       e8:fd:f8:5d:fe:4a                    1 Shanghai High-Flying Electronics Technology Co., Ltd
0 0x0800 IPv4             32:3a:fd:84:56:64                12866 Locally Administered Address (LAA)
0 0x0806 ARP              00:e2:69:89:51:e4                  935
0 0x8100 802.1q           00:3e:e1:c0:f0:9c                   24 Apple, Inc.
      10 VID              00:3e:e1:c0:f0:9c                   24 Apple, Inc.
0 0x86dd IPv6             00:3e:e1:c0:f0:9c                 2881 Apple, Inc.
0 0x8899 RRCP             24:d7:9c:e4:3e:e9                  700 Cisco Systems, Inc
0 0x88cc LLDP             56:e6:36:5f:17:a8                   92 Locally Administered Address (LAA)
1 0x0800 IPv4             00:3e:e1:c0:f0:9c                   12 Apple, Inc.
1 0x86dd IPv6             00:3e:e1:c0:f0:9c                   12 Apple, Inc.
ok ▂

There are six columns in a if.etype.show output:

The first column is the VLAN level, where IXDP descends up to 4 levels for 0x8100 and 0x88a8 type of VLANs.

The second column is the Ethertype that has been seen (or the VLAN Id (VID) for 0x8100 and 0x88a8).

The third column is a textual desciption of the Ethertype.

The fourth column is the Ethernet address that has most recently encountered sending a packet. This may be helpful to determine MAC addresses that are communicating in wrong or unexpected VLANs.

The fifth column is the atomic counter counting the occurences of packets of that type.

The sixth column is the manufacturer information of the MAC address in column four.

`if.mac.show` #

show rx mac statistics for interface ( i - )

Example:

ok if.drivers
0 enp2s0           igc
1 enp3s0           igc
ok 0 if.mac.show 
9a:88:c6:d1:3d:4a                   97 Locally Administered Address (LAA)
3e:10:d5:14:22:7e                   66 Locally Administered Address (LAA)
24:d7:9c:e4:3e:e9                   21 Cisco Systems, Inc
00:11:32:d5:11:11                    1 Synology Incorporated
52:54:00:9c:53:c1                    1 Locally Administered Address (LAA)
00:3e:e1:c0:f0:9c                  295 Apple, Inc.
56:e6:36:5f:17:a8                    8 Locally Administered Address (LAA)
32:3a:fd:84:56:64                   94 Locally Administered Address (LAA)
00:e2:69:89:51:e4                   23
a0:52:72:1a:ca:41                    1 Apple, Inc.
00:23:7d:86:71:ff                    3 Hewlett Packard
ce:0e:14:15:67:bc                   95 Locally Administered Address (LAA)
50:1e:2d:49:c2:24                  206 StreamUnlimited Engineering GmbH
06:b6:2b:f1:08:fa                    1 Locally Administered Address (LAA)
ee:0c:92:fe:f1:1c                    1 Locally Administered Address (LAA)
32:3a:fd:86:36:3f                   39 Locally Administered Address (LAA)
4e:3a:fd:86:36:41                    1 Locally Administered Address (LAA)
38:10:d5:14:22:7c                    1 AVM Audiovisuelles Marketing und Computersysteme GmbH
3c:2a:f4:07:f0:4b                    1 Brother Industries, LTD.
00:11:32:d5:11:12                   56 Synology Incorporated
ok ▂

This is from a freshly started EtherTAP in bridging mode, showing the MAC addresses and packet counts on RX of interface 0. Since RPCL has the recent manufacturer database compiled in, the manufacturer information is also shown. Locally administered addresses are additionally shown as such. If the manufacturer is not known, the information field stays blank (as for 00:e2:69:89:51:e4 above).

`if.qstats` #

show detailed queue/interface statistics ( queue interface - )

Example:

$ sudo control ethertap
$ 0 0 if.qstats 
queue 0 interface 0 enp2s0 igc fd 3
  rx_packets_ctr            2076
  tx_packets_ctr            1877
  refill_packets_ctr        2076
  drop_packets_ctr          0
  drop_fi_ctr               0
  faults_injected_ctr       0
  noif_packets_ctr          0
  tap_packets_ctr           0
  txbuffer_full_packets_ctr 0
  txring_full_packets_ctr   0
  completed_packets_ctr     1877
  xdp_options_zerocopy      OFF
  rx_dropped                0
  rx_invalid_descs          0
  tx_invalid_descs          0
  rx_ring_full              0
  rx_fill_ring_empty_descs  0
  tx_ring_empty_descs       119365579
$ ▂

The XDP statistic counters rx_fill_ring_empty_descs and tx_ring_empty_descs are shown as they are, but we regard both values as meaningless (they aren’t documented, but existent).

`if.stats` #

show summary statistics of all interfaces ( - )

Example:

ok if.drivers
0 enp2s0           igc
1 enp3s0           igc
ok if.stats
0 enp2s0 igc queues:1 COPY
  rx                 930
  tx                 1359
  drop               0
  noif               0
  err                0
  RX tap             enabled
  TX tap             disabled
  fault injection    disabled
    drop propability 0 (0.000%)
    flip propability 0 (0.000%)
    dropped by FI    0
    flipped by FI    0
1 enp3s0 igc queues:1 COPY
  rx                 1359
  tx                 930
  drop               0
  noif               0
  err                0
  RX tap             enabled
  TX tap             disabled
  fault injection    disabled
    drop propability 0 (0.000%)
    flip propability 0 (0.000%)
    dropped by FI    0
    flipped by FI    0
ok ▂

EtherTAP - Basic Usage

April 28, 2025

EtherTAP #

The Layers #

EtherTAP #

IXDP #

libxdp/eBPF/NIC drivers #

The ethertap Command #

EtherTAP Options #

-q|–queues #

-r|–ringsize #

-i|–initfile #

-d|–daemon #

EtherTAP Modes #

Bridging Mode #

Mirroring Mode #

Control #

Control Tasks #

control background process #

show background process status #

terminate background process #

EtherTAP and RPCL #

Show Uptime #

Show RPCL Variables #

Example: EtherTAP in Bridging Mode #

Example: EtherTAP in Mirroring Mode #

The Drop Machash #

drop.hash.insert #

drop.hash.reset #

drop.hash.show #

Interface Statistics and Counters #

if.drivers #

if.etype.show #

if.mac.show #

if.qstats #

if.stats #

The `ethertap` Command #

`drop.hash.insert` #

`drop.hash.reset` #

`drop.hash.show` #

`if.drivers` #

`if.etype.show` #

`if.mac.show` #

`if.qstats` #

`if.stats` #