Managing the network stack

The administration of the EVL network stack is done via the evl-net command line interface from the EVL command set. The general syntax is as follows:

$ evl net --help
usage: evl-net [options]:
-e -i <ifname> [-p][-b]            enable out-of-band port in network interface <ifname>
   -p <pool-size>                  max number of out-of-band socket buffers (0=default)
   -b <buffer-size>                size (in bytes) of out-of-band socket buffer (0=default)
-d -i <ifname>                     disable out-of-band port in network interface <ifname>
-s <host> [-i <ifname>][-g]        solicit <host> via <ifname> if given
-S <host> [-i <ifname>][-g]        like -s, marking ARP entry as permanent
   -g                              allow routing to destination via gateway(s)
-Q[RrTtosfacx] -i <ifname>         query network interface information about <ifname>
-F[<bpf-module.o>] -i <ifname>     install/remove eBPF filter (RX)
-N[<vlan-id-list]                  add/remove VLAN id. filter (RX)
-n                                 show VLAN id. filter (RX)

Enabling an out-of-band port on a network device

In order for EVL to send and/or receive network traffic via a network interface device, the latter should be enabled as an out-of-band port. This can be done using the -e switch from the evl net command, where -i specifies the network interface to enable as an out-of-band port. In addition, you can fine-tune the number of buffers in the per-device pool maintained by EVL which should be pre-allocated for out-of-band use with the -p switch. The size of this pool depends on the volume of the egress traffic, since EVL consumes those buffers on the transmit path only, the receive buffers are allocated by the NIC drivers instead. You can also set the size (in bytes) of such buffer with the -b switch, which should accomodate for the largest MTU you intend to use with your device.

~# evl net -ei eth1.42 -p 4096

Every buffer in the per-device pool is pre-mapped for DMA operation when the out-of-band port is brought up. Such mapping lasts as long as the port stays enabled. Changing the buffer count or buffer size requires bringing down the port then up again with the appropriate settings (dynamic update is not available for these parameters).

Disabling an out-of-band port on a network device

The converse operation to evl net -ei <interface> is evl net -di <interface>, which brings down an out-of-band port.

~# evl net -di eth1.42

Bringing down a network device automatically dismantles the out-of-band port enabled for it.

Soliciting a peer for out-of-band communication

As explained in this document, EVL can guarantee that egress traffic is sent from the out-of-band stage directly, only if its front caches contain the routing information needed to reach the peers. Otherwise, the outgoing traffic is offloaded to the in-band stage. Both evl -S<ipaddr> <interface> and evl -s<ipaddr> <interface> call forms issue a solicitation request to a particular peer identified by its IPv4 address, in order to update those caches appropriately. Unlike -s, the -S form makes some of the gathered information permanent in the ARP front cache, which is likely what you want. argument_)

By default, the core searches for a path to the peer which should be directly reachable from the current host, i.e. no more than one hop away (aka on-link routing scope modifier). Conversely, the -g switch allows the EVL core to follow gateways, i.e. look for indirect routes to the peer (aka universe routing scope modifier).

The argument following the option key may be an IPv4 address, a host name, or the 'broadcast' keyword which substitutes to 255.255.255.255. You need to make sure that an out-of-band port is enabled on the device which may be selected.

For instance, say your local device eth0.42 tunneling traffic with VLAN id. 42, reachable at 10.10.10.11, should be used to send traffic to a peer located at 10.10.10.10:

~# ip -br addr show dev eth0.42
eth0.42@eth0     UP             10.10.10.11/24 fe80::522d:f4ff:fe1d:32d0/64 
~# ip -d link show dev eth0.42
4: eth0.42@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 50:2d:f4:1d:32:d0 brd ff:ff:ff:ff:ff:ff promiscuity 0 allmulti 0 minmtu 0 maxmtu 65535 
    vlan protocol 802.1Q id 42 <REORDER_HDR> addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 100 tso_max_size 65536 tso_max_segs 100 gro_max_size 65536

You could issue the following commands to turn on the out-of-band port on eth0.42, then solicit the peer at 10.10.10.10:

~# evl net -ei eth0.42
~# evl net -S 10.10.10.10

EVL currently implements solicitation for AF_INET addresses only (IPv4). Local host (127.0.0.1) and broadcast (255.255.255.255) pseudo-addresses are valid input. However, you have to solicit a specific device for the broadcast address, so you must provide an interface name with -i to the command in this case.

Querying the status and statistics about of an out-of-band port

You can get some statistics about any given port using the evl net -Q[<modifier>] sub-command. <modifier> indicates which particular information you want to obtain, as a terse list of space-separated values. The command outputs all of the available information in verbose format if no modifier is given (note the absence of space between the option key and the first modifier if given).

  • R displays the number of packets sent through the port.
  • r displays the number of bytes sent through the port.
  • T displays the number of packets received through the port.
  • t displays the number of bytes received through the port.
  • o displays the out-of-band capability status of the underlying NIC driver the port uses for sending and receiving traffic.
  • s displays the size of a buffer in the per-device pool.
  • f displays the number of free buffers still available in the per-device pool.
  • a displays the overall number of buffers in the per-device pool.
  • c displays the number of UDP checksum errors
  • x displays the number of failed allocations on RX-TX respectively.
~# evl net -Q -i eth0.42
oob capability: yes
    rx packets: 0
      rx bytes: 0
    tx packets: 0
      tx bytes: 0
      skb size: 4096
      skb free: 2048 / 2048
      csum errors: 0
      rx nomem: 0
      tx nomem: 0

~# evl net -QoRTf -i eth0.42
1 2092625 137399 2048

The data above only and specifically applies to out-of-band I/O, which is accounted separately from in-band I/O statistics as displayed by ethtool.

Memory allocation failures on RX (rx_nomem) is usually caused by the NIC driver not having enough buffers for receiving data directly from the out-of-band stage (e.g. low size of an oob page pool). Memory allocation failures on TX (tx_nomem) is caused by an under-sized per-device buffer pool, which cannot cope with the amount of outgoing traffic.

Setting the VLAN filter list

In order to use VLAN tagging to discriminate out-of-band data from the rest of the incoming traffic, you need to tell the EVL core which VLAN identifiers should be monitored for this. If given with an argument string, the -N option switch allows passing a list of out-of-band VLAN identifiers to filter traffic into the out-of-band network stack. Without argument, the -N switch clears the active filter list in the EVL core.

For instance, say that you have created the eth0.oob VLAN device, with VLAN identifier 42, you would allow the EVL network stack to pick ingress traffic received by this device using the following command (note the absence of space between the option key and its argument):

~# evl net -N42

If you have multiple VLAN identifiers to mention, you may also use intervals and a list of comma-separated numbers, For instance, the following command would set a filter list composed of VLANs 41, 42, 55, 56 and 57.

~# evl net -N41,42,55-57

Conversely, you can clear the active filter list by passing no argument to the -N option switch, i.e.

~# evl net -N
VLAN filter cleared

In absence of both VLAN filter list and eBPF program loaded on some network device, the out-of-band network stack cannot accept any traffic.

Showing the active VLAN filter list

Using the -n option switch displays the current VLAN filter content to the standard output. For instance:

~# evl net -N41,42,55-57
~# evl net -n
41-42,55-57

Loading an eBPF program on an out-of-band port

As mentioned in this document, you can load an eBPF program module in order to implement complex selection rules of out-of-band packets. evl -F[<modpath] -i <interface> loads such a program if a module path is given, or removes any module already loaded on the port otherwise (note the absence of space between the option key and the module path if given).

~# evl net -Fnetrx.o -i eth0.42
[  782.270250] fec 2188000.ethernet eth0: out-of-band eBPF program installed

~# evl net -F -i eth0.42
[  819.919746] fec 2188000.ethernet eth0: out-of-band eBPF program removed
Last modified: Thu, 03 Jul 2025 16:39:27 +0200