Supermicro SuperServer E302-9D Review: A Fanless 10G pfSense Powerhouse

Packet Processing Benchmarks with pkt-gen

The pkt-gen benchmarks were processed using the Conductor python package infrastructure in a similar manner to the iPerf3 benchmarks presented in the previous section. Commands are executed on the source, sink, and DUT using the Conductor python package described in the testing methodology section. The setup steps on the DUT for each mode were described in a previous section. Only the source and sink [Run] phases are described here.

On the sink side, receivers are spawned for the two interfaces serially first. Simultaneous execution is then performed after the required wait time in order to monitor both interfaces. The spawn and timeout refer to keywords specified by the Conductor package.

spawn0:sh pkt.gen.recv.sh 1 ix0 [tested-mode]-pg-rx-1c.0.txt
timeout200:sleep 185
step1:killall pkt-gen
spawn1:sh pkt.gen.recv.sh 3 ix2 [tested-mode]-pg-rx-1c.1.txt
timeout201:sleep 185
step2:killall pkt-gen
timeout30:sleep 30
spawn2:sh pkt.gen.recv.sh 1 ix0 [tested-mode]-pg-rx-2c.0.txt
spawn3:sh pkt.gen.recv.sh 3 ix2 [tested-mode]-pg-rx-2c.1.txt
timeout202:sleep 185
step3:killall pkt-gen

The pkt.gen.recv.sh script handles the reception of the packets sent via the firewall on the appropriate interface and dumps out the statistics to the specified file.

On the source side, the first link is evaluated for 30s with each packet size, followed by the second link. In the third iteration, the tests are spawned off for both links simultaneously.

spawn0:sh pkt.gen.sweep.sh ixl2 172.16.0.2:53 172.16.10.2:53 [ixl2 mac] 1 [tested-mode]-pg-tx-1c.0.txt
timeout200:sleep 185
step1:killall pkt-gen
spawn1:sh pkt.gen.sweep.sh ixl3 172.16.1.2:53 172.16.11.2:53 [ixl3 mac] 3 [tested-mode]-pg-tx-1c.1.txt
timeout201:sleep 185
step2:killall pkt-gen
timeout30:sleep 30
spawn2:sh pkt.gen.sweep.sh ixl2 172.16.0.2:53 172.16.10.2:53 [ixl2 mac] 1 [tested-mode]-pg-tx-2c.0.txt
spawn3:sh pkt.gen.sweep.sh ixl3 172.16.1.2:53 172.16.11.2:53 [ixl3 mac] 3 [tested-mode]-pg-tx-2c.1.txt
timeout202:sleep 185
step3:killall pkt-gen

Here, the pkt.gen.sweep.sh script resident in the source's file system is a wrapper for calling pkt-gen multiple times with varying packet sizes in series. The appropriate CPU core allocation and output file specifications are also passed on to this shell script.

Two sets of metrics - the packet rate and the bandwidth - are gleaned from the log files and graphed below. Note that the bandwidth numbers reported by pkt-gen sometimes exceeds the line-rate - particularly when it misses a couple of samples in the previous timestamps. Despite that obvious discrepancy, we get an idea of the average bandwidth and packet rates for each packet size, as the source tries to saturate the links.

The pfSense installation running on the E302-9D seems to have a best-case packt forwarding rate of 0.6 Mpps per interface, and this goes down to around 0.35 Mpps in the worst case with a large number of rules and NAT being enabled.

On the bandwidth front, we see a best-case throughput of around 6.5 Gbps. This goes down as packet processing steps start getting enabled, as shown in the above graphs.