Linux - Jiri Brejcha

10 Gigabit Ethernet on Raspberry Pi 5

Raspberry Pi 5 comes with PCI Express connection and a number of HATs (hardware attached on top) and Bottoms (the opposite of that) are now available for sale. That unlocks some very exciting options. Let’s see how fast can a 10 Gigabit Ethernet adapter on Raspberry Pi 5 go, shall we?

Pineberry’s HatDrive! Bottom proved to be really handy for converting Pi’s PCIe connection to M.2 M-key format. My Kalea-Informatique 10 Gigabit adapter uses exactly that, so that’s a match. Why did I choose this adapter? Very unscientifically this time – it was the first readily available and I was in a fail-fast mood :)

10 GbE adapter connected to Raspberry Pi 5

Pineberry HatDrive! Bottom board with 10 GbE network adapter

Detail of the AQC107 chip powering the network adapter

Enable PCIe port on Raspberry Pi 5

First things first. We need to enable the PCIe connector on the Pi.

sudo nano /boot/firmware/config.txt

# Enable the port
dtparam=pciex1

# Configure PCIe Gen
dtparam=pciex1_gen=2

Build custom Linux kernel and include the Aquantia driver module

Vanilla Raspberry Pi OS doesn’t include the Aquantia AQC107 kernel module. So we need to burn a micro SD card with a vanilla Raspberry Pi OS Bookworm image, boot the Pi 5 and build a customised kernel.

git clone --depth=1 --branch rpi-6.8.y https://github.com/raspberrypi/linux
cd linux/
sudo apt install flex bison aptitude -y
sudo aptitude install libssl-dev
make bcm2712_defconfig

Edit the config file:

sudo nano .config

Add these 2 lines to .config file:

CONFIG_AQTION=m
CONFIG_AQUANTIA_PHY=m

Trigger customised kernel build on the Pi. This will take some time, so bear with us, please.

sudo make -j4 Image.gz modules dtbs
sudo make modules_install
sudo cp -v arch/arm64/boot/dts/broadcom/*.dtb /boot/firmware/
sudo cp -v arch/arm64/boot/dts/overlays/*.dtb* /boot/firmware/overlays/
sudo cp -v arch/arm64/boot/dts/overlays/README /boot/firmware/overlays/
KERNEL=kernel_2712
sudo cp -v arch/arm64/boot/Image.gz /boot/firmware/$KERNEL.img
uname -a
sudo reboot

After reboot, the LED light on the network adapter should come to life and we can capture first impressions.

Temperature

First thing you will likely notice is how hot this network adapter runs. It runs at 85° Celsius in idle which is slightly worrying and you can literally burn your fingers if you are not careful. Thumbs down on the thermal design front.

Under load, surprisingly, it ‘only’ runs 0.5° warmer.

How fast can it go then?

Raspberry Pi 5 officially supports PCIe Gen 1 and Gen 2. It is not certified for Gen 3.

PCIe Gen 1 mode

In this slowest mode, I got 1.71 Gbps/1.53 Gbps iperf3 TCP results with standard iperf3 settings. No jumbo frames, no other tweaks.

PCIe Gen 2 mode

Again, with standard iperf3 settings, I measured 3.44 Gbps/3.04 Gbps TCP throughput between 2 computers both connected to 10 Gbps switch ports via 10 GbE Full Duplex.

In idle conditions, this setup draws 7.5 W, and 8.9 W under 10GbE adapter iperf3 -R load (3.45 Gbps). Using more iperf3 parallel streams (the -P parameter) did not help at all.

PCIe Gen 3 mode

The adapter supports PCIe Gen 3, but it doesn’t work with the Pi. The Pi is not certified for Gen 3, so I can’t say anything bad about this. The Ethernet adapter is not recognised in Gen 3 mode, and no interface is present in ip a. Sometimes the Pi will fail to boot.

According to dmesg, the Pi forced Gen 2 mode:

brcm-pcie 1000110000.pcie: link down
brcm-pcie 1000120000.pcie: Forcing gen 2

Forcing PCIe Gen 2 mode athough Gen 3 has been configured

I powered my Pi from M2 MacBook USB-C port. So I thought, I might be running into under-voltage issues. I tested the official Raspberry Pi 27 W (5 V * 5 A) AC power and it made no difference.

Did you upgrade Raspberry Pi 5 firmware?

Yes, I did. It is running the latest version available as of March 2024.

Low CPU utilisation

One feature I really enjoyed is the extremely low CPU utilisation under load. I saw slower 2.5 GbE adapters hammer CPU with interrupts, but that’s not the case for this NIC. AQC107 does really good job at keeping the CPU cool.

Low Raspberry Pi 5 CPU load under network load

Cable analytics

Marvell supports Cable Diagnostics feature which uses TDR to measure cable length and detect Ethernet cable for defects. Unfortunately, it doesn’t seem to be supported on the AQC107 chip.

Can you get 10 Gbps out of this adapter at all?

I am glad you asked. How does an Intel NUC with this 10 GbE adapter sound? I’ve just tested it, here you go.

Summary

The high operating temperature really makes this adapter something I can’t recommend. With maximum throughput below 3.5 Gbps, I think you would be better off choosing a 2.5 Gigabit Ethernet adapter, which runs cool and delivers 2.35 Gbps/2.35 Gbps throughput.

Have you tested any other 10 GbE adapter? Did you get better results? Did you find any 2.5 Gbps Ethernet adapter that supports Cable Diagnostics? I am all ears.

Special thanks

Thanks to Luke Jenkins for exploring and sharing the kernel build instructions. Also, thanks to the WLAN Pi team. You can buy the team a coffee using this link.

Wi-Fi 7 comes to WLAN Pi M4

With the WLAN Pi team, we have designed and launched a M.2 adapter from A-key to E-key, which allows you to install a certified Wi-Fi 7 adapter Intel BE200 to your current WLAN Pi M4.

Is WLAN Pi selling ‘keys’ now? 😉

What is a ‘key’? It is formed of the notch on the Wi-Fi adapter PCB, and plastic blob separating pins inside the M.2 slot. The idea is to prevent users from plugging incompatible cards to the slot, and avoid any ‘magic smoke events’. Here is more about M.2 and the individual key types if you are interested.

WLAN Pi upgrade kit

Since Intel adapters use E-key and WLAN Pi M4 uses A-key, we needed to build an adapter. Badger Wi-Fi has the upgrade kit in stock. It comprises of the Oscium M.2 A-key to E-key adapter, Intel BE200 Wi-Fi 7 adapter, and 2 little bolts to secure the adapter and the Wi-Fi module.

Here is how the ‘butterfly’ setup looks like. Intel BE200 sits onboard of the A-key to E-key adapter, installed in the M.2 slot.

We are ready to connect existing tri-band antennas, and assemble the unit.

Software support

Make sure to either upgrade Linux packages to their latest versions using sudo apt update && sudo apt upgrade command, or download and flash the latest WLAN Pi software image on your SD card. Release 3.2.0 supports Wi-Fi 7 Intel BE200 adapter out of the box with no effort whatsoever on your part.

Wi-Fi 7 in action

For this demonstration I use a consumer Wi-Fi 7 router TP-Link Deco BE85 BE19000. Simply because it is available, Wi-Fi 7 certified, and it supports 320 MHz channel width – not that one would deploy that in an enterprise environment, but mainly to test the maximum Wi-Fi throughput of the Pi.

A bug in macOS doesn’t allow Macs to correctly recognise Wi-Fi 7 networks. Instead of Wi-Fi 7 320 MHz wide network, my MacBook reports Wi-Fi 6 and 160 MHz wide channel. So, we will use another WLAN Pi and its Wi-Fi radio as a Remote Sensor in WiFi Explorer Pro – you need the Pro version to do this.

Nice, Wi-Fi 7 AP!

Connecting the WLAN Pi as a Wi-Fi 7 client only takes few lines of wpa_supplicant config.

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

And we have successfully connected the WLAN Pi as a Wi-Fi 7 client to the AP using this command.

sudo wpa_supplicant -c /etc/wpa_supplicant/wpa_supplicant.conf -i wlan0

Run this command to make sure the WLAN Pi requests an IP address from DHCP server running on the router:

sudo dhclient -i wlan0 -v

What channel are we using? 320 MHz channel width? Indeed.

Before you ask, distance between the Pi and the router is sub 1 meter. What is the Wi-Fi data rate? We are using Wi-Fi 7 (EHT), 2 spatial streams, MCS 12 and 4096-QAM and short guard interval of 0.8 µs.

We can refer to Francois Verges’ MCS index tool to check how we are doing. Yes, I have tried, but I have only been able to achieve MCS 13 extremely rarely.

How far from the AP can we maintain 4096-QAM?

I hardly ever achieved MCS 13. To maintain MCS 12, I had to stay within about 1.5 meter distance from the router. I got best results with antennas position in this ‘V’ pattern.

With a different client device designed for Wi-Fi 7 from the ground up (with professional quality antennas and placement), I would hope for slightly longer MCS 12 and MCS 13 range.

It’s throughput test time

It’s time to run an iperf3 test and see how much traffic we can actually push over the air and also how much the WLAN Pi M4 can handle. Here is our test setup. I recommend the OWC 10 GbE Thunderbolt adapter (it uses Thunderbolt protocol, not USB) connected via USB-C to your Mac.

With the help of Oscium WiPry Clarity 6 GHz spectrum analyser connected to another WLAN Pi, we can monitor the life spectrum and see how much red the iperf3 test introduces. We are able to achieve download TCP speed of 2.27 Gbps and upload speed of 1.74 Gbps.

I used iperf3 -c 192.168.68.51 -P32 -R to test download speed, and iperf3 -c 192.168.68.51 -P32 for upload. Number of parallel streams set to 32 provided the best performance.

Summary

Wi-Fi 7 works well on the WLAN Pi M4. In fact, it works better than Wi-Fi 7 on Windows 11. We have covered Intel BE200 on Windows 11 in this blog posts.

I was expecting 2.5 Gbps-ish throughput, which we have got quite close to. During the test, CPU of the WLAN Pi was running around 80 % utilisation, and interrupts were reaching 100 %. So, hardware of the WLAN Pi itself posed a bottleneck.

mpstat 1 300 -P ALL

Orientation of the antennas mattered more than I expected to. Best position was a ‘V’ shape with antennas positioned away from the board. With AUX antenna placed 90 degrees relative to the Main antenna, data rates and throughput dropped. Perhaps there is RF noise from the board itself coming into play.

Use SSH key stored on GitHub instead of an SSH password to access your WLAN Pi

By default WLAN Pi, and Linux in general, uses a username and password-based SSH authentication. It involves quite some typing, some brain capacity to remember the password, and it is not the most secure method either.

You can create a public and private key pair. Your SSH client automatically logs in using the private key. The SSH server uses the public key to confirm that you possess the right private key. No password needed, and it also is more secure. The private key is never sent over the network, and this method protects you against man-in-the-middle attacks.

The beauty of this GitHub method is that GitHub stores your SSH public key centrally, which you can easily update, and you can install it to the machine you want to SSH to, by a single command ssh-import-id-gh. You can even add this to a startup script so that it automatically updates your trusted keys.

Let’s do this

ssh-keygen is the program that generates a public/private key pair on your local system. The private key is stored in ~/.ssh/id_rsa, and the public key is stored in ~/.ssh/id_rsa.pub.

The security of this method depends on keeping the private key safe and secure. Make sure not to leave the private key behind.

ssh-keygen -t rsa -C "your@email.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/jiri/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/jiri/.ssh/id_rsa
Your public key has been saved in /Users/jiri/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:.....
The key's randomart image is:
+---[RSA 3072]----+
.....
+----[SHA256]-----+

Display the public key, which is a text file at the end of the day, and copy its content to clipboard:

cat ~/.ssh/id_rsa.pub
ssh-rsa
.....

Save this public key to your GitHub account. Browse to github.com, log in, and open Settings:

Click New SSH key, name the key, paste your public key from the clipboard and save it:

To verify that your key has been added you can browse to https://api.github.com/users/jiribrejcha/keys, where jiribrejcha is your GitHub username:

The last step is to SSH into your WLAN Pi or Linux machine and tell it to use this public key from my GitHub, where jiribrejcha is my GitHub username:

ssh-import-id-gh jiribrejcha

If the command isn’t installed, you can fix that by:

sudo apt install ssh-import-id

Passwordless SSH access

When you authenticate to a server using public key authentication, the SSH client offers a copy of the public key to the server and the server then compares it against the keys listed in your ~/.ssh/authorized_keys file. This key was added automatically by the ssh-import-id-gh command. If the key matches, the server indicates that it is able to proceed with the authentication. The private key is then used to sign a message that includes data specific to the SSH session. The server can then use its copy of the public key to verify the signature.

We have just SSH’d to the Pi without a password prompt.

Special thanks

To Colin Vallance for sharing this tip.