Networking - Jiri Brejcha

Full 5 Gigabit Ethernet on Raspberry Pi 5 with iocrest Realtek RTL8126 adapter

I’ve tested a number of 10 Gigabit Ethernet adapters on Raspberry Pi 5 based on the AQC107 chip. One adapter that negotiates PCIe Gen 3, achieves 5.5 Gbps speed and overheats. Another one which only works in PCIe Gen 2 mode and peaks at 3.44 Gbps. And even a full-size PCIe card made by TP-Link which negotiates PCIe Gen 2 link speed and doesn’t go beyond 3.44 Gbps either.

The Realtek RTL8126 chip we are testing today is so far the most suitable for Raspberry Pi 5. It is capable of 5 Gigabit Ethernet at full speed. TCP iperf3 throughput peaks at 4.7 Gbps. It doesn’t overheat. And it doesn’t excessively utilise the Raspberry Pi 5 CPU.

This particular one is sold under the iocrest brand. Like the other boards and adapters there is no increst branding on it and it will likely be sold under various brands. The RTL8126 chip is the key component here.

Raspberry Pi 5 with 5 Gigabit Ethernet network adapter

How did we connect it to the Pi? Via PCIe bus. We breakout the Raspberry Pi 5’s PCIe connector via Pineboards (aka Pineberry Pi) board to M.2 M-key slot. And in that slot we install the iocrest 5 Gigabit Ethernet network adapter – that’s the black M.2 module, plus a PCB with RJ-45 connector on a grey ribbon cable.

iocrest 5 GbE adapter connected to Raspberry Pi 5 via PCIe Gen 3 link

Here is how it looks from PCI device perspective.

Performance

It has no problem negotiating full duplex 5 Gigabit Ethernet and filling the interface with traffic fully.

iperf3 with default TCP settings peaks at 4.7 Gbps up and down. More parallel streams don’t improve the result any further. This is in PCIe Gen 3 mode.

Full 5 Gigabit Ethernet throughput in PCIe Gen 3 mode

Just for the record, if we downgrade PCIe bus to Gen 2 link speeds, we are talking 3.43 Gbps down and 3.31 Gbps up iperf3 TCP throughput-wise.

Throughput in downgraded PCIe bus to Gen 2 mode

Thermal footprint

Fully loaded by TCP traffic, I see temperature of 81.2° C (178° F) on the top surface of the RTL8126 chip. Yes, it is on the warmer side, but Raspberry Pi 5 SoC runs quite warm too and it is nowhere near 122° C temperatures I observed on this “hot” 10 Gigabit Ethernet adapter.

Chip temperature, installed in Intel NUC with M.2 slot

By the looks of it, there is no temperature sensor on the PHY so I can’t measure internal temperature.

CPU utilization and temperature of fully loaded adapter with TCP traffic

Linux software support

I happened to have Raspberry Pi OS with 6.8.0-rc7 kernel running on the Raspberry Pi 5. Out of the box, the adapter did not work. iocrest included driver download link pointing to this Chinese website but I am not so sure I want to use that one.

After installing driver from Realtek’s website, the adapter works just fine.

Power draw

This adapter in PCIe Gen 3 mode draws about 1.5 W in idle, and 2.1 W under full iperf3 load.

Switching the adapter to Gen 2 mode doesn’t make any power savings. I measured 0.1 W less in Gen 2 mode.

The whole setup of Raspberry Pi 5 with fan, Pineboards PCIe adapter, and this 5 GbE adapter in PCIe Gen 3 mode draws about 5.1 Watts in total under full iperf3 load.

Does it work on Windows 11?

Yes, it does. I installed one in Intel NUC 12th generation. It runs at full speed full and Gen 3 x1 mode.

Windows 11 driver (as of May 2024) downloaded automatically via Windows Update only allows this adapter to use 2.5 GbE. To unlock 5 GbE we download driver directly from Realtek’s website and we are all set.

Driver from Realtek’s website with full 5 GbE support

5 GbE full duplex with driver from Realtek’s website

With the adapter inserted in M.2 M-key slot, we won’t be able pop the NUC bottom lid back on. The adapter is just a bit too tall.

Bottom lid won’t fit with the adapter installed

Throughput also looks good. I might revisit Windows throughput testing tools at some point. But for now, I take 4.74 Gbps down and 4.42 Gbps up speeds. Increasing number of parallel streams did not improve throughput in any way.

For the record, Jumbo frames seem to be supported but I had no reason to explore this further this time.

Summary

As I mentioned towards the beginning, 5 Gigabit Ethernet based on Realtek RTL8126 chip seems to strike the perfect balance for Raspberry Pi 5. It delivers 4.7 Gbps up and down, doesn’t consume much power, and doesn’t produce excessive amount of heat.

Long-time test will tell how it actually performs but for now I am happy with what I’ve seen.

From driver perspective, I am wondering if the latest Linux kernel supports this chip natively or if I can enable the right kernel module manually.

How to connect full-size 10 Gigabit Ethernet PCIe adapter TP-Link TX401 to Raspberry Pi 5 and Intel NUC

It is refreshing to be able to test hardware which actually has a product name :) TP-Link TX401 is a 10 Gigabit Ethernet copper PCIe adapter.

How to connect standard PCIe card to Raspberry Pi 5

I am testing on Raspberry Pi 5 and Intel NUC. Both do have an M.2 M-key slot and they won’t take this card natively, will they?

Pineboards (previously known as Pineberry Pi) makes a great PCIe Gen 3 compatible board that breaks out Raspberry Pi 5 PCIe connector to M.2 M-key slot. And from there we can use another adapter – MZHOU M.2 to PCIe 4X Adapter. It allows us to insert a standard size PCIe card into M.2 M-key slot.

Pineberry Pi M.2 M-key adapter -> M.2 to PCIe adapter -> PCIe card

The Ethernet adapter is correctly recognised. We just need to build a custom Linux kernel with AQC107 kernel module enabled. Steps by steps instructions are here for your reference. They work for all AQC107 based adapters I’ve tested.

It negotiates 10 Gbps Full duplex link with my switch.

But it only works in PCIe Gen 2 mode on Raspberry Pi 5 in this setup. That means that throughput will be significantly limited to 3.44 Gbps download TCP speed and 3.07 Gbps upload. Using more parallel streams did not help in any way. We are limited by the 4 Gbps throughput of PCIe Gen 2.

I was not able to make PCIe Gen 3 work using this setup. Understandably, high-speed buses don’t like the extra connectors and adapters.

Updated: It wasn’t available back then when I tested this, but Pineboards now sells uPCIty Lite HAT for Raspberry Pi 5 which completely removes the need for the intermediate MZHOU adapter.

How to connect standard PCIe card to Intel NUC

The same M.2 M-key to standard PCIe card adapter works with my Intel NUC 12th Generation.

Windows 11 automatically downloads the latest AQC107 driver using Windows Update.

It negotiates 10 Gbps Full duplex.

The TP-Link card successfully negotiates PCIe Gen 3 x4.

PCIe Gen 3 allows us to achieve TCP throughput of 9.48 Gbps with no effort in the download direction and 9.49 Gbps in the upload. So this card can clearly do 10 Gigabit Ethernet, it just needs PCIe Gen 3 link speed.

No overheating problem

Unlike unbranded Chinese adapters using the same AQC107 chip, this adapter is designed does not overheat. You can read some horror stories about chip temperatures of 122° degree Celsius (252° F) here.

PCIe Gen 2 full load temperature 63.3° C

TP-Link TX401 in PCIe Gen 3 mode on Windows 11 runs at normal temperature

Summary

This adapter achieves nearly 9.5 Gbps of TCP throughput in either direction on Windows if you allow it to use PCIe gen 3 link speed.

Unfortunately, it only negotiated PCIe Gen 2 with Raspberry Pi 5 and Ethernet throughput is limited to about 3.4 Gbps. So for Raspberry Pi, I would recommend a 2.5 GbE adapter which it can fully handle. Alternatively, a 5 GbE adapter. Coming up next. Stay tuned.

It is a good product though with solid cooling. It still produces some heat but that’s a feature of the AQC107 chip. Its advantage is that it keeps the actual system CPU utilisation low even when fully loaded.

The TX401 is a great fit for a desktop machine. If you run a Mac or NUC, I recommend the external 10 Gigabit Ethernet network adapter connected via USB-C using Thunderbolt 3 protocol. No drivers needed.

Direct 20 Gbps connection between Mac and Windows 11 machine with no Ethernet adapters

Problem statement

Here is the challenge. We have a MacBook Pro M2 and an Intel NUC 12th generation PC running Windows 11. We want to transfer a significant amount of data between the two and potentially sync content of 2 directories. The Mac has no Ethernet adapter.

Solution

Both machines support Thunderbolt 4 and USB4. I happen to have a 0.5 m (1.6 ft) Thunderbolt 4 cable in my tools bag. We connect the two machines back to back. They establish USB4 peer to peer 20/20 Gbps connection, and automatically assign locally significant IP addresses from the 169.254.0.0/16 APIPA range.

For the record, I am using the Thunderbolt 4 cable shipped with my trusty OWC 10 Gigabit Ethernet Thunderbolt adapter.

Direct MacBook to Intel NUC USB4 20/20 Gbps connection

The MacBook side

Let’s start with the Mac. Head over to System Settings and Network. Select the Thunderbolt Bridge adapter and explore its config.

Thunderbolt bridge interface and IP address

As far as I can tell, the machines have decided to use USB4. From what Windows network manager is telling us, they negotiated 20/20 Gbps link speed. I expected 40 Gbps but I think I set a wrong expectation in my head. 20 Gbps up and 20 Gbps down full duplex makes up 40 Gbps.

Windows PC on the other end of the Thunderbolt link

A quick iperf3 test gives us amazing throughput of 16.4 Gbps of TCP traffic from the Mac client to PC server. That’s fast!

By default macOS uses standard MTU size of 1500 Bytes. This is important hold that thought.

In the downstream direction, that is from Windows PC towards the Mac, we “only” get 5.3 Gbps. Windows claims 20/20 Gbps link speed, so what’s wrong?

Limited 5.3 Gbps TCP throughput from PC to Mac

Yes, we need to bump MTU (Maximum Transmission Unit) size to the maximum value of 9000 Bytes on my Mac. Apparently, Windows defaults to 62000 Bytes MTU on this peer to peer link type, and there is no UI option to change it. But that’s fine for now.

Let’s retest upload speed. Now we are talking. That’s 16.4 Gbps TCP from Mac to PC and 12.8 Gbps from PC to Mac. I am starting the file transfer.

12.8 Gbps TCP from PC to Mac with Jumbo frames enabled

We are not done yet.

Intel NUC and the Windows part

Windows sees this link as a peer to peer USB4 connection.

The two machines negotiated a 20/20 Gbps link. Windows uses 62000 Bytes MTU by default with no obvious UI option to change it. Mac uses 9000 Bytes. MTU mismatch is bad and we should fix that.

Adapter settings don’t offer MTU adjustment in the UI

Let’s deal with the MTU, and set it to 9000 Bytes on Windows. Same as the Mac.

Set MTU to 9000 Bytes on Windows 11 for this adapter

With matching MTU on both sides of the pipe, we get 15.1 Gbps TCP throughput from Mac to PC, and 13.6 Gbps from PC to Mac. Slightly more symmetrical in both directions.

Summary

I knew Thunderbolt 4 peer to peer connection was possible between 2 Macs but I’ve never tried connecting a Mac to a PC. It works.

Use a Thunderbolt 4 cable, not just a regular “USB-C to USB-C” cable. If there is a Mac involved, increase macOS MTU size to Jumbo 9000 Bytes and match MTU setting on both machines.

The outcome is a peer to peer 20/20 Gbps USB4 link with TCP throughput around 15 Gbps in either direction.

One step closer to 10 Gigabit Ethernet on Raspberry Pi 5 – it is hot

This journey started as an exploration of maximum PCIe capabilities of Raspberry Pi 5 (and hopefully Compute Module 5) platform. I am mainly interested in multi-gigabit Ethernet and Wi-Fi 7 adapters connected via the PCI Express (PCIe) x1 bus.

Last time, we got throughput of 3.44 Gbps. The adapter and the Pi hit the bottleneck of PCIe Gen 2. Unfortunately, they failed to establish PCIe Gen 3 mode.

Generic 10 GbE adapter in M.2 form factor

This time we are going to use a slightly different adapter. It is available from various sellers under different names, but they all look and work the same. I picked up one from “KALEA-INFORMATIQUE” which happened to be readily available in the UK.

Pineberry’s HatDrive! Bottom breaks out Raspberry Pi’s PCIe connection to M.2 M-key format, and that’s where this 10 Gigabit Ethernet adapter plugs into.

Build custom kernel with AQC107 support

This Ethernet adapter uses the same chip and driver the one we previously tested. Here are the steps to make compile a custom Linux kernel that supports the adapter.

Wait, why is it still not working?

We have connected everything, built a custom kernel, we can see the device, but the Ethernet interface is not coming up.

Look at this official product photo and my photo below. Spot one difference 😉

Did you notice the orientation of the white ribbon cable? The official photo got it wrong. The printed text on the cable needs to be on the top on one side, and on the bottom on the other one.

What speeds did you get in PCIe Gen 2 mode?

After correcting the orientation of the flexible cable, the interface came up, negotiated 10 Gbps full duplex.

I started throughput testing against MacBook with my trusty 10 GbE Thunderbolt adapter.

In PCIe Gen 2 mode, we got TCP throughput of 3.45 Gbps on the downlink and 3.07 Gbps in the upstream direction. Using more iperf3 parallel streams did not increase performance.

Were you able to use PCIe Gen 3 mode?

Yes! And I got 4.63 Gbps of TCP downstream and 5.5 Gbps (potentially up to 6 Gbps) upstream.

I managed to get up to 6 Gbps in the upstream direction

That’s hot news… yes 122° Celsius hot!

This adapter has a thermal problem. It comes with a heatsink, but even in idle mode it overheats.

In PCIe Gen 3 mode with iperf3 test running, we are talking 122.1° C hot! The Pineberry board was very hot and you can literally burn your fingers by touching the heatsink.

In PCIe Gen 2 mode, it ‘only’ runs at 96.4° C

Long story short. Don’t buy this adapter, unless you want to add a fan or significantly larger heatsink.

Toaster, 10 Gigabit adapter, aren’t they the same thing?

Make your own opinion based on these couple of thermal photos.

Thermal IR footprint of the Ethernet adapter

This Ethernet adapter as well as the OWC 10 GbE Thunderbolt both use the same Aquantia AQC107 (part of Marvell now) chip. It does really good job at keeping CPU utilisation low. I’ve seen much cheaper 2.5 GbE adapters that hammer CPU with interrupts until the CPU just can’t take no more.

But, compare size of the two heatsinks. Unlike this one, the OWC adapters delivers good thermal results. Don’t take me wrong, it still runs warm, but not anywhere near.

Same AQC107 chip, massive thermal mass difference

Summary

On the positive side, this is the first 10 Gigabit adapter I tested which actually worked in PCIe Gen 3 mode on Raspberry Pi 5. I got TCP throughput of up to 6.0 Gbps.

As far as I can tell, the limit of Raspberry Pi 5’s PCIe bus is around 6 Gbps if you look at it through the iperf3 TCP traffic lens. AQC107 silicon does an amazing job at keeping the Raspberry Pi’s CPU utilisation low. This helps us get as much throughput as we can from the Pi. But it produces a significant amount of heat.

The fact is that this adapter overheats. Don’t buy it unless you wish to use it with a fan or design a much larger heatsink yourself.

Wi-Fi 7 on Windows 11

The other week I tested Wi-Fi 7 on Linux. It worked great. Let’s see how Intel BE200 Wi-Fi 7 adapter performs on Windows 11 23H2.

Intel NUC 12th generation with Intel BE200 Wi-Fi 7 adapter

Drivers

Download and install latest driver from Intel’s website. Windows Update itself won’t install any driver, so some manual steps are required. I originally tested driver version 23.20.0.4, and now updated to 23.30.0.6.

Setup

We are using TP-Link Deco BE85 BE19000 consumer Wi-Fi 7 router connected to a 10 Gigabit iperf3 server running on MacBook connected via OWC 10 GbE to Thunderbolt adapter. We have done this on Linux before, so let’s see how the same Wi-Fi adapter performs on Windows.

Performance

On the router, we have configured and verified 320 MHz wide Wi-Fi 7 channel. But when we connect the Windows client, looking at the data rate, it is surprisingly low – if you forgive me calling 2882 Mbps ‘low’ 😊 Considering that the NUC is about 1 meter away from the router, I would expect ~5 Gbps data rate. So what’s going on here?

Connected Wi-Fi 7 client, but low data rate

Interestingly enough, it is the same data rate as we see when connected using 5 GHz 160 MHz channel. Yes, I know, that’s a no-no in Wi-Fi design. We are just testing here.

Hmm, 160 MHz wide 5 GHz channel gives us the same data rate

netsh wlan show interfaces command output

Since Windows doesn’t expose the channel width in the UI, we don’t quite know what is happening on the air. Let’s generate some 6 GHz traffic, and check using Oscium’s WiPry Clarity tri-band spectrum analyser. I love this little USB tool. In this example I use a WLAN Pi as a Remote Sensor. It scans for Wi-Fi networks and streams spectrum information to WiFi Explorer Pro on Mac.

WLAN Pi Remote Sensor with Oscium WiPry Clarity scanning 6 GHz

Windows client is using 160 MHz instead of 320 MHz

Bingo! Apparently, on Windows Intel BE200 uses 160 MHz channel width and doesn’t support 320 MHz wide channel. That halves our data rate and throughput. I wish Windows made channel width more obvious in the UI. Intel BE200 adapter supports 320 MHz wide channels on Linux without a sweat, so hopefully it will get fixed in a future Intel driver or Windows release.

Updated: Apparently, I didn’t read Intel’s release notes closely enough, my bad 😊 Intel BE200 adapter on Windows 11 is only able to use Wi-Fi 6E today. Windows 11 will introduce Wi-Fi 7 support in a future update. Since Wi-Fi 6E supports channel widths up to 160 MHz, that’s why we are not being able to use full 320 MHz channel width. What really confused me was the “Protocol: Wi-Fi 7 (802.11be)” misleading Wi-Fi network status reported by Windows. Thank you Ben for spotting the note in Intel’s documentation.

What does that translate to? Lower data rate and lower throughput. I would expect download and upload to be around 2.5 Gbps using 320 MHz wide channel. With the latest Intel driver 23.30.0.6, we get 1.71 Gbps TCP download speed with 16 parallel streams, and upload of 2.17 Gbps. But only the upper 160 MHz half of the 320 MHz wide channel is used.

1.71 Gbps TCP download speed with 16 parallel streams

Upload TCP speed 2.17 Gbps with 16 streams

I also ran a quick Speedtest.net test (I know it is not a proper throughput testing tool) on a 900/900 Mbps WAN link.

On a Linux Wi-Fi 7 client, I measured nearly 890/890 Mbps. Original Intel driver 23.20.0.4 performed 383/818 Mbps. The latest Intel driver 23.30.0.6 delivered more symmetric numbers, and results were closer to the actual WAN link speed.

Speedtest.net speeds using the latest 23.30.0.6 Intel driver

Summary

Wi-Fi worked well, but application speeds including Speedtest.net and other tools performed quite poorly and subjectively ‘felt slow’. iperf3 test showed higher performance, but the main problem for the purpose of a throughput test is that the adapter only uses 160 MHz out of the available 320 MHz.

When it comes to recommended channel width in real world, it depends. 80 MHz or 40 MHz wide channels are most likely the best place to start depending on your circumstances and region.

For reference: Disable 6 GHz on Intel BE200 adapter

If you are performing tests on an SSID that has multiple bands enabled, and you want to force the client to drop off 6 GHz and join using a 5 GHz channel instead, Intel BE200 driver has the option to disable the 6 GHz band.

10 Gigabit Ethernet on Intel NUC

Earlier this week, I tested a 10 Gigabit Ethernet M.2 network adapter on Raspberry Pi 5, and it didn’t quite cut it. Mainly due to limited PCIe Gen 2 performance. Now, the question is can this 10 Gigabit adapter actually push 10 Gbps of traffic at all?

To find out, we are going to slightly reconfigure this Intel NUC 12th generation mini PC. It has an M.2 M-key slot for NVMe drive. Let’s use this slot for our 10 GbE adapter. And we will boot Windows 11 off an external USB SSD drive.

Intel NUC with 10 GbE adapter connected to 10 GbE switch

Install Windows 11 23H2 version on a USB SSD drive, boot Windows, run Windows Update, voila!

Windows Update installed latest driver automatically

With default iperf3 settings we get 6.44 Gbps/7.93 Gbps in the downlink and uplink direction respectively. Not bad, but is that it? Of course not.

I don’t really want to enable Jumbo frames as it’s not always possible to enable Jumbo frame support end-to-end, especially if part of the network doesn’t support it or isn’t under your management. Fortunately, 8 parallel TCP streams in iperf3 do the trick for us. We get 9.48 Gbps download speed.

In the upstream direction from this NUC to my MacBook with 10 GbE adapter, we also get 9.48 Gbps. I am happy. You? 😉

Summary

After all, this 10 GbE M.2 network adapter is indeed capable of pushing 9.48 Gbps of traffic in either direction. But! It is not really a good choice for a system like Intel NUC. I can’t pop the lid back on, the heatsink is too tall. Frankly, I can’t recommend this adapter at all. It runs hot at 84° Celsius in idle.

If you are looking for a daily driver, and your system supports Thunderbolt, get yourself this OWC 10GbE to Thunderbolt adapter. Here is my test. It works out of the box on Windows (I tested this Intel NUC 12th Gen) and macOS (I tested MacBook Pro M1 and M2). Interestingly, it uses the same chip as the above M.2 adapter. Just compare the two products and their heatsink sizes. The AQC107 keeps the main CPU utilisation very low, but it produces a significant amount of heat.

OWC 10 GbE to Thunderbolt network adapter connected to Intel NUC

OWC 10 GbE to Thunderbolt network adapter connected to MacBook

10 Gigabit Ethernet on Raspberry Pi 5

Raspberry Pi 5 comes with PCI Express connection and a number of HATs (hardware attached on top) and Bottoms (the opposite of that) are now available for sale. That unlocks some very exciting options. Let’s see how fast can a 10 Gigabit Ethernet adapter on Raspberry Pi 5 go, shall we?

Pineberry’s HatDrive! Bottom proved to be really handy for converting Pi’s PCIe connection to M.2 M-key format. My Kalea-Informatique 10 Gigabit adapter uses exactly that, so that’s a match. Why did I choose this adapter? Very unscientifically this time – it was the first readily available and I was in a fail-fast mood :)

10 GbE adapter connected to Raspberry Pi 5

Pineberry HatDrive! Bottom board with 10 GbE network adapter

Detail of the AQC107 chip powering the network adapter

Enable PCIe port on Raspberry Pi 5

First things first. We need to enable the PCIe connector on the Pi.

sudo nano /boot/firmware/config.txt

# Enable the port
dtparam=pciex1

# Configure PCIe Gen
dtparam=pciex1_gen=2

Build custom Linux kernel and include the Aquantia driver module

Vanilla Raspberry Pi OS doesn’t include the Aquantia AQC107 kernel module. So we need to burn a micro SD card with a vanilla Raspberry Pi OS Bookworm image, boot the Pi 5 and build a customised kernel.

git clone --depth=1 --branch rpi-6.8.y https://github.com/raspberrypi/linux
cd linux/
sudo apt install flex bison aptitude -y
sudo aptitude install libssl-dev
make bcm2712_defconfig

Edit the config file:

sudo nano .config

Add these 2 lines to .config file:

CONFIG_AQTION=m
CONFIG_AQUANTIA_PHY=m

Trigger customised kernel build on the Pi. This will take some time, so bear with us, please.

sudo make -j4 Image.gz modules dtbs
sudo make modules_install
sudo cp -v arch/arm64/boot/dts/broadcom/*.dtb /boot/firmware/
sudo cp -v arch/arm64/boot/dts/overlays/*.dtb* /boot/firmware/overlays/
sudo cp -v arch/arm64/boot/dts/overlays/README /boot/firmware/overlays/
KERNEL=kernel_2712
sudo cp -v arch/arm64/boot/Image.gz /boot/firmware/$KERNEL.img
uname -a
sudo reboot

After reboot, the LED light on the network adapter should come to life and we can capture first impressions.

Temperature

First thing you will likely notice is how hot this network adapter runs. It runs at 85° Celsius in idle which is slightly worrying and you can literally burn your fingers if you are not careful. Thumbs down on the thermal design front.

Under load, surprisingly, it ‘only’ runs 0.5° warmer.

How fast can it go then?

Raspberry Pi 5 officially supports PCIe Gen 1 and Gen 2. It is not certified for Gen 3.

PCIe Gen 1 mode

In this slowest mode, I got 1.71 Gbps/1.53 Gbps iperf3 TCP results with standard iperf3 settings. No jumbo frames, no other tweaks.

PCIe Gen 2 mode

Again, with standard iperf3 settings, I measured 3.44 Gbps/3.04 Gbps TCP throughput between 2 computers both connected to 10 Gbps switch ports via 10 GbE Full Duplex.

In idle conditions, this setup draws 7.5 W, and 8.9 W under 10GbE adapter iperf3 -R load (3.45 Gbps). Using more iperf3 parallel streams (the -P parameter) did not help at all.

PCIe Gen 3 mode

The adapter supports PCIe Gen 3, but it doesn’t work with the Pi. The Pi is not certified for Gen 3, so I can’t say anything bad about this. The Ethernet adapter is not recognised in Gen 3 mode, and no interface is present in ip a. Sometimes the Pi will fail to boot.

According to dmesg, the Pi forced Gen 2 mode:

brcm-pcie 1000110000.pcie: link down
brcm-pcie 1000120000.pcie: Forcing gen 2

Forcing PCIe Gen 2 mode athough Gen 3 has been configured

I powered my Pi from M2 MacBook USB-C port. So I thought, I might be running into under-voltage issues. I tested the official Raspberry Pi 27 W (5 V * 5 A) AC power and it made no difference.

Did you upgrade Raspberry Pi 5 firmware?

Yes, I did. It is running the latest version available as of March 2024.

Low CPU utilisation

One feature I really enjoyed is the extremely low CPU utilisation under load. I saw slower 2.5 GbE adapters hammer CPU with interrupts, but that’s not the case for this NIC. AQC107 does really good job at keeping the CPU cool.

Low Raspberry Pi 5 CPU load under network load

Cable analytics

Marvell supports Cable Diagnostics feature which uses TDR to measure cable length and detect Ethernet cable for defects. Unfortunately, it doesn’t seem to be supported on the AQC107 chip.

Can you get 10 Gbps out of this adapter at all?

I am glad you asked. How does an Intel NUC with this 10 GbE adapter sound? I’ve just tested it, here you go.

Summary

The high operating temperature really makes this adapter something I can’t recommend. With maximum throughput below 3.5 Gbps, I think you would be better off choosing a 2.5 Gigabit Ethernet adapter, which runs cool and delivers 2.35 Gbps/2.35 Gbps throughput.

Have you tested any other 10 GbE adapter? Did you get better results? Did you find any 2.5 Gbps Ethernet adapter that supports Cable Diagnostics? I am all ears.

Special thanks

Thanks to Luke Jenkins for exploring and sharing the kernel build instructions. Also, thanks to the WLAN Pi team. You can buy the team a coffee using this link.

Throughput speed test of the fastest tp-link and Devolo Magic 2 Wi-Fi power line adapters (PLC)

I am in the market of buying a new pair of power line adapters. Power line is a great alternative or complement to Ethernet and Wi-Fi. It provides low latency and jitter and is very flexible and easy to install.

The current tp-link TL-PA6010 adapters have served me well, but they are now reaching their maximum throughput. So, I decided to get a new pair of the fastest adapters on the market (Devolo Magic 2 Wi-Fi) and also a pair of the best adapters from tp-link (TL-PA9020P). These will be used to connect my home office and lab networks to my router.

Since there are multiple brands offering a variety of products with a variety of advertised speeds, I am curious to see if the more expensive adapters are worth the premium price, what real throughput they would provide and if and how much a passthrough socket improves the power line speed.

Left to right: Devolo Magic 2 Wi-Fi, tp-link TL-PA9020P, tp-link PL-PA6010 (not sold anymore, this would be an equivalent)

Specification

I tested my current low-end adapters and two new high-speed ones:

Throughput, ping, jitter, power and Wi-Fi tests

Power line speeds vary and depend on the distance between the two adapters, your electrical wiring and interference. Please take the numbers below as relative ones, which would allow you to compare how these adapters perform under the same conditions and in the same setup.

All throughput numbers below were TCP measurements taken by iPerf3 running on a WLAN Pi (a single-board computer with 1 Gbps Ethernet) and the client was my MacBook with 1 Gbps USB-C Ethernet adapter. There were no intermediate network devices between them:

MacBook iPerf3 client <-> PLC1 <-> PLC2 <-> WLAN Pi iPerf3 server

The average download speed (measured 5 times at each of the locations in my house) ranges from 13% to 26% of the advertised speeds and goes nowhere near them. With £16 per 100 Mbps, the cheapest adapter seems to be the best value for money, unless you need higher speed and are willing to pay for it. It also is the most power efficient.

Devolo Magic 2 proved to the be the fastest solution with 331 Mbps average download speeds, while TL-PA9020P provided slightly better upload speeds than Devolo.

Each of the parameters (i.e. Download average) consisted of five iPerf3 tests in each location and I then computed the average values:

Built-in Wi-Fi access point

Devolo Magic 2 Wi-Fi remote adapter comes with a built-in dual-band 802.11ac Wi-Fi AP (not just a repeater as some of the cheaper adapters), but it is unstable and resets the power line connection every single time I connect and generate some traffic. I used the latest firmware available in July 2020. If a built-in Wi-Fi is a must-have for you, do NOT buy this adapter. Wait until it gets fixed or look for alternatives.

This is what happens. The SSID is broadcast, a Wi-Fi client can associate to the AP, but when the iPerf test starts, the client gets disconnected and power line connection is torn down for 10 seconds or so and then re-establishes. I was able to reproduce this bug every single time and it was not just one-off random problem.

On the positive note, it supports 2.4 GHz only, 2.4 + 5 GHz or 5 GHz only modes. It does not let you change channel width on 5 GHz though and always uses 80 MHz, which may sound like a good idea in a small town, but it is a disaster in a shared building with many other access points and neighbours present.

If high-speed power line without Wi-Fi is what you are after, then the Magic 2 non-Wi-Fi model could be a good option for you.

Passthrough socket

Passthrough socket allows you to plug an electrical appliance to the power line adapter without generating the socket your adapter is plugged into unusable. Cheaper adapters usually do not provide this.

The other benefit is that adapters with passthrough socket use filters to suppress noise coming from the connected electrical appliance and this improves speed by 13% – 15%.

Pros and cons

Devolo Magic 2 Wi-Fi
+ Fastest average download speed
+ Comes with a mobile app and each unit has a management web GUI
– Built-in access point resets the whole unit and Wi-Fi is not usable
– It runs quite warm compared to the other two and is the largest

tp-link TL-PA9020P
+ Very good and symmetrical performance
+ Stable
– No built-in Wi-Fi
– Still quite expensive compared to the slower and cheaper units

tp-link TL-PA6010 (or similar)
+ Great value for money
+ Stable
– Relatively low speeds
– No passthrough socket, no Wi-Fi

And the winner is…

My personal preferences are very likely different from yours and that is fine. I am looking for symmetrical TCP throughput of at least 200 Mbps, ideally a passthrough socket support and all other features are nice to have.

Devolo Magic 2 Wi-Fi proves to be unstable as the built-in access point crashes the whole adapter and resets the power line connection. Its back side also becomes quite warm regardless the load.

So, I decided for tp-link TL-PA9020P. It is stable, does all I need it to do and both adapters come with 2 Ethernet ports which gives me flexibility to plug my own access point in or connect using wired Ethernet connection.