Just a very quick reminder that if you power your CW AP using an incompatible 802.3at power injector, you will likely see the AP successfully boot up, but it disables its radios few seconds later. The result is no SSID put on the air.
What to look for in the logs?
If you console or SSH into the AP, you will see this error message. Followed by radio interfaces going down.
set_sys_cond_state: condition critical state 4
condition critical state 4 error message
That’s it. Use officially supported injectors, and save yourself from the trouble I ran into 😊
This journey started as an exploration of maximum PCIe capabilities of Raspberry Pi 5 (and hopefully Compute Module 5) platform. I am mainly interested in multi-gigabit Ethernet and Wi-Fi 7 adapters connected via the PCI Express (PCIe) x1 bus.
This time we are going to use a slightly different adapter. It is available from various sellers under different names, but they all look and work the same. I picked up one from “KALEA-INFORMATIQUE” which happened to be readily available in the UK.
Pineberry’s HatDrive! Bottom breaks out Raspberry Pi’s PCIe connection to M.2 M-key format, and that’s where this 10 Gigabit Ethernet adapter plugs into.
Raspberry Pi 5 with 10 GbE adapterDetail of the Ethernet adapter
We have connected everything, built a custom kernel, we can see the device, but the Ethernet interface is not coming up.
lspcilspci -v output
Look at this official product photo and my photo below. Spot one difference 😉
The official product photoThe actual correct setup
Did you notice the orientation of the white ribbon cable? The official photo got it wrong. The printed text on the cable needs to be on the top on one side, and on the bottom on the other one.
The eth1 interface and its IP details
What speeds did you get in PCIe Gen 2 mode?
After correcting the orientation of the flexible cable, the interface came up, negotiated 10 Gbps full duplex.
In PCIe Gen 2 mode, we got TCP throughput of 3.45 Gbps on the downlink and 3.07 Gbps in the upstream direction. Using more iperf3 parallel streams did not increase performance.
Downstream throughputUpstream throughput
Were you able to use PCIe Gen 3 mode?
Yes! And I got 4.63 Gbps of TCP downstream and 5.5 Gbps (potentially up to 6 Gbps) upstream.
PCIe Gen 3 downloadPCIe Gen 3 uploadI managed to get up to 6 Gbps in the upstream direction
That’s hot news… yes 122° Celsius hot!
This adapter has a thermal problem. It comes with a heatsink, but even in idle mode it overheats.
Detail of the heatsink107.7° C in idle
In PCIe Gen 3 mode with iperf3 test running, we are talking 122.1° C hot! The Pineberry board was very hot and you can literally burn your fingers by touching the heatsink.
122.1° C hot under loadIn PCIe Gen 2 mode, it ‘only’ runs at 96.4° C
Long story short. Don’t buy this adapter, unless you want to add a fan or significantly larger heatsink.
Toaster, 10 Gigabit adapter, aren’t they the same thing?
Make your own opinion based on these couple of thermal photos.
Thermal IR footprint of the Ethernet adapterAnd here is a toaster for scale 😅
This Ethernet adapter as well as the OWC 10 GbE Thunderbolt both use the same Aquantia AQC107 (part of Marvell now) chip. It does really good job at keeping CPU utilisation low. I’ve seen much cheaper 2.5 GbE adapters that hammer CPU with interrupts until the CPU just can’t take no more.
But, compare size of the two heatsinks. Unlike this one, the OWC adapters delivers good thermal results. Don’t take me wrong, it still runs warm, but not anywhere near.
Same AQC107 chip, massive thermal mass difference
Summary
On the positive side, this is the first 10 Gigabit adapter I tested which actually worked in PCIe Gen 3 mode on Raspberry Pi 5. I got TCP throughput of up to 6.0 Gbps.
As far as I can tell, the limit of Raspberry Pi 5’s PCIe bus is around 6 Gbps if you look at it through the iperf3 TCP traffic lens. AQC107 silicon does an amazing job at keeping the Raspberry Pi’s CPU utilisation low. This helps us get as much throughput as we can from the Pi. But it produces a significant amount of heat.
The fact is that this adapter overheats. Don’t buy it unless you wish to use it with a fan or design a much larger heatsink yourself.
Intel NUC 12th generation with Intel BE200 Wi-Fi 7 adapter
Drivers
Download and install latest driver from Intel’s website. Windows Update itself won’t install any driver, so some manual steps are required. I originally tested driver version 23.20.0.4, and now updated to 23.30.0.6.
Setup
We are using TP-Link Deco BE85 BE19000 consumer Wi-Fi 7 router connected to a 10 Gigabit iperf3 server running on MacBook connected via OWC 10 GbE to Thunderbolt adapter. We have done this on Linux before, so let’s see how the same Wi-Fi adapter performs on Windows.
Wi-Fi 7 on Windows 11 test topology
Performance
On the router, we have configured and verified 320 MHz wide Wi-Fi 7 channel. But when we connect the Windows client, looking at the data rate, it is surprisingly low – if you forgive me calling 2882 Mbps ‘low’ 😊 Considering that the NUC is about 1 meter away from the router, I would expect ~5 Gbps data rate. So what’s going on here?
Connected Wi-Fi 7 client, but low data rate
Interestingly enough, it is the same data rate as we see when connected using 5 GHz 160 MHz channel. Yes, I know, that’s a no-no in Wi-Fi design. We are just testing here.
Hmm, 160 MHz wide 5 GHz channel gives us the same data ratenetsh wlan show interfaces command output
Since Windows doesn’t expose the channel width in the UI, we don’t quite know what is happening on the air. Let’s generate some 6 GHz traffic, and check using Oscium’s WiPry Clarity tri-band spectrum analyser. I love this little USB tool. In this example I use a WLAN Pi as a Remote Sensor. It scans for Wi-Fi networks and streams spectrum information to WiFi Explorer Pro on Mac.
WLAN Pi Remote Sensor with Oscium WiPry Clarity scanning 6 GHzWindows client is using 160 MHz instead of 320 MHz
Bingo! Apparently, on Windows Intel BE200 uses 160 MHz channel width and doesn’t support 320 MHz wide channel. That halves our data rate and throughput. I wish Windows made channel width more obvious in the UI. Intel BE200 adapter supports 320 MHz wide channels on Linux without a sweat, so hopefully it will get fixed in a future Intel driver or Windows release.
Updated: Apparently, I didn’t read Intel’s release notes closely enough, my bad 😊 Intel BE200 adapter on Windows 11 is only able to use Wi-Fi 6E today. Windows 11 will introduce Wi-Fi 7 support in a future update. Since Wi-Fi 6E supports channel widths up to 160 MHz, that’s why we are not being able to use full 320 MHz channel width. What really confused me was the “Protocol: Wi-Fi 7 (802.11be)” misleading Wi-Fi network status reported by Windows. Thank you Ben for spotting the note in Intel’s documentation.
What does that translate to? Lower data rate and lower throughput. I would expect download and upload to be around 2.5 Gbps using 320 MHz wide channel. With the latest Intel driver 23.30.0.6, we get 1.71 Gbps TCP download speed with 16 parallel streams, and upload of 2.17 Gbps. But only the upper 160 MHz half of the 320 MHz wide channel is used.
1.71 Gbps TCP download speed with 16 parallel streamsUpload TCP speed 2.17 Gbps with 16 streams
I also ran a quick Speedtest.net test (I know it is not a proper throughput testing tool) on a 900/900 Mbps WAN link.
On a Linux Wi-Fi 7 client, I measured nearly 890/890 Mbps. Original Intel driver 23.20.0.4 performed 383/818 Mbps. The latest Intel driver 23.30.0.6 delivered more symmetric numbers, and results were closer to the actual WAN link speed.
Speedtest.net speeds using the latest 23.30.0.6 Intel driver
Summary
Wi-Fi worked well, but application speeds including Speedtest.net and other tools performed quite poorly and subjectively ‘felt slow’. iperf3 test showed higher performance, but the main problem for the purpose of a throughput test is that the adapter only uses 160 MHz out of the available 320 MHz.
When it comes to recommended channel width in real world, it depends. 80 MHz or 40 MHz wide channels are most likely the best place to start depending on your circumstances and region.
For reference: Disable 6 GHz on Intel BE200 adapter
If you are performing tests on an SSID that has multiple bands enabled, and you want to force the client to drop off 6 GHz and join using a 5 GHz channel instead, Intel BE200 driver has the option to disable the 6 GHz band.
Earlier this week, I tested a 10 Gigabit Ethernet M.2 network adapter on Raspberry Pi 5, and it didn’t quite cut it. Mainly due to limited PCIe Gen 2 performance. Now, the question is can this 10 Gigabit adapter actually push 10 Gbps of traffic at all?
To find out, we are going to slightly reconfigure this Intel NUC 12th generation mini PC. It has an M.2 M-key slot for NVMe drive. Let’s use this slot for our 10 GbE adapter. And we will boot Windows 11 off an external USB SSD drive.
Remove NVMe from the M.2 slotInstall 10 GbE network adapter insteadIntel NUC with 10 GbE adapter connected to 10 GbE switch
Install Windows 11 23H2 version on a USB SSD drive, boot Windows, run Windows Update, voila!
Windows Update installed latest driver automaticallyIt uses PCIe Gen 3And x2 link width10 Gbps Full Duplex
With default iperf3 settings we get 6.44 Gbps/7.93 Gbps in the downlink and uplink direction respectively. Not bad, but is that it? Of course not.
iperf3 with default settings
I don’t really want to enable Jumbo frames as it’s not always possible to enable Jumbo frame support end-to-end, especially if part of the network doesn’t support it or isn’t under your management. Fortunately, 8 parallel TCP streams in iperf3 do the trick for us. We get 9.48 Gbps download speed.
9.48 Gbps download
In the upstream direction from this NUC to my MacBook with 10 GbE adapter, we also get 9.48 Gbps. I am happy. You? 😉
9.48 Gbps upload
Summary
After all, this 10 GbE M.2 network adapter is indeed capable of pushing 9.48 Gbps of traffic in either direction. But! It is not really a good choice for a system like Intel NUC. I can’t pop the lid back on, the heatsink is too tall. Frankly, I can’t recommend this adapter at all. It runs hot at 84° Celsius in idle.
If you are looking for a daily driver, and your system supports Thunderbolt, get yourself this OWC 10GbE to Thunderbolt adapter. Here is my test. It works out of the box on Windows (I tested this Intel NUC 12th Gen) and macOS (I tested MacBook Pro M1 and M2). Interestingly, it uses the same chip as the above M.2 adapter. Just compare the two products and their heatsink sizes. The AQC107 keeps the main CPU utilisation very low, but it produces a significant amount of heat.
OWC 10 GbE to Thunderbolt network adapter connected to Intel NUCOWC 10 GbE to Thunderbolt network adapter connected to MacBook
WLAN Pi is primarily a Wi-Fi tool, but occasionally I need an iperf server that would be able to deliver more than 1 Gbps of TCP throughput. In a controlled lab environment, I normally use PoE powered NanoPi R5S. I know the IP address of the iperf server by heart. Outside of the lab, I could really do with a WLAN Pi, its preinstalled software, display, buttons and everything it does out of the box. So the question is: “Can we add 2.5 GbE to WLAN Pi M4?”
M.2 slot to the rescue
WLAN Pi M4 doesn’t have any USB 3 ports. How do we add 2.5 Gbps Ethernet to it? If you don’t mind losing the Wi-Fi adapter in favour of 2.5 GbE mGig port, we can install this 2.5 Gbps Ethernet adapter in M4’s PCIe M.2 slot. It is based on Realtek RTL8125B chipset. I paid £17 for it including shipping to the UK.
To my surprise, it just works*. Yes, I hear you, no one likes these asterisks, do you? 😉 Continue reading, it’s not the end of the story.
WLAN Pi M4 with 2.5 Gbps Ethernet2.5 Gbps full duplex
The underwhelming default driver
Linux (and WLAN Pi image) has a driver for this adapter, but upload speeds, that is from iperf client to WLAN Pi iperf server, are very poor. We are talking 300 Mbps poor.
Poor 300 Mbps upload speed
Install Realtek’s latest driver to fix performance
Downloading, compiling and installing the latest Linux driver from Realtek’s website fixes the performance issue. We get symmetric 2.35 Gbps of TCP throughput with standard packet size.
2.35 Gbps of iperf3 TCP throughput
Installation of this driver isn’t as straightforward as it might look. I ended using vanilla Raspberry Pi OS image instead of the WLAN Pi one. Mainly because it is not easy to get the kernel headers for WLAN Pi image and we need them to be able to compile the new driver.
Summary
Yes, it is possible to achieve 2.35 Gbps symmetric TCP throughput on the WLAN Pi M4 with this adapter. But you should be aware of these facts:
This Ethernet adapter doesn’t fit inside WLAN Pi M4 case
You will have to give up the M.2 Wi-Fi adapter in favour of mGig Ethernet
From software perspective, the Realtek driver that ships in WLAN Pi image doesn’t unlock full performance of this adapter (iperf client pushing traffic to WLAN Pi iperf server). Installing the latest driver isn’t trivial on WLAN Pi.
We, WLAN Pi team, currently don’t support this setup. If you have a use case for 2.5 GbE support on the M4, please let us know.
Sabrent NT-SS5G is a 5 GbE USB adapter, which allows you to achieve higher throughput than 2.5 GbE adapters, and break the 2.35 Gbps barrier. It works great on Windows. If you are a macOS or Linux user, I recommend you consider other options like this instead.
The adapter itself is larger than 2.5 GbE adapters, it uses AQC111U chip, and ships with short 2 detachable USB-A and USB-C cables. USB-C port on its back connects the adapter to your computer. A metal shell protects it, serves as a heatsink, and also adds to its weight.
Windows 11
Install the driver from Sabrent’s website and you are good to go. In my tests with this Topton M6 Mini PC, I measured 2.93 Gbps down and 3.44 Gbps up with default iperf3 settings.
2.93 Gbps down and 3.44 Gbps up with default iperf3 settings
In adapter options, you can actually configure quite a few things including Jumbo frame support. Note that these are fixed values.
macOS
I can’t recommend this adapter for macOS users. It forces you to disable macOS System Integrity Protection (csrutil), otherwise it won’t work. It might be okay for a proof of concept or lab setup, but I would hesitate from using it in production.
This is how to install the driver if you were interested:
Install the driver using the pkg file provided by Sabrent. It installs a Kernel Extension (kext), which drives this adapter.
Enable the extension by going to System Preferences > Security & Privacy > enable the extension > Reboot.
After reboot, unplug the adapter and plug it back in.
It should work as long as you leave the System Integrity Protection disabled.
From throughput perspective, it saw download speeds of 3.30 Gbps, and upload of 3.45 Gbps. This was with default iperf3 settings, standard 1500-byte MTU and one stream. Great results considering that this adapter’s USB interface maximum theoretical throughput is 5 Gbps.
In my view, you might be better off buying a 2.5 GbE adapter, which can push 2.35 Gbps up and down consistently and with no driver installation needed. I tested one here. Alternatively, a 10GbE Thunderbolt Ethernet adapter is even faster choice, but more costly, and larger form factor. Or, if your other half approves, treat yourself to an M1 Mac Mini with built-in 10 GbE 😉
Linux
I tested this adapter on 64-bit Raspberry Pi OS running on Raspberry Pi 4. Although the default driver distributed in Linux Kernel 5.15 works, it doesn’t even deliver symmetric 1 Gbps.
Sabrent connected to Raspberry Pi 4Upload speeds well below 1 GbpsDefault aqc111 driver details
Let’s download the latest driver from Sabrent’s website. Unfortunately that doesn’t seem to be able to compile for 64-bit OS. I tried compiling on 32-bit Raspberry OS, to no avail. If you have any ideas, please do let me know.
So, on Linux, a Realtek RTL8156B based 2.5 GbE adapter might be a better choice for you. Here is the one I tested.
Plugable makes this inexpensive 2.5 Gigabit Ethernet USBC-E2500 adapter. It is based on Realtek RTL8156B chip. On Windows and macOS it works out of the box. If you want to use it on a Linux machine like WLAN Pi Pro or Raspberry Pi 4, expect some troubles along the way, but good performance when you get there.
The USB-C to USB-A adapter is allows you to use it with a MacBook (USB-C) or Raspberry Pi 4 (USB-A)The adapter itself has a plastic shell and is very lightweight
Windows 11
When they say “update the driver using Windows Update first”, they mean it. Windows 11 will recognise the adapter and you can start using it, but the default driver distributed with Windows 11 significantly reduces this adapter’s performance.
727 Mbps down and 2.34 Gbps up with default driver
Now, let’s use Windows Update to download the latest driver.
Don’t forget to update the driver using Windows Update
As you can see, download throughput (from iperf3 server to iperf3 client) has dramatically improved.
1.78 Gbps down and 2.35 Gbps up with updated driver
Although the box suggests Jumbo frame support, Windows driver settings don’t give me any option to edit the MTU size. So, I assume Jumbo frames are not supported.
MacOS Monterey
On macOS, this adapter works out of the box with no additional driver installation required. That’s a very nice surprise. And performance is great.
Symmetric 2.35 Gbps throughput on macOS
Auto-negotiation worked just fine. If you want to configure speed or MTU manually, you can, but Jumbo frames are not supported on macOS either.
Jumbo frames are not supported
Linux
Now the bad news. If you are considering to use this adapter on a Linux machine, the default driver cdc_ncm is a trouble as it only supports 2.5 Gbps Half duplex. Setting Full duplex manually using ethtool command doesn’t work either.
Default driver only supports Half duplex
As you might expect, with the default driver and Half duplex, throughput is very poor.
1.22 Gbps down and 704 Mbps up with the default cdc_ncm driver on WLAN Pi Pro
On WLAN Pi Pro and Raspberry Pi 4 running 5.15 Linux Kernel I managed to fix the duplex issue by the steps listed below. But I hit new auto-negotiation issue between the Plugable adapter and Cisco Catalyst WS-C3560CX-8XPD switch. It took the adapter to eventually negotiate 2.5 Gbps Full duplex around 15 minutes of constantly flapping the interface. Forcing speed and duplex on the Plugable adapter by ethtool did not work. Certainly not ideal, and definitely worth testing before you commit to the Plugable adapter. With other multigigabit adapters, the Plugable had no negotiation issues.
1.7 Gbps down and 2.09 Gbps up with r8156 driver on WLAN Pi Pro1.91 Gbps down and 2.06 Gbps up on Raspberry Pi 4 using the correct r8156 driverRaspberry Pi 4 also known as WLAN Pi Community Edition
How to force Linux to use the right driver
To enable Full duplex capability, we need to tell Linux to use Realtek r8156 driver instead of the default cdc-ncm.
After using FriendlyElec R5S single-board computer as a portable iperf3 server, I decided to also order and test Topton M6 Mini PC. It is more powerful, based on Intel CPU, and runs Windows 11 Pro or Ubuntu. I personally chose Windows (yes, I am brave), mainly because I also wanted to use this device as a Windows Wi-Fi client for other things than iperf3 testing.
Dimensions and case
Compared to the R5S, Topton M6 Mini PC is still portable, but about twice as large. Plastic case wraps the unit, but it is more fragile if you plan to carry it in your backpack or tool bag. There is a built-in fan which is always on. Not a big deal if you use it as an perf3 server, but little inconvenient when it runs on your desk for a longer period of time.
iperf3 performance
Topton M6 has a single onboard 2.5 Gigabit Ethernet port consistently capable of 2.35 Gbps up and down iperf3 throughput with default settings.
Consistent 2.35 Gbps iperf3 throughput
Now, can we make it go faster? Let’s see. We will use USB-A 5 Gigabit Ethernet capable Sabrent adapter. This can either be connected to a USB-A port or USB-C port of the Mini PC. In my tests, I have found that the USB-C port has limited throughput and only tops around 350 Mbps. When I connected the Sabrent 5GbE adapter to USB-C, it only auto negotiated 1 Gbps Full Duxplex.
Use any of the three USB-A 3.1 ports instead to avoid that limitation.
Use USB-A ports, not the USB-C With the USB adapter, we get 2.94 Gbps down and 3.27 Gbps up
With the USB adapter, the whole setup get less portable. But it allows us to achieve 2.94 Gbps down and 3.27 Gbps up from clients perspective. Is it worth the extra spend? If you need to break the 2.35 Gbps barrier of the built-in 2.5 GbE port, this might be a workable solution for you.
Power adapter with an adapter
This Mini PC is quite strict when it comes to its power source. It requires 12V/2A USB-C PD adapter. Unfortunately, your USB-C MacBook or iPad chargers won’t work.
It draws around 7.5 Watts in idle mode.
If you happen to only use this PC in the US, happy days, as the power adapter ships with US plug. If you select UK during the ordering process, you will receive the US power adapter with UK adapter, which adds to its overall size.
My way around this is to use a standard non-USB-PD 12V/2A adapter with 5.5×2.1mm barrel jack connector, and a barrel jack to USB-C adapter. This particular “power brick” has a standard IEC C14 power cable connector, which you can find in any data centre and with the right European, UK, or Australian plug.
Power adapter with barrel jack + barrel jack 5.5×2.1mm to USB-C adapterDetail of barrel jack 5.5×2.1mm to USB-C adapter
Battery power
Simply use a USB-C cable and USB PD battery pack capable of delivering 12V/2A. No surprises there.
Powered by PoE
I prefer powering equipment using PoE over local power bricks. If you are in the same boat, you can power this Mini PC by a PoE splitter.
Please pay close attention to the splitter specs. We want the one with a barrel jack and 12V/2A. Since the Mini PC uses USB-C power connector, we will use a barrel jack 5.5×2.1mm to USB-C adapter. Here is the complete setup. Press the power button and voila!
Under the hood
Most of the components are soldered to the main board with little room for upgrades. I ordered the lowest 8GB DDR4 and 128GB NVMe spec with Windows 11 Pro OEM preinstalled (no actual Windows license included).
I was hoping for the Wi-Fi adapter to be replaceable, but it is not the case. It is Intel AX201 and soldered to the board. Good enough, just not ideal for Wi-Fi professionals. M.2 slot would be ideal.
A quick look at the bottom side of the PCB shows the NVMe drive.
NVMe drive is practically the only replaceable component
Final verdict
Personally, I think this Mini PC has some great potential for certain use cases, but as an iperf3 server, I would rather use the FriendlyElec NanoPi R5S I reviewed here.
If you absolutely need to break the 2.3 Gbps barrier, it can be done with the help of a USB 5 GbE adapter, but it is not very cost effective. The Mini PC cost me £186 including shipping to the UK. The Sabrent 5GbE USB adapter costs around £65.
Finally, it you need top performance, don’t care that much about small form factor, and money is no object, the latest Apple M1 Mac Mini can be configured with built-in 10 GbE.
