Periodically, we hire Alethea Communications to perform third-party testing for our Access Point (AP) products. Alethea specializes in network stress testing services and products. They use various test methods with simulated and real client devices. This test used real client devices in a realistic environment with up to 100 devices. This method gives very reliable data for real-world performance.
This test focused on high-end 4×4 APs with one 2.4 GHz radio and one 5 GHz radio. The tested APs and their costs are summarized below.
Real Client Devices
In the previous tests, the client base consisted mostly of devices using the Intel Wi-Fi chipset. In this round, Alethea expanded the client base. They performed tests with a realistic client mix using the latest available drivers. The following are examples of test clients used:
- Dell Latitude E7470 with Intel 8260 chipset and Ubuntu 16.04
- Dell Latitude E7470 with Intel 8260 chipset and Win 10
- iPhone XS, iPhone 6S, iPhone 6, iPhone SE, Apple Mac Mini
- Samsung Galaxy S9, S8, S7, S5, Tab A
The number of simultaneous clients performing tests were 1, 32, and 100, depending on the test case.
Realistic Test Environment
Alethea used a classroom environment for the test. They positioned the test AP in the center of the area. All testing was done on the 5 GHz band and the 2.4 GHz radio was disabled. Alethea used the 80 MHz channel for single-client tests and 40 MHz channel for 32- and 100-client tests.
The nominal AP power level was +20 dBm. The test team calibrated average received AP signal level at the client to the same level, approximately -43 dBm, for all of the tested APs. AP power level settings were adjusted to equalize the differences.
Reasons for the calibration procedure:
- APs have different antenna patterns. Some are focused more downward and others more toward the sides. In a single AP test like this, the AP with the greatest gain to the sides would have an advantage when there are many clients further away from the AP. In a high-density environment, a more downwards-focused pattern causes less interference to nearby APs. The more downwards-focused pattern provides better network level capacity and is more suitable for high-density environments.
- The principles for power level measurements and settings may vary between vendors.
- Ruckus does not provide certain absolute dBm scale output power settings, only power level reduction values (dB) from maximum.
The measured signal level data shows that APs set to the same power level had significant differences in power levels as measured by clients. This calibration step was important to achieve realistic and comparable results in the targeted application.
Test data presents the aggregate throughput from all test clients. DL and UL tests were performed separately. In the DL test, all clients performed a simultaneous downlink transfer. Similarly, in the UL test, all clients performed the UL transfer test.
Throughput with 1 Client
UAP-AC-SHD performs the best in both directions according to these test results.
Throughput with 32 Clients
Cisco 3802i performs the best in the DL test; Mist AP41 performs the best in the UL direction. Note the drop in aggregate UL throughput compared to the same in the DL direction.
Throughput with 100 Clients
Cisco 3802i performs the best in the DL direction. UAP-AC-SHD is the best in the UL direction in this test. During this test, Cisco 3802i performance decreased considerably (3.5 Mbps aggregate, 0.035 Mbps/client) in the UL direction with 100 clients. It’s worth noting how UL performance is significantly lower than DL performance when the number of clients increases. Several factors contribute to this, including transmitter power level, efficiency of channel access, and Wave 2 MU-MIMO in the DL direction.
Video Streaming Performance
Cisco 3802i had the best performance, followed by UAP-AC-SHD and Ruckus R720.
Mixed Traffic Performance
A mixed traffic test simulates a real network. The AP needs to balance between various client types, signal levels, and various applications. Alethea performed the tests with default QoS settings.
The mixed traffic test included simultaneous traffic:
- Downlink transfer with 24 clients
- Uplink transfer with 24 clients
- 12 VoIP calls between 24 clients
- 24 clients running video download with 1 Mbps codec
Below is data for each group of clients performing tests with a certain application.
Throughput During Mixed Traffic
Voice Quality During Mixed Traffic
Video Streaming During Mixed Traffic
Ruckus R720 and UAP-AC-SHD performed the best in this test. Meraki performed well in voice quality and throughput tests, but it failed all video streaming tests. Aruba had good video streaming performance, but only a modest simultaneous throughput.
Alethea used a five-grade scale for the overall scoring. The highest and lowest values of each test determined the full range, which they further divided into five equal sub-ranges. Depending on the test results, each AP got a score from 1 to 5 for each test.
According to test data, UAP-AC-SHD scored the highest overall, followed by Cisco 3802i. While this may be surprising for some readers, it’s not surprising for us. Performance is an important long-term focus area for our development teams.
When selecting the best product for your network, you are likely to consider the balance between performance and cost. I calculated the cost of performance by dividing the cost by the overall score.
In addition to the leading performance, UAP-AC-SHD provides exceptional value. The cost of performance is only 18-32% of a competitor’s equivalent.
I took this test data from Alethea’s raw data tables and created the summaries in this blog post. Alethea has also published the comprehensive test reports. You can view the reports here. They contain a lot of interesting details and findings which go beyond the information presented in this blog post. For example, it is very interesting to observe the performance differences between client devices with different operating systems.
Ubiquiti offers a large variety of Wi-Fi access points for different use cases. You can get a quick overview of all UniFi APs from this knowledge base article.
You may be pleasantly surprised by the performance, value, and ease of use Ubiquiti products offer. I encourage you to give them a try.