More Than Throughput: Next Generation Wi-Fi Testing with Ixia's WaveDevice

Closing Thoughts

As mentioned in the introduction, we've always been faced with the problem of seeing subjective differences in RF performance between devices, but lacking the data and repeatable tests to back it up. In my time at AnandTech, I've always been working to try and improve our reviews, and RF testing has been one of the major areas where I've sought to improve our reviews and benchmarks. While we've had some basic tests, we've never gone into this area with the same level of depth and breadth that we have with many other components of the stack. As connectivity is probably the most important thing in computing, it was evident that we had to tackle this subject at some point.

Our first leap into this area is the addition of Ixia's WaveDevice to our test suite. From the start, this system was conceived as an all-in-one chassis that could give data to prove or disprove subjective observation, to bring repeatable testing to seemingly one-off edge cases, and to do so at scale for Wi-Fi. It turns out that this system is quite powerful, and can show how a device performs in tests that directly correlate with user experience. These tests include throughput with respect to range, roaming latency, and ecosystem performance. The rate versus range test shows the quality of the RF front-end and the ability for the modem to properly decode and encode data in the face of decreasing SNR. The roaming latency test shows how well a device can detect and react to changing reception conditions. The ecosystem performance test can show how well a device can acquire the channel without conflicting with other traffic.

In the case of the iPad Pro and Pixel C, we found that WaveDevice was able to show a number of notable interesting data points from both an end user perspective and an engineering perspective. With the rate vs range test, it was possible to clearly see how well a device would perform in response to worsening reception from a user experience perspective. From an engineering perspective, it was possible to identify the root cause for the Google Pixel C’s poor Wi-Fi performance by using WaveAnalyze and an RF analysis blade in WaveDevice. While determining the root cause is still beyond what we can do with limited information on the hardware, an OEM would be able to act on the information provided by WaveDevice to improve their product before it reaches mass production.

In addition to the rate vs range test, the roaming latency test was quite illuminating. While root cause analysis is more difficult and best left to actual engineers, it’s quite obvious that the iPad Pro passed this test with flying colors while the Pixel C shows some serious deficiencies. If you regularly encounter large Wi-Fi networks with multiple access points all under a single SSID/name like eduroam, it’s obvious that the Pixel C will be an exercise in frustration if you’re hoping to keep a working Wi-Fi connection on the move. Even when the device roams successfully, the time that the device spends moving from one access point to the next is long enough on average to result in noticeable connection interruptions. When it doesn’t roam successfully, it seems to get stuck on a single access point and basically drops off the network entirely without manual intervention or has to re-authenticate and acquire a new IP address, which is guaranteed to cause most traffic to be dropped.

Of course, while this data is interesting, it's not very helpful without an understanding of how Wi-Fi works. Starting from the physical link layer, pretty much every modern radio in a mobile device is going to have a superheterodyne radio architecture, which uses an intermediate frequency before stepping down to a baseband frequency that allows for better signal processing. There’s a lot of bits and pieces here, but the key component here is the local oscillator that allows for successive stages where a baseband signal is encoded into a carrier or decoding a carrier to the baseband signal. Everything else is basically a lot of complicated circuits that are designed to help tune the RF circuits to oscillate at the correct frequencies and to boost a signal’s power multiple orders of magnitude from its baseband state.

With this radio as the foundation, we can then focus on modulation and coding schemes. Wi-Fi uses two primary methods of maximizing throughput as close to the Shannon limit as possible. These two methods are known as Quadrature Amplitude Modulation (QAM) and Orthogonal Frequency Division Multiplexing (OFDM). OFDM is basically a method of slicing up spectrum into small subcarriers. When done correctly, there are a number of benefits from a design perspective like simpler radio design, high spectral efficiency, simpler signal processing algorithms, and improved interference immunity.

If you can think of OFDM as slices of frequency, QAM is used on each slice of frequency. By using phase variation and amplitude variation, it becomes possible to represent multiple bits with a single frequency slice. In the case of Wi-Fi and LTE, we’re looking at up to 8 bits per “slice”, which means that there are 256 potential combinations of phase and amplitude to consider. However, noise limits the ability for a receiver or transmitter to differentiate between these combinations, so depending upon the channel conditions it may be necessary to increase the difference between each state to avoid data corruption.

The final method worth noting that can improve performance is MIMO. At its heart, MIMO is a form of parallelism to improve bandwidth and/or range. By exploiting the fact that signals will often have multiple propagation paths, it becomes possible to use these multiple paths to send multiple streams of data simultaneously. When taken to its logical conclusion of MU-MIMO, it’s possible to see additional throughput advantages as the device and access point can utilize beamforming to focus transmissions to reach a specific location rather than transmitting in all directions.

All of these aspects taken together form the physical link layer, which is best understood as the base hardware mechanics. The next layer up is the data link layer. This layer is used to help abstract away the underlying mechanics of all networking technologies so that the layers further up the networking stack don’t have to be tailored towards any one type of network technology. For the purposes of our reviews and understanding Wi-Fi, the key area of interest here is the method used to emulate a full-duplex network with a half-duplex technology. Full duplex in this case means simultaneous transmission and reception, while half-duplex only allows for transmission or reception, not both at the same time.

In the case of Wi-Fi, this emulation method is known as Carrier Sense Multiple Access with Collision Avoidance, or CSMA/CA. At a high level, devices listen to the channel to wait until it’s clear before sending the access point a request to send, at which point the access point must respond with a clear to send before the device can transmit on the channel.

In addition to multiplexing, the MAC layer determines how to control the physical link layer, which includes selection of modulation and coding schemes and power management of the radio. Due to the nature of Wi-Fi, the device has to use a number of heuristics to determine what kind of power save mechanisms and link rates to use rather than specific algorithms like traditional cellular networks. Of course, the MAC layer is also needed to determine how to route traffic and to transparently handle errors or corruption, but this is beyond the scope of this article.

Overall, reaching the level of understanding with regard to theory and practice has been one of the biggest undertakings that we’ve ever had at AnandTech. As I mentioned earlier, wireless testing has been one of the major frontiers that we’ve yet to fully explore. It turns out that digital logic and computer science don’t help much with understanding RF, and as a result something that might have seemed simple from our iPerf tests has exploded into months of research and experimentation. I’d like to thank Ixia for providing their significant expertise and equipment. More importantly, I’d like to thank all of our readers that have really provided the drive to make all of this possible. I look forward to seeing what we couldn’t see before.