In a previous blog post, All You Wanted to Know About Testing Wi-Fi Adapters, But Were Afraid to Ask, I promised to blog about Wi-Fi data collection. Well, that day has come. A recent Twitter discussion convinced me that shedding some light on this topic could be beneficial for the Wi-Fi community, because many people have only vague ideas about what’s going on under the hoods of their favorite sniffer or site survey applications. “Is that really important?” you may ask. Yes, guys, it’s very important, and yes, you should care. I’ll try to explain why.

Before We Begin

The topic at hand dictates the use of examples. These examples are TamoSoft’s Wi-Fi applications as well as competing (and wanna-be-competing) products. Some of them are great products made by competitors that I truly respect, while others are inferior, pale imitations. Nevertheless, I will try to be as unbiased as I can and operate only with factual, technical information regarding their functionality.

What Kinds of Apps Need to Collect Wi-Fi Data

There are many types of applications that need at least some access to the surrounding Wi-Fi environment. This could be a geolocation app that analyzes MAC addresses of nearby access points to show the relevant map, a hotspot manager, or anything else. However, I’ll focus only on professional software. By professional, I mean software networking professionals use to do jobs related to deploying, maintaining, and troubleshooting Wi-Fi infrastructure. This gives us the following short list:

• Basic Wi-Fi monitors – We use them for simple troubleshooting tasks, such as choosing a better Wi-Fi channel for an AP when there is too much interference from neighbors. The term “basic” shouldn’t misguide you, though. These apps are often quite advanced and useful, and they might be all you need to solve your problem. Examples: WiFi Explorer, inSSIDer.
• Wi-Fi sniffers – Just like the basic Wi-Fi monitors mentioned above, sniffers usually also provide simple “overview” info, but unlike basic monitors, they allow the capturing of Wi-Fi packets (i.e., data, management, and control packets) as well as meta information, such as the PHY rate or signal level for each captured packet. Examples: CommView for WiFi, Eye P.A.
• Site survey tools – We use them to survey offices, malls, stadiums, etc. and map sources of Wi-Fi signals. The collected information can be used to analyze and/or improve AP placements, minimize interference, improve PHY rates, etc. Examples: TamoGraph Site Survey, Ekahau Pro.

In this blog post, I’m not going to consider Wi-Fi applications that glean data from WLAN controllers. They work on totally different principles. The object of my interest here will be applications for Wi-Fi site surveys, scanning, and packet capture.

Why You Should Care

There is more than one way to collect Wi-Fi data, just like there is more than one way to take a photo. Need a picture for your office pass? Your grandma’s 640×480 pixel webcam will do the job just fine! A family photo with a picturesque backdrop? A modern smartphone is probably okay, but I’d go for a good camera. Shooting a model for a large poster? Then you need very good optics and a large matrix, simply because a cheap camera doesn’t give you the level of the detail that you need to print a large picture.

I hope you get the idea by now: The devil is in the details. When you select a Wi-Fi tool for your job, you’d better make sure that the tool is adequate for the job. Now, let’s see how this wisdom applies to the wireless world and look into how applications can collect Wi-Fi data.

Scanner Interface

Most operating systems, including Windows and macOS, have simple and easy-to-use programming interfaces that provide access to the Wi-Fi scanner. The Wi-Fi scanner is an integral part of the OS that, basically, does exactly what its name suggests: If your computer is equipped with a Wi-Fi adapter, the scanner constantly scans the ether and detects nearby APs. You can access the scanner even if you’re not a programmer; you just need to open the terminal and type a few commands. Here is how it looks on Windows:

And it’s pretty much the same on macOS:

Generally, the scanner is a great tool that allows apps to get a quick snapshot of the Wi-Fi environment. If that’s what your app is supposed to do, the scanner is totally adequate, but it has problems that make it inadequate for more complex jobs. To make the technical explanation less boring, I’ll just share the following dialog that I recently overheard in an office.

Application: Please perform a scan cycle.
Scanner: Done.
Application: What do you mean, “done?” You’ve only spent 1.5 seconds on that, but a minute ago it took you 5 seconds.
Scanner: I’ve skipped some of the channels. Nobody was using them anyway.
Application: But that could have changed! We’re walking right now.
Scanner: Please don’t tell me how to do my job, okay?
Application: Okay. Let’s see what you’ve reported… wait… I’m standing right in front of the AP with the “OFFICE_ONE” SSID broadcasting on channel 44, but it doesn’t show up in your scan report. How could you have possibly missed it?
Scanner: Don’t yell at me! I’m associated to the “OFFICE_TWO” AP on channel 11 right now, and the operating system told me to download some urgent updates. What am I supposed to do, stop the download and switch my only precious radio to a totally different channel just because you need to do that stupid scanning?
Application: Fair enough. Let me disconnect you from that network so that you don’t get distracted… done. Now, scan again. Oh, and don’t forget channels 12 and 13.
Scanner: Won’t do it. I’m American. FCC forbids that.
Application: What? I bought this laptop last week in Berlin!
Scanner: You did, but during the last scan cycle, I found three nearby APs that had the “US” country code in their beacons, which clearly means that we’re in the USA right now. Sorry, my German friend, in America, we always play by the rules.
Application: Damn it, we’re still in Berlin! It’s just that some idiots didn’t change the default country code. Fine… can you at least scan DFS channels?
Scanner: DF… what?
Application: Dynamic Frequency Selection! Ever heard of it?
Scanner: Not telling.
Application: What do you mean, “not telling?” It’s either “yes” or “no!” You can’t tell me which channels you support?
Scanner: Nope. Not allowed by the OS. Do you want me to go ahead with scanning or what?
Application: Good Lord! Well, go ahead. Let’s see what you’ve found… thanks. Seven seconds for a single scan cycle this time? Wow. But I don’t understand… the signal from this AP is -20 dBm. We walked very close to that AP a minute ago, and yes, at some point, the signal could have been that strong, but now that we’re 20 meters away from it, how can it still be at -20 dBm?
Scanner: Umm… it’s complicated. I might cache a value or two from time to time. Or, if I received several beacons, I might use a special algorithm to compute the average value…
Application: This is very vague. Can you be more specific? Can you show me the algorithm?
Scanner: Sorry, I’m under an NDA.

That was only a fragment of the heated conversation. Anyway, to sum up:

• The scanner interface provided by the operating systems is great for simple scanner applications; it doesn’t disrupt your wireless connectivity.
• It is unusable for packet capture (no such functionality) or site surveys (a dozen problems, as described above).
• Example of adequate usage: inSSIDer. Examples of inadequate usage, in which the basic scanner is used for site surveys: VisiWave (Windows) or NetSpot (macOS).

Standard Packet Capture Interface

So, what are application architects supposed to do if the scanner interface is not good enough for their purposes? Right—they could take full control of the Wi-Fi adapter and actually capture packets. This allows them to choose which channels should be monitored, how much time to spend on monitoring each channel, how average signal levels are computed if multiple packets have been captured from an AP, and take on other important tasks that are beyond the scope of a blog post.

For the sake of clarity, I’d like to stress that I’m talking about real, actual Wi-Fi packet capture, in which the application gets access to the actual 802.11 management, control, and data frames, as well as per-packet info on PHY rate, signal level, noise, etc. This is important because you shouldn’t confuse real Wi-Fi packet capture with emulated packet capture, like the one you see when, for example, you run Wireshark on Windows and it captures data packets on your Wi-Fi adapter. That’s not even close to what you need, because you don’t get access to, say, beacon packets or signal levels.

Now that this important clarification is behind us, let’s ask the big question: Do operating systems provide such a wonderful interface that would allow us to capture Wi-Fi packets? On macOS, the answer is “yes.” It’s undocumented and it’s not perfect, but if a development team has enough time and expertise, this is doable.

On Windows, the situation is lamentable. Back in 2006, Microsoft released Windows Vista. One of its new features was a new interface for programmers that was supposed to do exactly that; i.e., give us a way to put Wi-Fi adapters into monitor mode, with full control over the capturing process. That was a long-awaited change. After all, why did Wi-Fi have to be treated as a second-class citizen? Ethernet adapters had allowed packet capturing since Windows 95, and it was about time to do the same for Wi-Fi adapters. Awesome idea, Microsoft! Unfortunately, things went terribly wrong:

• The newly introduced interface was optional. In other words, it was up to the chipset vendor to decide whether to support it. Some vendors, such as Broadcom, implemented support for it right away, and others never did. Some vendors, such as Intel, initially implemented it but then ditched it.
• Those vendors that implemented it generally did so very poorly. It wasn’t entirely their fault; the Microsoft specs were rather ambiguous, and eventually, the whole thing became a big mess. Vendor A added 4-byte FCS checksums at end of each frame, Vendor B did not, Vendor C simply discarded packets with bad FCS, and Vendor D added those 4 bytes, but that was not the FCS. Anyway, you get the idea. How are you supposed to tell if a packet is correct if you have no idea exactly how FCS handling was implemented by the specific vendor?
• The Microsoft specs were very limited. To give you a couple of examples, the API didn’t provide a way of specifying the secondary channel in case of 40 MHz channels; i.e., whether the user wanted to monitor channels 6 and 2 or channels 6 and 10. It also failed to report any parameters in 802.11n and newer WLANs correctly; PHY rates above 54 Mbps were not supported.
• Now, the final shot in the head: This interface was deprecated in Windows 10. It is quite probable that with the next Windows 10 update, users of site survey applications based on the deprecated API might suffer the fate of Cinderella. At the stroke of midnight, your carriage might turn back into a pumpkin; the app might simply stop working.

To sum up:

• On Windows: The standard Wi-Fi capture interface may work for some adapters. Most of the major chipset vendors don’t support it anymore, and those that still do (e.g., Broadcom) might discontinue it any day. It is not usable for professional packet capture due to major limitations: wrong PHY rates, wrong FCS, no support for wide channels, etc. It also offers no control over regulatory domains, and typically no support for DFS channels.
• On macOS: totally fine. You can easily tell if an application is actually using the packet capture interface on macOS. When it does, the standard Wi-Fi icon on the menu bar changes its appearance to a triangle with an “eye” inside (shown below). If you don’t see the “eye”, then the application doesn’t capture packets on your MacBook’s integrated adapter.
  

  

• Example of adequate usage: TamoGraph Site Survey for Mac. Example of inadequate usage, in which the standard Wi-Fi capture interface was used to make a packet sniffer: Acrylic Wi-Fi Sniffer. A quote from the vendor: “Acrylic Wi-Fi Professional makes use of current and accessible hardware for capturing in monitor mode (promiscuous mode) in Windows. Remember that you can capture in native mode with any WiFi card.”

Custom Drivers and Custom Devices

So, what do Wi-Fi software makers do when things are so gloomy on Windows? If you want to do it right, do it yourself! We either write custom drivers for Wi-Fi adapters or make dedicated hardware capture devices. That way, we don’t depend on Microsoft, Apple, or anyone else. A Wi-Fi adapter is a piece of hardware, and if we can fully control it, we can do pretty much anything.

Writing custom kernel-mode drivers is very difficult, expensive, and time consuming. One needs a team of very good software engineers with vast experience in kernel-mode programming. Despite the difficulties, there are companies capable of that.

To give you a historical perspective, about 15 years ago, a company called CACE Technologies released a specialized USB capture adapter called AirPcap. It was a decent device for that epoch and an awesome commercial idea—AirPcap Nx was a rebranded Compex WLU200NX card that was available for about $40 wholesale, whereas the AirPcap Nx was sold for about $700 apiece (apparently, that silver metallic paint was pricy).

We took this photo in 2011. That’s Compex WLU200NX and AirPcap Nx.

The device came with a software development kit, and anyone could implement support AirPcap adapters. While AirPcap is very old (no 802.11ac or 802.11ax support, no LDPC coding support, etc.), many apps still support it for historical reasons.

Anyway, because writing custom drivers is so challenging, today, there are very few companies that make custom Wi-Fi drivers (or custom hardware) for use with Wi-Fi software, such as packet sniffers and site survey tools:

• TamoGraph Site Survey and CommView for WiFi by us (TamoSoft). We write custom drivers.
• Ekahau Pro by Ekahau. Currently, it captures packets using a special device called Sidekick, but it used to support a few USB adapters using custom drivers.
• AirMagnet Survey and AirMagnet WiFi Analyzer by NetAlly (formerly by Fluke Networks). It includes custom drivers.
• Omnipeek Network Protocol Analyzer by LiveAction (formerly by WildPackets and Savvius). This one also includes custom drivers.

Yes, the list is short; you don’t have a wide variety of choices.

CommView for WiFi capturing packets from a modern 802.11ax adapter.

The applications that come with custom drivers for specific Wi-Fi adapter models explicitly list models on the respective “hardware support” pages. Some of them, such as Ekahau, go one step further and offer portable custom packet capture devices.

Sidekick, a dedicated hardware capture device and spectrum analyzer in one.

Whether a dedicated capture device is better for capturing than an integrated or USB adapter is debatable (sorry, Mikko, I respectfully disagree with your point of view, but it’s a long topic), but dedicated devices are definitely far more expensive; some $3,000 for the Ekahau Sidekick vs $50 for a USB adapter.

So, to sum up: Custom capture drivers and custom capture devices are the only way to actually capture packets without any limitations.

How Do I Tell?

Indeed, how do you tell if the packet sniffer or site survey application you plan to purchase uses an adequate packet-capturing method? I’d recommend the following:

• Ask the vendor. Decent vendors don’t have a problem telling you exactly how their apps collect Wi-Fi data.
• Check your Wi-Fi adapter’s network connectivity. If you can open a web site in your browser while the app is running (assuming there is only one Wi-Fi adapter in the system), then the app cannot capture packets. But again, if this is a simple scanner app rather than a site survey app, that’s totally okay.
• Check the list of supported hardware. A custom driver is always made for specific hardware; it’s never generic. This means that if an app uses a custom driver, the list of supported adapters is quite limited. For example, our list contains only a few 802.11ax adapters and a couple dozen 802.11ac adapters (and yes, we hand tested them all). If a Wi-Fi packet sniffer or site survey app support page reads something like “We support any Wi-Fi adapter” or “We support 100+ adapters,” it’s a big warning sign.

A Few Words About Android and iOS

At the time of writing, iOS supports neither packet capturing nor Wi-Fi scanning in any shape or form. Android supports scanning, but not packet capturing, unless you “root” the device.