The Wi-Fi 6 elephant in the room

Everyone is talking about Wi-Fi 6 (802.11ax). From product launches, to technical briefings, to product technical launch briefings, it’s all the Wi-Fi space is abuzz with. Everyone loves talking about “the shiny new”, after all – it’s the technological advances that bring evolutionary and revolutionary changes to our technical lives but there is always that one person (Lee, I’m looking at you) that asks the question: at what cost? I’m not talking capital outlay, I’m not necessarily talking about operational outlay, I’m talking political cost. Our users have long memories and when we mess up (either as an industry, an implementation, or for whatever reason) the political damage that a flubbed product, widget, service, feature, or deployment gone awry can cause is pain that we don’t readily recover from. Everyone wants Wi-Fi 6, no one wants to get it wrong. Frankly speaking, one of the reasons this is so particularly painful is because the last technology ‘upgrade’ we went through (802.11ac wave 1 to wave 2) was painful for a lot of users. Even though this represented a ‘minor’ technical upgrade, in Cisco land, the pain was palpable for more users than not. In order to understand why this was the case then, and not the case now, we have to go back in time. Way back in time. Like, to 1995…

did-you-know-the-famous-microsoft-sound-in-windows-95-was-created-on-a-mac-520844-3.jpg

The venerable Windows 95!

If you have grey in your hair, you’ll remember Windows 95. It was awesome. It was great. it had a strong heritage robust peripheral support, and performed well. For its day, (OS/2 users aside) you could do no better. That is unless you were ultra-geeky and decided to spring for one of those SMP (Symmetric MultiProcessing) capable multi-CPU motherboards. In my instance, I was rocking a dual slot-1 motherboard. When the Pentium II CPUs came out, I was first on the list and once I had two of those bad boys installed in my gaming rig, I booted it up to test out some Quake… and was disappointed. You see, Windows 95 (and Windows 3.1 and DOS, etc) were single threaded Operating Systems. This meant that the OS that ran my gaming machine couldn’t see more than one CPU so no matter what I threw at it, I was never better than one CPUs worth of performance. It wasn’t until the brave of us switched to Windows NT 4.0 (don’t talk to me about NT 3.51) and later Windows 2000 – that we were able to ‘unlock’ that hardware’s potential because the Operating System itself, was now SMP capable and could not only see, but take advantage of both CPUs!

Screen Shot 2019-05-03 at 4.36.28 PM.png

Windows NT4.0 with SMP support showing 2 CPUs!

Fast forward to the Cisco 3702 Access Point. This is an 802.11ac wave 1 AP and sports a host CPU of a Freescale P1020 running at 800MHz as confirmed by the clock speed shown on the AP boot messages:

PowerPC CPU at 800Mhz, revision number 0x2151

While not a terribly shocking revelation, there are plenty of data sheets around that detail that the P1020 is indeed a dual-core CPU (thank you, Moores law), but what most people don’t realize is that the venerable IOS operating on the AP3700 (and all other APs back to the AP350!) is a single threaded Operating System. This then means that Cisco was building some awesome dual core capable APs for quite some time now, but was still ultimately limited by an OS that could only execute a single thread at a time. They were ultimately bottlenecked, much like Windows 95 running on my dual Pentium II machine. Now, the Cisco development team aren’t idiots. They knew about this problem and have been working on it for many years – but it reached a critical point prior to the 802.11ac wave 2 APs being launched. This was the inflection point where it because no longer possible to ignore the wasted hardware resources in the AP platform due to a deficiency in the OS. Enter AP-COS. This is Cisco’s rebuilt from the ground up Operating System for all AP platforms moving forward – starting with the 802.11ac wave 2 platforms (the 1800, 2800, 3800, and 4800s). Speaking frankly, it took Cisco some time to get this right. If you were on the leading edge of the 802.11ac wave 2 platform adoption 4 years ago (yes, the 1852 was launched in June of 2015!) you felt the pain of a newly developed Operating System. Let’s park this particular side of the conversation for a moment though…

 

ac AP boot

Cisco 2802 boot messages showing two CPUs

Now, Cisco is well known for their special sauce. Ever since the Cognio acquisition (that us old timers can’t talk enough about), they have ‘rolled their own’ radios. While not precisely true, they have licensed their Intellectual Property to Marvell, the chief supplier of the radios for Cisco for several years now. As my good friend Dave delves into in his blog post, the 802.11ax platforms from Cisco mark a departure from Cisco working with Marvell to build their own radios in their flagship APs. As of the 9120 802.11ax (thanks Pat for the correction here!) (Wi-Fi 6) Access Point, Cisco has moved their special sauce into a discreet add on card and are now using commodity radios from Broadcom and Qualcomm Atheros (QCA) for their client serving radios. The upshot of this is that Cisco is not the only manufacturer using these radios. You could see how, when Marvell was the only radio supplier, firmware, hardware, drivers, etc – all coming from a single proprietary supplier could be a bottleneck from a development perspective. What if there was a radio-reset bug? What if there was a new feature? What if there was a client issue? In short, Cisco would be the only one impacted. This created several unique scenarios where the radio may have needed some attention (for a variety of reasons) but there was only on vendor tackling the problem. Cisco’s move to commodity radios presented a notable step towards stability in the market and they lost none of their proprietary hardware functions, now that things like CleanAir were moved off to a discreet hardware component. In short, the best in stability without sacrificing the things that make Cisco, Cisco.

 

ax AP boot

Cisco 802.11ax AP boot messages with 4 CPUs

Fast forwarding to Windows 10 – I mean, Wi-Fi 6 (802.11ax) platforms from Cisco. We now have a mature Operating System on our APs. We now have radio hardware (and all of the follow-on components such as firmware and drivers, etc) that represent the industry standard in compatibility and stability. These two things mean that the major reasons that we had all of the consternation from the 802.11ac wave 2 platforms have completely disappeared. Now, Cisco is an organization comprised of people, and people are fallible, but my personal experience with the Wi-Fi 6 (802.11ax) platforms from Cisco have been nothing short of rock-solid. We always have the benefit of hindsight, but in this instance, I believe that our past is well and truly behind us. Who knows what the fully ratified 802.11ax specification will bring us (okay, those on the IEEE board do!), but there are several tangible indicators that we’re very much over our growing pains of the past – the most important of which are hands on experience which have been smooth sailing, even under some pretty significant load. If you ask me, Wi-Fi 6 (802.11ax) from Cisco represents a huge leap forward on several fronts – performance, features, compatibility, and most importantly of all, stability.

Advertisements

AirCheck G2 gets a v2

It’s no secret that I’m a fan of the Netscout AirCheck G2 and have been since before it’s release. I’m happy to see that today they announced the version 2 of the firmware for the AirCheck G2 which brings some pretty neat features to the product. The official page goes into greater detail on the updates, but the two I’m most impressed with are the new interferers page and the integration of iPerf testing for the unit.

The interference detection is a nice to have feature for those field teams that need an initial look at the non-Wi-Fi devices in the air around them. It’s leveraging the integrated WLAN radio for spectrum analysis so it’s not perfect, but it readily enough identified several of the more common interference devices around me (bluetooth and microwave ovens). In addition to identifying the interference, the ‘locate’ functionality that you’ve come to expect with the AirCheck also works with the source of interference. In my testing, I was able to demonstrate that moving away from an active microwave oven did indeed show a corresponding drop in detected signal strength. Let’s be fair, it’s not a fully blown FFT based Spectrum Analyzer, but in a pinch, and for common items, it’s far more insight than we’ve ever had in a a handheld tester.

Screenshot0001 Screenshot0004 Screenshot0007

The iPerf server is another interesting new feature – not only for the AirCheck side of the equation, but for the far side tester as well. The newly announced Test Accessory from Netscout is reminiscent of the LinkSprinter products – handheld, portable, battery or PoE powered, and cloud enabled tester.

IMG_9364

The Test Accessory

This integrates quite nicely with the v2 firmware with the new iPerf test option once you connect to your SSID. The nice bit is that this removes most all of the headache of doing iPerf testing – configuring the server and figuring out what it’s IP address is. You can plug the Test Accessory into your network anywhere and it will phone home to the Link-Live service. The AirCheck will query the service and automatically populate the testers IP address for your test. This makes a very simple to use throughput tester that’s easy to carry and accessible to everyone. If you don’t have a Test Accessory or if you’re comfortable with iPerf testing, you can also just use the AirCheck as a standard iPerf endpoint, so you’re covered either way you want to go here – with a stock, static internal iPerf tester or with a field accessible, cloud enabled tester that goes anywhere.

Screenshot0008Screenshot0009 Screenshot0010

These features take an already dead-useful testing tool and expand it’s role for the Wireless LAN deployer to go beyond just ‘is it up’ testing. With the ability to now detect a number of interferers and do actual throughput testing of a Wi-Fi network, the field implications are that your existing installation teams (or other G2 users) can more deeply validate the functionality of a network – and when things go wrong, have another level of insight that they previously did not have. I’m quite happy that Netscout is clearly investing in, not only the G2, but additional products that augment and expand it’s functionality. The G2 is an overbuilt hardware platform and it’s refreshing to see that Netscout is taking advantage of that extra horsepower. If you’re an AirCheck G2 user, you really need to go get the firmware update now. If you’re not an AirCheck G2 user, what are you waiting for?

Does the world need another spectrum analyzer?

The worst tool in your toolbox is the one you don’t use. I found myself pondering that point when the fine folks over at Oscium sent me one of their WiPry-Pro Spectrum Analyzers – purpose built for iOS. While I don’t want to turn this into an Apple vs Android conversation, I personally use an iPhone and when I temper my tools needs with the devices I use (or am reluctant to use), the WiPry makes for a very handy first exposure tool. Now, many Wireless LAN Professionals will argue the merits of triage using a protocol analyzer vs a spectrum analyzer – my take on that piece of the problem is that you should be able to effectively use whatever is at your fingertips. The Oscium solution makes the spectrum analyzer very rapidly available for casual, at a glance, look at your network as well as a good indicator of where you should go next. Out of the box, the device is very intuitive with a lightning connector on one end and an SMB antenna port on the other end. When you attach the included antenna, and download the WiPry app from the app store, you get a good look at where most people head first – Layer 1 visibility into the 2.4GHz spectrum.

Oscium WiPry

With specifications similar to other similar solutions, it get’s the broad visualization done in fairly short order. You can see here an analog video camera with their channel 9 mask on to highlight the interesting slice.

Oscium ch9

One of the great additions that the WiPry brings to iOS is the ability to bring interesting bits into one view – like SSID names. I know of plenty of people that prefer the Android platform for this one ability. Now that we have it in a handy to use format wrapped around tons of Layer 1 data, I’d consider it a pretty compelling reason to stick with iOS.

Oscium SSIDs

In short, the form factor of the card, the usefulness of the data presented, and the Open API component of the app makes this at the top of my list for my next purchase. I’d recommend you go look at one too. While you’re at it, they have a sweet lineup of Oscilloscopes and Logic Analyzers. They’ve brought a whole lineup of analyzer products to iOS and I for one am keen to get much more hands on time with them.

Does the world need another spectrum analyzer? For my iPhone, yes – making it the best tool in my toolbox; the one I use.

The most useful Prime Infrastructure report

Cisco’s Prime Infrastructure has come a long way over the past couple of years. From it’s beginnings as WCS, then to NCS, then through the sordid Prime Infrastructure 1.x versions, we’ve finally arrived at a place where it’s reasonable to dig back into the product. To say that Prime Infrastructure (PI for short) is an overwhelming product is an understatement. I decided to write about an obscure but extremely useful report (yes, a boring report) that I think you should use.

As we all know, in the RF world, performance revolves around Channel Utilization – of which there are several definitions. For simplicities sake, I’m referring to Channel Utilization as reported by the venerable Cognio card (AKA: CleanAir) – the baseline reference that most Wireless LAN Professionals use to call ‘Channel Utilization’. This is the amount of energy detected on a channel during a specific dwell time. This metric is roughly the wired equivalent of ‘link utilization’. Wouldn’t it be nice to see a historic report of the ‘link utilization’ of all of the APs radios in our environment over a period of time? Wouldn’t it be nice to see if a change we made recently (disabling lower data rates for example) made a historic significance to our Channel Utilization? Yes, of course it would! Without further ado, I bring you, the Channel Utilization report that you always wished you had, but never knew was always at a your fingertips:

Reports Launch Pad -> Wireless Utilization -> RadioReport Launch Pad

(that last bit is important!)

Report Details

This gives you a historical report of every radio in your infrastructure (all 2.4 and 5GHz) and a trending of not only their Channel Utilization, but their TX and RX utilization for further correlation/troubleshooting:

Prime Infrastructure CU

What do you think? Do you find this report useful? If so, drop me a comment and let me know how you use it. What other reports do you find yourself favoring in Prime Infrastructure?

You can have my Cognio card when you pry it from my cold dead hands

There is a group of WiFi Professionals (me included) that just can’t let go of their Cognio based products. With the Cisco purchase in 2007 (which ultimately manifested itself in the CleanAir product) we have seen a slow but steady decline of high-fidelity PC based spectrum analyzers. We’ve seen people try to compete in a variety of ways; with lower fidelity devices (Metageek) or with the high cost BandSpeed based product (AirMagnet Spectrum XT) but it’s not rare to find a wireless professional still lugging around an old laptop to use their Cognio based (AirMagnet Spectrum Analyzer, Cisco Spectrum Expert, or Cognio Spectrum Analyzer) CardBus Card. It seems unlikely that we’ll see a USB based Cognio product anytime soon (if ever) so I thought it was high time to figure something else out.

Option 1) For years many of the lager laptops from Lenovo (and even Apple!) have sported ExpressCard slots. By using an Addonics ExpressCard34 to CardBus converter, you can load the Cisco Spectrum Expert software on in Windows and your card works just fine!

Pros)

  • It works!

Cons)

  • It requires an ExpressCard34 slot on your PC.
  • There are several converters on the market. Some work, some do not. Make sure you get one that maps the PCIe bus, not the USB bus.
  • It’s bulky the whole card fits outside of the machine and it’s not very pretty.

Cognio adapter in an Addonics converter

Option 2) The Sonnet Echo ThunderBolt to ExpressCard34 adapter will allow you to take the above Cognio/Express card solution and map it to ThunderBolt compatible interface on your laptop. This means that any MacBook past the Early 2011 MacBook Pro (which I’m using) or any PC with a ThunderBolt compatible interface (many modern Lenovo machines) now have a cable-attached (important for flexibility) way to use their Cognio, PC-based Spectrum analyzer on new hardware!

Pros)

  • It works without having a built in ExpressCard slot!
  • It’s cabled so you can move/relocate the whole bulky assembly to the back of your laptop lid easily.

Cons)

  • It requires a ThunderBolt port on your laptop.

Big, but relocatable thanks to the cable!

It should be noted that both of these solutions will not work through a hypervisor (VMWare Fusion or Parallels, for example) and require direct access to the PCIe bus – this means running Windows natively on your hardware. You Mac users, this means BootCamp. It should also be noted that many people call ThunderBolt many things and there are several varieties of the bus. Make sure it’s not a DisplayPort only interface!

In short, if you’re still lugging around an old laptop just for this (or any other wireless CardBus based adapter), you now have a solution that’s cheaper than an new AirMagnet card and far less bulky than carrying around that crusty old XP machine. It’s time to upgrade!

Make sure it's ThunderBolt!

The FCC.

Here in the states, we have a regulatory body called the Federal Communications Commission (FCC). As it pertains to the Wi-Fi world, they tell us what channels we can use how obnoxious we can be (strength) in those channels. We have what you would consider to be a ‘blanket rule’ that basically states ‘within a given number of frequencies, you can do anything you want as long as you limit yourself to a maximum power’. A very intentional byproduct of these rules is the relatively low cost of WiFi components. Since we don’t have to submit everything we operate to the FCC for validation, we have no ‘validation costs’ to pass onto our end users. In short, the FCC, as a regulatory body imposes rules and restrictions on our use of wireless frequencies in the name of the greater good. This generally works very well, creating the ecosystem of ‘small cell’ give and take that we live in today. You are given the choice to make your own determination if analog video cameras, microwave ovens, X-Box controllers, etc should take priority or if your Wi-Fi should. Political challenges aside, we’re masters of our own domain.

So what happens when someone does something outside the norm? What happens when someone violates the FCC specifications? What happens when someone fires up 10 watt outdoor analog video feed in 2.4GHz and points it at the broad side of a hospital?

As it turns out, someone recently did just that. I was asked to assist with locating what was being detected as a whole bunch of analog video cameras that hogging up all of Channel 1 in 2.4GHz along the broad side of a hospital. As you could imagine, with a good 100 or so Access Points all excluding channel 1 (due to interference) from their channel plan, this meant that a two channel plan was all that was left (6 and 11). After much sleuthing, we determined that the signal was traveling well in excess of 10 city blocks! In my book, that certainly fit the bill for ‘obnoxious’. With more than a little hesitancy, I went to the FCC web page for complaints and filed one.

We're concerned with Wi-Fi Jammers.

We’re concerned with Wi-Fi Jammers.

Once my complaint went off into oblivion, I’ll be honest with you, I didn’t expect to hear anything from them at all. Instead, a few weeks later, I got a letter in the mail with the usual FCC ‘devices must accept interference’ text that you’d expect from a Federal entity. I was heartened by the fact that I got a response however, and there was a ‘for more information call this number’. They offered, I did. The nice Federal employee took my call, listened to my acknowledgement of the letter, listened to my insistence the letter was insufficient, and listened to my complaint that there was something going on that the FCC clearly needed to get involved in. She thanked me for my time and stated that she would escalate my case. This was the last I personally heard from them.

I was fortunate enough to have some contacts near the building that we suspected was generating the noise. Sure enough, a couple of weeks later, they informed me that a FCC field agent showed up asking questions. Shortly thereafter, the video camera signatures stopped being detected, the channel cleared up, and things got back to normal at my customer.

The point of all of this is that you do have a friend in the FCC. They’re not the most communicative, timely, or ‘feel good’ organization I’ve ever worked with, but if you have no other choice, and you can prove reasonably that there is a strong need for them to get involved with a neighbor that being obnoxious, they will. Start to finish, once I engaged the FCC, it took roughly 2 months to get back to normal. Don’t expect them to be quick. Don’t expect them to believe you. Don’t expect them to understand what you’re saying. Do be patient. Do be persistent. Do be kind. You don’t want to make a Fed angry.

Here are some good times to engage the FCC:

  • You can prove beyond the shadow of a doubt that something beyond your sphere of influence is causing harmful interference to your Wi-Fi network.

Here are some good times to not engage the FCC:

  • Your ER department won’t buy new Microwave ovens
  • Your security team is installing analog video cameras
  • Your customers are bringing gaming consoles onto your property and using them
  • Your co-worker put a 20dBi antenna on a 200mW radio outside (although, this is a good way to get them called on you!)
  • You believe you have external interference, but don’t have a Spectrum Analyzer to prove it

If you need a Spectrum Analyzer, head on over to the MetaGeek folks and check out their Wi-Spy Mini or their Wi-Spy DBx for a good cost effective way to tackle interference issues. If they get too big, rest assured that Big Brother is out there – just a complaint form and a couple phone calls away…

The Unstoppable MetaGeek – now with CleanAir!

Rarely does such an organization come around that expresses it’s agility and prowess with as much regularity as MetaGeek. The most recently of which is their ability to use Chanalyzer Pro (their premium Spectrum Analyzer software) to talk to the Cognio chipset in a Cisco CleanAir Access Point. PC based Spectrum Analyzers have had a sordid history to say the least. Way back when, Cognio made what you would call ‘the best of the best’ PC based Spectrum Analyzer. This took the place of many of the bulkier, more expensive Spectrum Analyzers and proved to the world that a) it was important to get Layer 1 visibility for enterprise WLANs and b) that they could make it affordable for most services based partners. Everyone OEM’d the Cognio analyzer, AirMagnet, Fluke, and WildPackets. Along came Cisco. They purchased Cognio, killed off all of the OEM agreements, rolled the hardware into their Access Points, and started selling the Cognio product with the Cisco name on it (Cisco Spectrum Expert). Unfortunately, they didn’t do much with the CardBus product and let the non-AP components stale. The aging interface form factor left quite a few holes in the market and along came a few people here and there to make it all shake out like this (generally):

  • Cisco Spectrum Expert: Highest resolution, CleanAir AP and CardBus form factor, Cognio based
  • AirMagnet Spectrum XT: Middle resolution, USB form factor, bandspeed based
  • AP based Spectrum Analyzers: Low resolution, integrated into many APs, Atheros based
  • MetaGeek Wi-Spy: Low resolution, USB form factor, keyboard controller based

Ryan and team over at MetaGeek did an excellent job of using very affordable components to give us an alternative to the aging CardBus adapter and the newer, more expensive AirMagnet adapter. They were an awesome product for the money but never really achieved huge market penetration due to the fact that the Cognio and bandspeed products still offered higher resolution. With the Cognio hardware all locked up in the Cisco Access Points, it seemed inevitable that we’d never have a good way to access it. Imagine our surprise when at this years Cisco Live event, MetaGeek was there – showing off their integration between Chanalyzer and the CleanAir Access Points! Ladies and Gentlemen, this is the *exact* same Cognio hardware, high resolution Spectrum Analyzer goodness that we all know and love from the old days. When I first heard about this, there was much trepidation about MetaGeek perhaps not being able to address the ‘full power’ of the Cognio (ahem, CleanAir) chip in its rawest form, but I’m here to tell you, when compared side by side with a legacy CardBus based Cognio adapter, the data is identical! The user interface is the updated, Chanalyzer interface with all of the modern enhancements they’ve made over the years with the WiSpy products, but you’re using the high-fidelity data that Cognio gives us. Here’s how it works:

You can connect to a CleanAir AP that is autonomous or lightweight (registered to a WLC) and it can be either servicing clients or in dedicated ‘SE-Connect’ Mode. You get the highest resolution, widest image when it’s in this last mode so let’s start there. Log into your controller, select your AP from the wireless tab and change it from ‘local’ to ‘SE-Connect’. Click Apply and let the AP reboot and join back to the WLC.

Screen Shot 2013-08-12 at 9.02.03 PM

Once it’s joined back, select the AP again and you’ll find both the IP address of the AP and something called the NSI key:

Screen Shot 2013-08-12 at 9.08.06 PM

Lauch Chanalyzer Pro with CleanAir and goto the File Menu. Select the intuitive ‘Connect to a CleanAir AP:

Screen Shot 2013-08-12 at 9.12.25 PM

Once you do that, enter the values from the AP page that you previously saw including the IP address, NSI key and a friendly name for this AP:

Screen Shot 2013-08-12 at 9.13.07 PM

Once you’ve done that, mash the Connect button and you’ll start to see the familiar Chanalyzer Pro interface with all of the wonderful resolution we all grew so fond of all those years ago! For reference, I ran Chanalyzer Pro with CleanAir on the same machine at the same time as a Cisco Spectrum Expert instance (using the CardBus adapter). Aside from the waterfall flowing up in the Cisco product, and down in the Chanalyzer product, you’ll see striking similarities in the respective waterfall views:

Screen Shot 2013-08-12 at 9.21.24 PM

Screen Shot 2013-08-12 at 9.21.41 PM

and at the same time, getting all of the other awesome details out of the Cognio SaGE like interferer auto-classification and AirQuality Index. Proving once again that MetaGeek are the top kids on the block when it comes to innovation and integration – but don’t take my word for it, head on over to MetaGeek, grab yourself a copy and give it a spin!

Full Disclosure: As an delegate of the Wireless Field Day event, I was given a copy of Chanalyzer Pro with CleanAir to play with without promise or commitment to write anything – much less something positive. 🙂 MetaGeek is a regular supporter of the Tech Field day events and generally makes awesome products and is regularly engaged in Social Media – you should go follow them at @metageek and catch up on the NoStringsAttached Show where Blake Krone and I also talk with MetaGeek about Chanalyzer with CleanAir!