We all find bugs in software from time to time and most of the time they are fixed in relatively short order…..maybe….
It is always an unsettling feeling when those bugs resurface in later versions of code after being “fixed.”
We recently received reports from a school that the wifi was “not working.” Early on in troubleshooting I could see that stations were associated to APs in the reported troublesome area. All the normal quick checks looked ok. There wasn’t high channel utilization, retries were low, the APs were not overwhelmed with clients, etc, etc. I touched base with the person who reported the trouble and they informed me that the issue affected all devices and that it seemed to be in one specific area; the library. I again checked the APs via the web ui and upon quick glance everything still looked ok. I logged into the controller CLI and issued the station database command to see what the clients were doing specifically on the library access points. And then it got cold……..
As you can see in the graphic above the TX packets are few or none at all. This problem occurred a couple years ago in later versions of System Director 6 code and was fixed with a newer release so I had seen the problem before. I took down all of the necessary information so that I could put a ticket in with TAC then rebooted the two APs in the library. Both of the APs affected were AP 832s and both were experiencing the same problem. All other APs (AP1020s) were working fine. After the reboot the APs returned back to working order.
I began putting my documentation together to submit a ticket when we received word that another school was having wireless issues. This school was having trouble in it’s library as well. Like most of our schools, all high density areas (for the most part) are serviced by AP832s. I logged into the CLI of that school’s controller and found the same TX freeze occurring. It became obvious to me at this point that the problem was likely affecting more than these two schools. I checked a few other controllers and found most AP832s were in a frozen TX state. I started to think what the common denominators could be. All the APs affected were AP832s. All the controllers were running the same code, System Director 8.1-2-0.
And there it was….
I glanced at the uptime of the AP832s on the controllers that I had up. All of the AP832s had been up for 99 days. I started going through other controllers and checking AP832s that had been up for 99 days and found them all to be in a TX frozen state as well. A few I found that hadn’t reached 99 days were still operating normally. Clearly there was a bug that reared it’s head on the 99th day of uptime.
So you are probably asking yourself why so many of the APs had reached the 99 day uptime mark at the same time. I was wondering the same thing at first and then it hit me. I had done a mass upgrade of controllers over the summer to System Director 8.1-2-0 which mostly put all of the controller/AP uptimes in sync.
We launched a preemptive strike and rebooted all AP 832s less a group of three APs at one middle school which hadn’t reached 99 days yet. We knew that if we rebooted all of the APs the issue would be resolved for another 98 days or until FortiRu provided us a fix. All of the Ap832s returned to working order.
At that point I assembled all of my documentation and submitted a ticket to TAC. We did some initial troubleshooting but they needed to have some APs in the frozen state to get the information they needed. We put the ticket on hold until our test group of three APs reached 99 days uptime. That occurred over this past weekend, November 20. When I returned to the office I checked the APs which were at an uptime of 102 days. All three were in a TX frozen state.
As of right now, the issue is with TAC. It looks like we are the first to report the issue. They have all the information that they requested; logs, diags, etc so I should be hearing back from them soon. In the meantime, a quick reboot is just the ticket to thaw out the TX.
Here are a few commands I used to gather AP/radio data from within the AP CLI:
stadb display assigned
stadb display assigned -v <mac-addr>
stadb display rxq_info <client mac-address>
stadb display txq_info <client mac-address>
sys exec /wl -i radio0 msglevel err
sys exec /wl -i radio1 msglevel err
radio txqinfo radio0 (radio zero)
radio txqinfo radio1
radio txq radio0
radio txq radio1
radio show radio0 (radio zero) – radio specific parameters
radio show radio1 – radio specific parameters
radio stats radio0 (radio zero) – radio specific stats
radio stats radio1 – radio specific stats
dev cmd radio0 reset – resets radio without rebooting the AP
dev cmd radio1 reset – resets radio without rebooting the AP
I was recently called out to a high school who had put in a trouble incident indicating that they were experiencing slow wireless performance and client disconnections in a particular area of the building. When I arrived I found that there was a high concentration of clients in the cafeteria which numbered around 200, 150 of those on a single Meru AP832 (clients were primarily on one side). Currently we employ AP832s (3×3 AC) in high density areas unless it is recent construction in which case the whole school is done with the AP832. The cafeteria in this high school along with all other high schools in the district qualify as high density areas and are outfitted accordingly. I fired up MetaGeek inSSIDer for a quick glance to make sure everything looked ok. I quickly noticed that I had two BSSIDs on the same channel very close in dBm on both bands, which we all know is cause for concern. My man Rowell Dionicio explains this issue well in a recent blog post at Network Computing. You might be thinking, “Wait a tick #MeruMitch! Isn’t that what single channel architecture is?” Not really. In this particular instance I experienced a bug where a handful of AP1020s (2×2 N) thought that one of two radios inside were a radio from an AP832.
This effectively broke my virtual cell (virtual BSSID) which in turn created co-channel contention. For those not in the know, a virtual cell is a virtual BSSID that Meru groups all of it’s physical BSSIDs behind to create it’s “special sauce single channel architecture“. Watch the video and… DON’T SHOOT THE MESSENGER!
At the time I did not capture any further information so for the sake of this blog post I re-created the scenario at my house. Going forward please pretend that 5NINER-Meru = LCPS. Also the “Shed” AP was actually moved into my house for the purpose of this dramatic re-creation.
There were a total of seven AP1020s which suffered from this bug and a few of them were within hearing distance of where this high concentration of clients was located. I checked the controller and found the channel utilization through the roof in this area. I removed the corrupted APs from the controller and added them back. I set all AP1020 radios back to 11/44 and AP832 radios back to 1/149. You did read that correctly…ALL. Both virtual cells were back in business.
Ok, now pick yourself back up off the floor and/or clean up the brain matter that exploded all over the wall just now when your head exploded. Brain matter on the wall is unbecoming. I digress. Once the physical radios were all back behind the virtual BSSIDs created by the ESS profile, the client’s performance dramatically improved and everyone was happy again.
You may argue that this would be simply a one off issue due to the AP radio corruption that took place. In fact, it is not. It would be very easy to have a problem like this occur with the APs working as they should in an improperly designed network. To properly design a Meru single channel wireless network you should group like model APs in their own virtual cell when in a mixed model environment. A group of AP832 AP’s virtual cell will have a different virtual BSSID from a group of AP1020 APs which in turn means each virtual cell should be assigned unique channels. They can all happily co-exist in the same ESS profile though.
I understand the controversial side of single channel architecture but the fact of the matter is that it works great when it is designed and installed CORRECTLY! All the same RF fundamentals, designing, deploying, etc are relevant when using single channel architecture. Poor marketing among other things give it a bad name. Taking it out of the box, racking the equipment, throwing everything on the same channel, and letting it rip WILL yield poor performance. Counter to some marketing I have read, single channel architecture is NOT for the network group that “doesn’t have time to worry about maintaining the wireless network” or “doesn’t have time to properly survey.” That is real folks! I have read or heard that garbage before. So please…I ask from my little single channel heart, give me and my precious FortiRu a chance. We definitely deserve a spot “in the ballpark” among the big boys.
As you may have already read, I broke my promise to write a post every night while at Cisco Live. Some of you may have laughed when you saw that I intended to write every night. I guess we will chalk that up to a first timer mistake. My original plan was to blog about things I learned and high points in the sessions I attended while at Cisco Live. I still plan to do that but the sessions were NOT the greatest take away of the event. Although the event as a whole and the sessions themselves were GREAT it was the people, by far, that made Cisco Live an event to remember. So, please be patient with this post. It is lengthy and I went against peer recommendation to break it up and just write one single post.
I met my buddy Brian at Dulles airport. We checked in and made it through TSA with ease. I grabbed a bite to eat and relaxed for awhile in the terminal since we had some time to burn. When our plane finally arrived we boarded quickly and began our journey. The flight to Vegas was great. I took the time to write a quick blog post about the week to come. I was also able to read through my latest copy of QST magazine and do some research on my newly purchased Icom 7300 HF radio. The flight from Virginia was roughly 4.5 hours. Upon arrival in Vegas we were picked up by our driver Mark. He drove us to Mandalay Bay and dropped us off so that we could check in. Check in was smooth and we soon retreated to our rooms. After settling in Brian and I met up with Rob Boardman and we hit the buffet. Rob is kind of a big deal. You may know about his now essential piece of wifi equipment, the Hub Holster. Please check out his site and pick up a set for yourself. You will thank him later. After a nice dinner Rob said he would show me around the Vegas strip since he had been there before and I hadn’t. We spent several hours walking and taking in the sites. Needless to say, there are a lot of life lessons roaming the strip. We finally retired to our rooms sometime after midnight.
I woke up close to noon on Sunday. I guess I needed to catch up on rest from the hike Rob took me on the previous night and the time difference. It felt good to just wake up on my own since there was nothing really planned for Sunday. I began my Sunday by studying for the “Conducting Cisco Unified Wireless Site Survey” exam which was scheduled for the following day. I spent 3-4 hours reviewing my quick reference guide before I decided I had enough. I decided to walk across the street and grab a burger from McDonalds.
While I was sitting there eating I realized that the Quarter Pounder I was eating was the most normal thing I had seen over the last day and a half in Las Vegas. I had some time to kill after I ate so I walked around the outside of Mandalay Bay. I touched base with Sam Clements to see when he was going to arrive so we could go get registered and pick up our badges. Soon there after we met up, registered, and picked up our badges. We walked over to Social Media Central where a bunch of the “wifi guys” were beginning to accumulate. This is where I met Blake Krone, Ryan Adzima, Vic Nunes, Richard McIntosh , Stew Goumans, and Wes Terry in person for the first time. We shook hands, introduced ourselves, and chatted for awhile. It was here where I was formally introduced to Pokemon Go.
Some of the fellas had been playing so we decided to give it a try too. You know what they say, “If you can’t beat em, join em,” or “You gotta catch em all,” or something like that. After a few minutes all of us that had gathered at Social Media Central were herded together for the opening “TweetUp” picture.
After our picture, us wifi guys decided to hit up Cantina for some Mexican food. After some good conversation, catching some more Pokemon, and eating some tacos, we decided to hit the strip for awhile before retiring to our rooms to get some rest for the first day of Cisco Live!
I awoke Monday morning still fairly tired. I still wasn’t used to the time difference but it didn’t make much of a difference because I was excited to begin my sessions. My first session was “Design and Deployment of Wireless LANs for real time Applications” with Jerome Henry. It was here were I was also first introduced to the one they say is from Saskatchewan, Brennan Martin. Jerome offered up a great session. He gave many real life examples of client TX power, designing cell sizes, etc.
After the first session Rob, Brennan, and I made our way to the opening keynote with Chuck Robbins. Chuck did a great job of inspiring us to be awesome and buy more Cisco stuff. After the opening keynote we proceeded to the World of Solutions area for the first time. Upon entering World of Solutions your senses are immediately stimulated by the very large booths, bright lights, and piles and piles of swag. I would venture to say that I collected most of my swag on the first day. My newly acquired Cisco Live backpack was nearly full by the end of our visit that first day. I didn’t get to spend a great deal of time at WoS on the first day because I scheduled my exam for that afternoon. Feeling pretty good about my previous day’s review, I strolled down to the exam area to give the “Conducting Cisco Unified Wireless Site Survey” exam the good ol’ college try. The exam area was very well organized and the helper people were well prepared. They quickly got me registered and my photo taken. I was then ushered to my testing workstation where I began my exam. I felt great during the exam because I had realized that the previous day’s review was just what I had needed. I finished up my exam and realized my confidence had proven true. I passed the exam!
I met back up with the fellas and proceeded on to the second session of the day, “7 Ways to Fail as a Wireless Expert” with Steve Heinsius. Here is a link to his presentation:
After a great session by Steve we made our way back to World of Solutions. We visited and met new friends at the NetScout booth where we drooled over the new AirCheck G2. If you haven’t heard of or read up on the Aircheck G2, do yourself a favor and do some research. The device is amazing. In the couple weeks since Cisco Live I have met with a rep and we are checking into purchasing one for ourselves. While at the booth we met the lovely Kendall Hershey who is the creative genius behind the tweets of the NetScout Twitter handle. We tried to convince Kendall that we were big time bloggers and that we would write all about the AirCheck G2 if we were given one to test. ***The jury is still out on whether or not it worked***. All in all, the whole gang at the NetScout booth was great! We continued our stroll around WoS collecting more and more swag. We stopped to take a break at one of the many small tables around the area where we found ourselves chatting with another lovely lady who we discovered was the voice of the Cisco Twitter handle, Laura Babbili.
We talked with her for awhile, discussing the ins and outs of being in charge of such a prolific social media medium. We may or may not have also tried to get her to specifically mention us from the Cisco Twitter feed. Either way, we still liked meeting Laura! We cruised around WoS for a little while longer before finally heading to Ri Ra Irish Pub for some more socializing. Overall, a great start to Cisco Live!
Tuesday’s sessions began with “Understanding RF Fundamentals and Radio Design for 802.11ac Wireless Networks” featuring Fred Niehaus. My new friend Rowell and I were in a live tweet war during the session.
Both of us trying to bring you the cutting edge information that Fred was speaking about. Fred is a legend in the wifi industry. It was great to hear his take on the technology and how it began and how far it has come. It turns out Fred is also a fellow amateur radio operator. Fred spoke about operating modes, the evolution of radio technology, micro vs macro cell, FRA, and the very cool new DART connector.
Fred showed us very early versions of wireless access points. He demonstrated how durable they were built. He used an old Cisco Aironet 350 as an example by dumping water on it and throwing it across the room. He also had some early versions of the most recent Cisco 3800 series APs. At the end of the session I threw out a frivolous tweet to have an autographed Aironet 350.Needless to say, Fred delivered by answering me later in the evening.
After the session we met up with Ekahau’s very own Jerry Olla to talk Ekahau Site Survey. He wanted to meet with us and discuss how we were using ESS, what we liked, and what we thought might need attention. He also showed us some cool toys he had been playing with in his spare time. It is quite remarkable what Ekahau does for the wireless community. Not only do they make an OUTSTANDING product, but they also reach out to the community for feedback. This is what I think makes Ekahau so great! They are always willing to hear our voices and take our requests very seriously. They continuously strive to make the ESS experience better. They truly are an example of a customer first company that designs a product that makes their customer’s jobs easier. If you don’t use Ekahau currently, I strongly recommend reaching out to them to see what they can do to help you. They are first class…even if they make you eat their “candy.”
After meeting up with Jerry, the small group of us returned to World of Solutions to meet up with the Ekahau folks manning the booth.
We had heard that they were handing out “special swag” to people who were giving ESS testimonials so we needed to get in on that action. It was great to meet Jussi, Hannele, and Anastasia in person. My apologies (on Rob’s behalf) for nearly taking down your booth :). Here is Rob and Brennan’s first attempt that did not make the cut:
Here is the testimonial that did make the cut (thanks to Hannele for the great camera work):
Like I said in the video, I will not hold it against Jussi or Ekahau for the assault on my mouth that their Finnish candy did.
After Rob nearly destroyed Ekahau’s booth and we gave our interview, we headed towards the lunch area but were quickly side tracked be a huge line that was weaving in and out of the conference area. It turns out that Cisco Live was handing out a Raspberry Pi Zero to everyone during that time.
After we picked up our Pi Zero and grabbed some lunch we continued through the World of Solutions area. Our first stop was at the AccelTex booth were we visited our boy Brian Smith, or Smitty as he is more affectionately known.
I’ve known Smitty for a couple of years now and use his antennas pretty much exclusively. AccelTex definitely has an antenna product line worth checking into. They have an antenna solution for any scenario. After spending some time hanging out with the AccelTex folks we moved along back to the NetScout booth to visit with Kendall again. Rob and I were still desperately trying to get some units to evaluate.
We decided to take the next best thing and listen to NetScout give a preso. We enjoyed the time so much we graffitied their table. After the presentation by NetScout we moved over to visit the fine people at Solarwinds. We talked with them for awhile grabbed some of their swag and continued on our journey. Rob and I decided to go check out the Cisco store and see what kind of gear they had. After all, my nine year old son asked me to bring him home a Cisco shirt (made a Dad proud).
I was on a mission to meet someone who has been part of my Cisco journey since I decided to start down the Cisco road back in 2007. Jamie Shoup has been an awesome resource for Cisco Press information and has allowed me to review several Cisco Press books (I swear I will write some feedback one of these days). When I found out I was definitely going to Cisco Live this year, I knew I had to make it a point to meet Jamie. She was one of the first people I met along my Cisco journey several years ago. Meeting Jamie was truly one of the coolest parts of going to Cisco Live.
As you can probably already tell, the group of us wifi guys are heavy on the social media. It was pretty cool early in the week when we began to notice that we were making an impact with the Cisco Live social media people. We frequented their area several times throughout the week and appreciated them interacting with us on a regular basis. As you can see there were several people that had the responsibility for keeping the social media flowing for the @CiscoLive Twitter handle. They did an exceptional job keeping people apprised of events and interacting with people who were active on Twitter. Kudos to this group of people…Cisco, you should pay them more.
Another cool part of my Tuesday at Cisco Live was running into Marcus Barman. As many of you already know, I do not use Cisco wireless products at my day job…(yeah, I know…). I became known as #MeruMitch amongst my peers at Cisco Live. Yup! I am that guy. The Single Channel Architecture guy. You can hold the commentary because I have probably already heard it :). Anyway, I digress, Marcus and I have been interacting with one another on Twitter for a few years because he was, what seemed like, the only other guy in the world that used Meru (I know there are more but the SCA guys don’t seem to be very vocal\social). As I was walking through WoS I received a tap on my shoulder. I turned around and found Marcus who said, “Aren’t you that Meru guy?” Yeah, it has gotten that far. I am “that Meru guy,” aka #MeruMitch. It was nice to meet Marcus because for a period of time it seemed like he was one of my only allies. After a brief chat and a handshake we continued on with our day.
Every year at Cisco Live they put on a party just for CCIEs to attend. This year was no different. Although I had no plans on going because I am a lowly CCNA-W (aspiring CCNP SWITCH and Wireless), my good friend Sam asked me if I would like to use his +1. Of course I jumped at the opportunity to go rub elbows with the elite\famous network engineers. The party was located at the Hard Rock Hotel. When we arrived we grabbed some food and drinks and mingled with other engineers who we correspond with on social media. It was here that I had the pleasure of meeting Amy Arnold . Most of you know her as amyengineer .
She referred to me as “that funny guy” which made my evening.
Nice to meet you in person @amyengineer ! Thanks for thinking I am "funny" 🙂
After a little more socializing and food we decided to move on to the next event. Before we left the CCIE party we had to do one more thing. We had to get our picture taken in front of the green screen.
This photo was titled, “This is what happens when you hang out with Canadians.” The Canadians taught me many new things like what a Bunny Hug (look it up) is and the ins and outs of Canadian Football to name a couple.
We boarded the bus and headed back toward the strip where we continued on to the Cisco Canada party. Yes, you read that correctly, I had to act as if I said words like “eh” and “Saskatchewan.”
I was looking forward to sampling such local delicacies like Poutine, Molson, Canadian bacon, and Maple Syrup, but instead I found tacos, stuffed potato skins, and hot wings. Not what I expected, but satisfying nonetheless. And that was my Tuesday. After writing this I realized that Tuesday was my busiest day. I have to admit I didn’t think my days would fill up like they did. It was a pleasant surprise!
After my previous big day, Wednesday settled into a nice slower paced, but full day. I started my day by attending a session by Jim Florwick entitled “Improve Enterprise, WLAN Spectrum Quality with Cisco’s Advanced RF Capabilities.” Here is a link to his session presentation:
Although I have already achieved CCNA Wireless, I could not pass on the opportunity to listen to Jerome speak again. He gives a great presentation and you will learn something regardless of your status. I ended up having to leave the session early because I had to go pick up my autographed Aironet 350 from Fred Niehaus.
I was pretty excited to meet up with Fred and talk amateur radio with him for a couple of minutes and get the autographed AP. I arrived back where he was presenting his session just in time for it to let out. He was talking with some fellow attendees so we waited patiently. Brennan suggested that I go pick the Aironet 350 up off of the floor because Fred had dumped water on it and tossed it again. I found it leaned up against the front wall.
I finally got a chance to meet Fred. He saw me walking towards him with my Sharpie at the ready so he knew what I was after. We talked for a few minutes about amateur radio and had a nice eyeball QSO. You can tell Fred is one of those guys who you can just sit and listen to him talk and learn a ton of stuff.
On our way back from our last session of the day we again collected near the Social Media Central location. It was here where I had a chance to meet Stephen Foskett. He is one of the fellas behind Tech Field Day which is an event where vendors entertain delegates with presentations based on their products. Tech Field Day is divided into sub-categories of which mobility (wireless “stuff”) is one. The presentations usually spark great conversation. Stew introduced Rob, Brennan, and I to Mr. Foskett and we let him know our interest in becoming a delegate for Field Day. I know we all look forward to potentially being picked to be a delegate for Mobility Field Day.
Wednesday night was going to be a big night for us. We had the Customer Appreciation Event at T-Mobile Arena where Elle King and Maroon 5 were set to perform. After the CAE we were going to attend a live Whiskey and Wireless show.
Stew hooked us up with tickets for a suite which was loaded with food and drinks. We had a great view of the stage and enjoyed several songs by Elle King. We had to peace out early to get back in time for the Whiskey and Wireless show so we didn’t get to see Maroon 5. We hustled back to Mandalay Bay because we were running a little behind. We arrived a few minutes late…then it happened. Sam saw that we had arrived and he called out to me to come sit with them at the table where the recording was taking place.
I knew what was about to happen. It was time for the world to know. It has been mentioned a few times that I should speak on how Meru and it’s single channel architecture work for us. I have been nervous about this because I know the strong feelings of some about Meru.
I wasn’t nervous anymore so I poured myself some whiskey and hoped to not sound like an idiot. The podcast will be released soon and I am excited to see how I did. The unedited recorded stream was up for a day or so and I received some positive feedback on it so I think I did ok. Thanks to Sam and Ryan for having me on and for not being to hard on me. Special thanks go to Robert Bartz for the encouragement to speak on the topic. I stuck around to listen to the rest of the show and I am glad I did. Sam and Ryan do an excellent job of entertaining and talking wifi at the same time.
The last day of Cisco Live was upon us. I had two sessions remaining. The first session was “Advanced Wireless Troubleshooting” with Tim Smith. The presentation can be found here:
After the session Rob, Brennan, and I decided to hit up 5.11 Tactical to check out their gear. We grabbed an Uber and headed off strip. I wanted to check out their satchel offering, and an offering they did have.
Brennan and I both picked up a new satchel and then we met up with Rowell. Brennan and Rowell had a deal to make and the transaction happened in a fairly empty parking lot in Vegas…that is all.
After the deal was done we grabbed another Uber back to Mandalay to have some lunch. We decided to get some food from Nathan’s Hot Dog stand. We picked up our food and decided to eat back at Social Media Central. We tore into our meals and then it happened. Peter Jones, Mr. Catalyst 3850, himself passed by our table. He stopped to talk with some other people but all of us noticed who was standing right beside our table. I waited for Peter to end his conversation and then we introduced ourselves. Our small talk quickly escalated and Peter began showing us all kinds of cool things about
the IEEE Ethernet roadmap and explaining how the Unified Access Data Plan ASIC worked inside the Catalyst 3850. We were all hanging on to Peter’s every word. He was super interested in how we used the 3850 so we each gave examples of how we use the product. He was truly interested in how customers were using it. You could hear his passion for his work in the way he talked. He then reached into his backpack and pulled out a bag of loose UADP ASICS and gave us each one. Along with the autographed Aironet 350, this ASIC was one of the coolest souvenirs I got while at Cisco Live. Before I could even ask, Peter stopped me, pulled out his Sharpie, and signed the ASIC for me.
We talked for a good amount of time and before I knew it my next session was quickly approaching. I thought about it for a minute and decided to stay and hang out with Peter because it isn’t everyday you get to talk with a guy like him. The time that we got to spend with Peter talking about the 3850 was great but the coolest part was that he wanted to hang out with us at the closing keynote! We all had to go drop off our gear but agreed to meet back in the lobby 15 minutes later. We all walked together down to the closing keynote where Kevin Spacey was going to talk about story telling.
While we were waiting for the keynote to begin, they were showing pictures from social media from throughout the week. We made the big screen a few different times. Brennan had an extra special picture show up on the screen. I hope he will write about it in his blog post. Kevin Spacey did a great job entertaining us. He was very funny and just listening to him speak so well made me think back to Public Speaking class in college.
After the keynote, the Canadians, Rob, and I departed for Gordon Ramsay’s BurGR for our final dinner together. Our food was great and we took time to reflect on the week and just hang out. Our boy Steve had to leave early to go meet his wife so that just left the three of us to go meet up with the rest of the gang and have one last drink together. We concluded almost right where we started, at Ri Ra. We met back up with Peter and had a drink. It was cool to just sit and have a cold one with my new buddies. We were all very tired and decided to turn in at a fairly decent hour to get rest for our travel home the following day. We said our goodbyes, shook hands, gave bro hugs and retreated back to our rooms one last time.
I woke up around noon which felt great! My flight wasn’t until later in the day so I was afforded some time to sleep in. I got all my things packed and met Brian down in the lobby. We met up with our driver Mark again who delivered us to the airport. When we arrived we realized our flight had been delayed an hour which really wasn’t a big deal. It allowed us a little time to relax before boarding the plane. I took that time to write down all the events of the week in a small notepad so that I wouldn’t forget all of the highlights. I am glad I did because it would have been tough to remember all of it. We eventually boarded our plane and had an uneventful trip back to Virginia. We arrived back at Dulles late in the evening. Brian and I said our goodbyes and parted ways.
I went to Cisco Live looking forward to learning a ton. I assure you I was not disappointed. What I thought was going to transpire was drastically different than what actually happened. I was still able to gain a bunch of knowledge and attend great sessions by leaders in the industry, but the biggest thing I took away was new friendships and solidified existing ones. These friendships will last for my entire career and maybe longer. The wifi community has something special going on. It is really hard to explain until you experience it first hand. Everyone wants everyone else to be successful in their wireless endeavors even if you do it just a little different (#MeruMitch). The relationships in our community are very unique and I am glad to be a part of it. I am by no means a big player but I know I have certain areas where I can contribute for the betterment of the community. I only look forward to what further events will bring. Cisco Live was definitely a great way to begin what I hope will be many events to come. A special thanks to Sam for his help preparing me for Cisco Live and answering all the questions that I had. Also a special thanks to my new Canadian friends (Brennan, Stew, Steve) and Rob. You guys definitely helped make my week great. It goes without saying that I have a new appreciation for our Canuck neighbors to the north. Canada, I salute you! And last but not least, thank you to my management at Loudoun County Public Schools for affording me the opportunity to participate in an event that has been a dream of mine for many years.
Ok…I know I promised blog posts every night. That was a first time attendee mistake. Live and learn. This will be the precursor to a longer more detailed blog post about an amazing week at Cisco Live. No blog post will ever do justice to the incredible time that I had. I met so many new people, established life long relationships, and learned a ton.
My apologies to my non-NerdHerd Twitter following friends and family who had to endure a week of weird posts crushing your feed. Anyway, I have been working on a list of the crazy awesome things that happened to me this past week and plan on formulating a blog post in the very near future.
Well, I’m in the air on my way to Cisco Live right now. First thing I had to do was check out that in flight Fi. We will call it “usable” but it should be noted that the signal is great! I am looking forward to meeting a lot of you who I’ve only talked with via the interwebs and taking in some first class sessions. If you feel so inclined get in touch with @rowelldionicio and ask to join his #ciscolive-2016 Slack group. A few of us have already been planning some meet ups and “stuff” there.
I am going to try to post daily and give updates of my adventures at Cisco Live. Hope to meet a bunch of ya’ll face to face!
It is just over a month until I leave for Las Vegas to attend my first Cisco Live. I am really excited to be able to attend this year. When I began my Cisco journey about 10 years ago I had no idea that I would be afforded the opportunity to attend Cisco Live so soon. I have to admit it has been a little overwhelming trying to figure out which sessions to attend, planning which Cisco Exam to take, and looking through everything else the event has to offer. Here is the way my session schedule shakes out right now:
Of course, being a wifi guy, I signed up for mostly RF related sessions. I had a little help choosing from my buddy Sam Clements . Another resource I used was Scott McDermott’s “Guide To Cisco Live” blog post. This is a must read for any first timer. I found that reaching out to veteran attendees and reading up on what to expect is extremely helpful.
One thing I am looking forward to is meeting other people from the wireless network community that I have been corresponding with for several years. I have only had the opportunity to meet a few of you in person so I am hoping to add to the list when I get to Cisco Live. I made sure to leave space in my calendar for moving around the event and socializing.
The following post was inspired by my dude Lee Badman (aka @wirednot, aka Beef Wellington). He is a very talented writer and WLAN professional among other things. If you don’t currently follow him on Twitter or read his work, I highly recommend following him and checking out his blog. His contribution to the wireless community is invaluable but one of his greatest contributions is his daily #WIFIQ. Search out the hashtag everyday and join the conversation.
Back on April 1st Mr. Wellington posted the following #WIFIQ:
#WIFIQ 4/1/16 How often do u sell, config or use WIPS or the likes of CleanAir, 7signal, NetBeez or other performance overlays or sensors?
This particular #WIFIQ stirred my interest differently than what was intended. I have already been familiarized with Cisco CleanAir and 7Signal’s offering but I hadn’t heard of this new “thing” that went by the name NetBeez. I surfed on over to the NetBeez site and read up on their product. What I found was the answer to one of my needs as a Network Engineer. Even better, I had almost everything to get NetBeez running already! If you already own a Raspberry Pi you can have a NetBeez agent running in very short order. The instructions are located here. This will only get you set up to run with your agent connected to Ethernet. As a Wireless Network Engineer I wanted more! I reached out to NetBeez to find out how I could get my agent to do wireless monitoring. Panos Vouzis, one of the co-founders at NetBeez, got back in touch with me a short time later and filled me in with the remaining information that I needed. It turns out that the wireless feature is only available in the paid version. Panos made an image available to me for my Pi and let me know which wifi adapter was needed. The NetBeez agent has only been tested and verified to work with the Asus Dual Band Wireless-N USB adapter (USB-N53). When I purchased the adapter it was $40. For some reason, at the time of this post, Amazon has it for significantly more. While I waited for my USB-N53 to arrive I was able to get the special NetBeez image loaded and running with the help of Panos. Everything was working great already even without the wireless adapter. A few days passed and my adapter finally showed up. The drivers are already built in so the only thing that needed to be done was to add a wireless profile for the wireless networks that my agent would connect to.
Now that I had my wireless profiles configured I could verify that my agent was connected to the wifi.
As you can see above I was able to obtain an IP address via DHCP and you can see my TX/RX rate and SSID that I am connected to. A little bit further down on the same page gives you more information on your connection.
Now comes the good stuff…adding targets! Targets are the actual tests that you want your agent to run. There are limitations to the free version here as well. The free version only grants you access to create three targets. Panos graciously gave me unlimited target creation for my evaluation. I did find out the hard way that you can go overboard with targets. I created over 100 at one point and I could not keep my agent online. I trimmed down my targets to a much more reasonable number and my agent became stable again. Thanks to Stefano Gridelli for helping me with this situation. We were able to quickly determine the problem and get my agent back online. I added a few targets that I deemed important for testing and reporting in my enironment.
Configuring actual targets are extremely easy. Clicking on the plus sign brings you to a page where you specify what you want your target to do.
Once you have given your target a name and added resources you need to assign the target to your agent.
You can also specify an email address here so your agent can send an alert if your target reaches a warning or down state.
At this point, you are in business! Deploy your agent and start monitoring. After deployment you can actively monitor your agent via the web console and even run reports.
I have used my NetBeez agent in several circumstances so far and it has worked great! As I type my Beez is deployed in a spot where we are currently collaborating with other teams in our enterprise to determine the cause of a problem.
A huge thanks go out to the NetBeez team, especially Panos and Stefano. They were very accommodating to my requests and ready to help whenever I needed it. If you are in the market for a distributed network monitoring device in a small/portable package, please visit their site and consider their product. NetBeez is easy to carry around and deploys quickly.