Friday, January 24, 2014

Arduino and the Iris Zigbee switch, part 2

Edit: The last portion of this investigation is here <link>

I've had a few days to play with the Iris switch <link> and I like it.  However it doesn't play well with others.  In that I mean I tried to bring it up on the network with my other devices and managed to kill my network of XBees.  Yes, I was down with all my devices not talking to each other and I had to visit each device and fix it.  This is totally avoidable, I was an idiot.  Do not follow in my footsteps in this.

First there are some things to know.  As I pointed out in my last post <link> about this switch, it takes special settings on the XBee for it to work with an Iris switch:

ZigBee Stack Profile 2
Encryption Enable 1
Encryption Options 1
API Enable 2
API Output Mode 3
Some Encryption Key
XBee Modem XB24-ZB, ZIGBEE Coordinator API, Version 21A7 (or better)

This is radically different from what I have been using in the past.  First, the Zigbee Stack Profile is 2, which means use the ZigBee Pro stack; I normally use a 0 in that spot.  When I changed it to profile 2, it stopped passing all the messages from my existing XBees through to the serial port.  Naturally, I thought that I could simply change the other devices to use the same profile parameter and everything should be fine.  Not so.

It seems the API Output Mode has a similar effect.  When you change API Output mode to something other than 0, you lose all the messages from other XBees except the ones that are explicitly allowed.  The message: ZigBee Explicit Rx Indicator (frame id=91) comes through fine, but you lose the ZigBee Receive Packet (frame id=90) which is what you normally get.  That means my remote XBees were happily sending messages and the receiver didn't pass them on to the serial port.  Like I said, I was an idiot because the setting clearly indicates either 'traditional' or 'explicit'.  I should have gotten a clue from this before I messed with the XBee network coordinator.

So, I couldn't see anything that was being sent by my other devices.  Naturally, I found this out AFTER I had reprogrammed my XBee coordinator.  My whole network was down.  The various XBee devices had saved their connection with the coordinator as it was set up before and I had to visit each one of them and make them drop the connection and establish a new one.  What a royal pain in the behind.

While I was doing that I decided there had to be a way to keep from having this happen in the future.  It turns out there is.  By adjusting some of the parameters, the remote XBees will sense that the coordinator is gone and automatically hunt for a new one.  This is a mixed blessing because if your coordinator dies, your remote devices will stop talking to each other.  I've had exactly zero problems with the XBee coordinator, so I decided to set the network up this way so that future experiments like this won't mean taking things apart to get to the XBee and forcing it to initialize the network.  The parameters are:

Network Watchdog Timeout 1
Channel Verification 1 (enabled)

The watchdog timeout causes the XBee to reestablish the network connection if it hasn't heard from the coordinator in three minutes.  Since my coordinator is sending the time repeatedly, all devices will hear from it often enough that the network shouldn't ever have a problem.  The Channel Verification being enabled means that any time the power fails the XBee will check to be sure there is an XBee coordinator out there when the power comes back on.  If it can't find the coordinator, it'll go looking for one.

These changes mean that if my coordinator dies, the network dies.  Fine, I can live with that.  It also means that there may be a very slight pause whenever the power comes back on from a failure.  When I tested that by unplugging a device, it took less than a second for it to find the coordinator and get back online.  So, now if I mess up the coordinator with some experiment, I won't have to go visit each darn XBee and make it talk to the new coordinator set up.  They'll do it by themselves after a three minute timeout, or I can unplug the power and make them do it right now.  Too bad I didn't discover this a couple of years ago.

Now, these changes may not be the right thing for you to do.  However, if you have an XBee in a hard to get to location, or setting way out there in the field hooked to a solar charger, it might be a good idea.  So, you understand your needs better than I can; make an informed decision.

All this information means something else: The Iris switch and my network won't work together.  Simple, I'll just set up a coordinator for the Iris Zigbee devices I may eventually have separate from the network I am using now and control them separately.  I should be able to interface an Arduino that has the code for Iris to anything I want to and go merrily on my way.  That may be the next project.

So, there was something good that came out of messing up my network of devices.  I learned a lot about modifying the network in general.  I also got to visit each of my XBees and clean out the spider webs and dead flies.  There was live scorpion in one of them, but since it's cool right now and the arthropod couldn't move very fast, I won.  I also decided to retire the separate Arduino for the Acid Pump and move the code and connection over to my Pool Controller.  Since I was already in the code for the Pool Controller, I hooked in the Septic Tank Float so I can get an alarm in the house if the tank has problems.  These were chores that I've been putting aside for months now.

How many people do you know that have a septic tank that can send them email?

32 comments:

  1. Dave:
    I've tried compiling your Arduino code but line 14:
    XBee xbee = XBee();
    gives me the error:
    Iris_switch:14: error: ‘XBee’ does not name a type

    Am I missing something?

    Thanks.
    Glenn.

    ReplyDelete
  2. Sounds like you're missing Andrew Rapp's XBee library that I talk about in the post. I love using the library, having written the code to decode XBee packets myself a few times, it's nice to be able use a library that actually simplifies that chore. The problem was that Andrew didn't include the code for the two ZigBee message types needed (he didn't see the need at the time) and left out one minor item in the Tx Status message. So, I added that stuff to aide me in this hack.

    You can grab my version of the library here: https://www.dropbox.com/sh/rkrvqg6pfrdqn31/wPQ2-lgfQU

    It's a regular Arduino library, just stick it in the libraries folder and start the IDE

    I've given the files to Andrew Rapp, and he will include them in the XBee library distribution, but it may be a while since he has several irons in the fire.

    ReplyDelete
  3. Aha, I found the problem.
    I had the correct library from Andrew (except for the mods you made of course, thanks for them by the way).
    The reason that the code wouldn't compile is that I was using IDE 1.0.5. It doesn't like some of the constructs and thus upchucks the whole works as I showed above. When I went back to IDE 1.0 keep everything the same, it compiles without any complaints.
    Hope this helps others running into similar issues. I'm going to stay with 1.0 and I bet it compiles your other sources just fine.
    I can hardly wait to test my IRIS now........

    Glenn.

    ReplyDelete
  4. Dave:

    One other quick question. I want to set up a little traffic monitor with an XBee module. I have a sparkfun USB explorer and I was wondering if you have ever seen and code that I could drop into my Ubuntu rackmount pc and run the sparkfun explorer into the usb port and monitor traffic like that?
    I wouldn't like to dedicate an arduino at this point to that task.

    Cheers.
    Glenn.

    ReplyDelete
    Replies
    1. The problem with an XBee monitor is that the XBees don't have a mode that allows you to see all the traffic on their network. If you use explicit addressing (the good way to do things) XBees that are not addressed for a message won't crank them out the serial port. If you use broadcast, you can monitor the traffic, but there are problems with this method as you get more devices up on the network. I describe this in great detail somewhere on this blog.

      So, out there in the wild there are bits of code that run under .net on a pc that can talk to an XBee, but I have never played with them. I can't even read that code ! It looks totally like a alien language to me.

      Delete
  5. IT WORKS!!!!
    It's been a trial getting the software compiling and finding that I needed to put a jumper in my shield so the TX and RX from the XBee to the Arduino inputs. Now that they are in the software and IRIS work like a charm!!!
    I put a 60 watt bulb in a desk light and the software report says it's using 58 watts.

    Colour me stoked. Next it's a second IRIS switch for the second electric heater in the barn then out they go so the 1500 watt heaters only come on one at a time.....

    FANTASTIC WORK DAVE on the software!!!

    And thanks a whole bunch....

    Glenn.

    ReplyDelete
    Replies
    1. CONGRATULATIONS !! You're the second other person to get one of these working. Now, go modify the heck out of the code; make it read nicely, make it work for multiple switches, make it your own.

      And, have fun doing it.

      Delete
  6. Dave - what exactly is the issue that stops the Iris from working with your other devices? I just picked up one of these and will play around with it tonight

    Anthony

    ReplyDelete
    Replies
    1. It seems that you get one of two choices with an XBee, pure Zigbee protocol, or something else. If you go with pure Zigbee, you can't do some of the things I did with the original devices because the messages don't get passed to the serial port. If you go with whatever the protocol is that Digi set up for the default, you can't see the Zigbee messages that you need to set the Iris switch up and run it. I went into the details up in the first few paragraphs.

      Zigbee is a terribly complex protocol that was designed for devices that don't even exist yet, and the manufacturers don't follow the protocol with the devices they provide and hide under the 'manufacturer specific' section. It's really annoying.

      Delete
    2. I think I get it. Once I get my device up and running, I'll dig deeper. Right now I can't get it to pair - the endpoint request, match descriptor response and the 2 custom messages are sent, but I never get an endpoint response back. Any ideas? The only thing I wasn't sure of was the encryption key and how to enter in XCTU. I just used 1234 and it seemed to accept it.

      Delete
    3. Double check the XBee setup. What you describe is a symptom of not having the API Output Mode set to 3. I think your key will work fine, but I never tried anything other than 1234567890 on mine. Well, that's not totally true, I did allow it to use a random key, but that scared me because I wouldn't know what the key was if I ever needed it.

      Delete
    4. I've spent the past couple of days trying to get this working. The setup is fine - I've tried 3 different XBees now and none of them connect. Very frustrating.

      Delete
    5. Seems that I spoke to soon. I can't get the Arduino code to work. But I had also downloaded Capnbry's C# code. That also didn't work. But I spotted that in his code he had created an Xbee object with ApiType.Enabled. Changed that to ApiType.EnabledWithEscaped and everything started working. Now to track down why the Arduino code doesn't work.

      Interestingly the C# code sends the Active Endpoint Request in response to Device Announce and it works, unlike with your Arduino code.

      Delete
    6. Heck, move the responses around to see what happens. Just because my switch works this way as well as a few other folk doesn't mean the manufacturer hasn't changed something. There is a definite timing problem in the sequence of responses that Capn found way back when he was playing with it. I encountered the same problem and solved it differently. You may need to move things a bit as well.

      On the Arduino you have to use API mode 2 (escaped characters) because that's the only way Andrew's library works. This combined with the API output mode 3 should let the messages get through.

      Delete
    7. I will dig into this over the weekend. The Arduino code works now that the XBee and switch are paired, so I need to look at why the pairing doesn't work. I've tried changing the sequencing, so there must be something else going on. I think I'll modify the C# library so I can get a dump of the exact messages going back and forth and see if I can do the same with the Arduino code.

      Delete
    8. It's totally easy to print the data that you're receiving from the XBee. You can either catch and print it inside the XBee library, or just print it inside the Arduino Code. I've done both a different times chasing various problems.

      have fun

      Delete
  7. Just wanted to add to the thread that Dave's demo code worked straight-away. The only brief issue I had was communicating through the IDE serial monitor. At first, hitting 1 to turn the table light on, resulted in the light turning on briefly than going back off. Problem solved when I switched to sending "no line ending" rather than "both NL and CR" which was my default for another project.

    My real interest in all this is reading the Smart Meter provided by my power company (Southern California Edison) without purchasing a commercial product.It apparently uses a Zigbee modem (Rainforest is one of the commercial consumer options).

    I'm sure several others are following the website and having success with Dave's excellent write-ups.

    ReplyDelete
  8. Current I'm using a XBee S2 as my coordinator and it is connected to an Arduino board. On the other hand, I have another 3 motion sensors to send its information to my coordinator.

    I'm notice that sensors are sending a "data request" packet to the coordinator on every 5 second, and I would like to use this message for device connectivity check (e.g. bad connectivity if receive less then 100 "data request" message is an hour). The problem is, with the XBee.h library I'm not able to read these "data request" messages as it is a network layer message

    Therefore I would like to know is there any way for my to read these "data request" message or any alternative to know that the sensor is disconnected from my XBee.

    Thank you,
    James

    ReplyDelete
    Replies
    1. I don't know of any way to read the network layer messages. That's actually why most people like the XBees, you don't havve to deal with that level of the communications. However, you can tell if the sensors are dead by interrogating them.

      I you have a processor attached to the sensor XBees, send something and have it respond. If you don't have a processor attached you can send one of the remote AT commands and look for the response.

      Delete
    2. Thanks for your reply,

      At first I'm also thinking of send ZDO message and wait for the respond. But with this method I only know the sensor is disconnected at that very moment. When the sensor is on its maximum range it will disconnect and reconnect to the network.

      Therefore I'm thinking by couting the "data request" message it can tell me overall situation.

      By the way, have you ever come across with any method regard to this issue?

      Thanks,
      James

      Delete
    3. On the other hand, have you try the "Joining Temporarily Enabled" from the XBee.
      Bellow is my code and I even received "ok" status from the At command response but still my devices are not able to join the network.

      XBee g_oXb = XBee();
      AtCommandRequest g_oAtReqs = AtCommandRequest();
      uint8_t AtCmd_CB2[] = { 'c', 'b' ,2 };

      g_oAtReqs.setCommand( AtCmd_NC );
      g _oXb.send( g_oAtReqs );

      Delete
  9. I just started goggling this Arduino - Edison Smart Meter adventure today, and came across your blog. I'm about to buy the ZigBee stuff. I will keep in touch. Have you see this on the Edison website?

    https://www.sce.com/wps/portal/home/customer-service/my-account/smart-meters/han-faq/!ut/p/b1/hdDBCoJAEAbgZ-kFnKktteOWomuW2YbaXkLDNkvdsMjXz6AOEdXcBr5_4B8QkICo01sh02uh6rR87ELf9k2HuowjC1b2GJnBQz6wLRLwUQc2HcAvQ_FfPgbxTswZGSKbDO11MB2goZMP4EdGBxweRssIXd_4A-avC2MHbdcLkDnrkCAjIS44pQRRf4IfLTwQslRZ95HYAqFPqCXlozutM2JKEE2-z5u80Q7qcoWkbVtNKiXLXNupCs5VggU7jk4x7d0BdVKJ0w!!/dl4/d5/L2dBISEvZ0FBIS9nQSEh/#/faq-HAN Device Registration2/faq-HAN Device Registration3/faq-HAN Device Registration4

    ReplyDelete
    Replies
    1. The basic XBees I use can't interface to a smart meter. At least I haven't ran across anyone that has made it work. Smart meters use the ZigBee protocol and encryption that has to be enabled by the power company. That means they intentionally limit you to devices that they approve ... and usually cost a lot more than a small processor and an XBee running the ZigBee stack.

      It would be really interesting to look into this though. I suspect there is a way to hack into it if you have the right tool set to watch the interaction and overcome the encryption.

      Delete
    2. I don't have a smart meter. The power company comes out each month and actually reads my meter. Well, they come out, plug a device into the meter and download it. Smart meters are making their way to me, but the power company is targeting the people that have high daytime use first.

      Delete
  10. Hi Dave,
    After compiling the sketch successfully and start monitoring, the output is shown with many lines of frame ID counting up, but did not show any response address. The RSSI, RX and TX led were blinking, but I still could not control the Iris smart plug even I have already configured the Xbee module. Do we have to change any parameters or configuration in your blog? Thanks in advance.

    ReplyDelete
    Replies
    1. Here are the outputs:
      started
      0
      Light Off

      sent switch off 1 frame ID: 1

      sent switch off 2 frame ID: 2
      0
      Light Off

      sent switch off 1 frame ID: 3

      sent switch off 2 frame ID: 4
      0
      Light Off

      Delete
    2. If you did what I did here to the XBee, it should be set up correctly. Note, the Iris switch has to initiate the conversation. You should reset the switch and watch what happens.

      Delete
    3. I want to ask that, how can I reprogram the xbee network watch dog and channel verification in api mode? The X-CTU did not show the place to change the parameter. Sorry, I was a little new to xbee. I pressed 8 times on the switch button, but it seemed nothing happen. Thanks

      Delete
    4. Hi Dave,
      After changing from the wireless SD shield to Xbee sparkfun shield, it works like a charm. Thanks so much

      Delete
    5. Great. If you have such a problem again, just get out 4 jumper wires and hook it up directly not using a shield. Some of the shields have so many settings it's hard to figure them out.

      Delete
  11. SOS SOS i cannot access the drop box file that dave put the link to. can anyone help me access that library thanks

    ReplyDelete
    Replies
    1. I have that stuff in github now. The XBee library is at:

      https://github.com/draythomp/Desert-Home-Arduino/tree/master/libraries/XBee

      Delete