My First 'Real' Zigbee Sensor

I've made a lot of sensor devices that reported using Digi XBees. The little RF devices work quite well and set up a self healing mesh network that really does what is needed. These came before Zigbee and the devices we see now for home automation. It was the beginning of the Internet of Things (IOT), but it wasn't actually on the internet. Most of this blog is about developing these devices to serve some purpose: reporting data about or controlling something around the house. Bringing industrial controls home for a common person. 

Zigbee devices started appearing, using the same techniques I was familiar with but with a documented protocol that could make different manufacturers devices work together. However, the first implementations were crippled by the manufacturers such that anyone buying them would be stuck using a single manufacturer's equipment. Much like pool controllers that are locked into a particular manufacturer. The Iris devices promoted and sold by Lowe's were such devices. Myself and others broke into the protocols of the Iris devices and built our own controllers eliminating being locked down like that. 

That philosophy, thankfully, died in favor of a huge number of real Zigbee devices. Zigbee 2 MQTT was the crowning touch to this; it allowed any manufacturer's offering to connect and translated the protocol into MQTT topics that anyone could use to control their own house.

But, you can look up the history of IOT if you want to know more, this is about creating my first device that used the Zigbee protocol. In my last post I described how I implemented a swimming pool controller using commercially available Zigbee devices, but left something out: measuring the air temperature at the pool controller, and a contact sensor for a float in my septic tank on the same side of the house.

The ESP32C6 had been announced that supported native Zigbee and Matter. Then Seeed Studio announced a tiny Xiao device that packaged the C6 as a component similar in size to a Digi XBee. This little thing was exactly what was needed for me to get my feet wet creating a sensor to measure temperature and check the status of that float. So, I ordered one (actually a couple).

Taken right from their web site

This thing is a little bit bigger than my thumbnail.

It comes in a cool container

Yes, it's that tiny, and subsequently a bit hard to deal with, so I ordered the expansion board to play with it. 

Taken right from their web site

This was big enough to handle and work with. It has plugs for adding sensors and powering them as well as that cute little display for debugging or showing off. Plus, there were instructions on using the latest Arduino IDE to develop software for it. I'm very familiar with the Arduino IDE since I've been using it as long as it has existed. That was a real plus in my mind. 

I downloaded the IDE, installed the board specific software, and wrote my first program for the C6, "blink". Then I expanded to "Hello World,"  I was ready to dig into bringing up a Zigbee device, and then later, conquer the world.

The first thing I noted playing with the Zigbee examples supplied by the developers was that they were designed to connect a C6 controller to a C6 device. I wanted to connect a C6 device to Zigmee2mqtt and Home assistant; how does one go about that? I had to poke at it quite a bit to get my device to join, and it was really frustrating. There was a whole world of considerations that I didn't have in my development work under the Digi XBee model. At this point, I need to tell you about a way to cheat. You know how frustrating it is to look for an example of something like getting that little display on the expansion board to work, and even make it behave like a terminal and scroll information? It's really hard to search for it because you get 1.6 million hits, many of them on forums, and a huge number of youtube videos some of which are related, others that are not. Plus the usual totally unrelated hits that just drive you nuts. The answer to that for me is to use AI to search for me.

Let AI do the searching dealing with the ads, pop-ups, videos in the corner of the screen and give you back something that might actually help. I used it extensively to summarize discussions to a few real findings and translate the jargon into something I could understand. DO NOT let it write code for you. That would be great, but it puts in stuff you didn't ask for and inserts more errors than it solves, but for gathering information, it's a gem. Especially in today's ad-ridden internet.

I got the device to join with Z2M and then ran into problems with the data it was providing to Home Assistant. That led me to what's called a 'converter.' This is a piece of yaml code that describes the Zigbee clusters and what to do with the data. The documentation on these pieces of 'code' is odd and I had to prowl through it a lot to get some idea of what was going on. Also, Z2M had changed the way the converters worked and should be written, and the 'helpful' sites on the web were mostly too old, or just flat wrong. AI to the rescue, sort of. AI had the same problem I did, separating the junk from the real information. It kept suggesting things that just wouldn't work, and were a mix of the old and the new. However, it was pretty good at finding problems with the exceptionally weird combination of syntax, keywords, and indentation that plague yaml. Yaml was supposed to provide a way of creating code from a description without having to learn to code. Instead, you have to learn how to make a very elaborate description that is properly organized using specific keywords, indented to the way some programmer thought it should be. Basically, it is part of the future that one has to adapt to in order to get anything new done in the IOT field. Also, it sucks.

Net though, I got a device working. It measures the temperature using an 18B20 and looks for a contact closure. Two endpoints that show up just great in Z2M (Zigbee2mqtt) under Home assistant. And then later in any dashboard or card that I want to use. I didn't find anything that could do this particular combination anywhere on the internet. Heck, there's only a few contact sensors out there at all. Here it is as it looked while I was working on it.

The 18B20 is plugged in to one of the expansion board ports; I used the chip version at first and then changed to the waterproof version when I installed it outside. This is a picture of it installed and actually working in the same enclosure that I used for the other pool devices.

It has been VERY reliable in  the months that it has been running. I used the Seeed 'case' for the expansion board to protect it a bit. Of course, I broke the first one I used; the little crack repaired with super glue is visible in the photo. Here's a close up:

Sigh, one has to be careful with those tiny plastic parts. Hey, superglue is great stuff, might as well use it. Speaking of glue, that "Alien Tape" that used to be advertised all over the place is actually great stuff. I use it all over the place, but one has to be careful because sometime it lets loose. I used it to mount the board, and it has held up really well so far. The other stuff is that 3M double sided 'tape'; you use that to mount something and then let it set overnight, and it is there forever. Shortcuts for the various projects we develop.

If anyone is interested in the actual code for this, it will be available on github when I finally overcome my reluctance to relearn how to use it. 

The Demise (sort of) of the Pool Controller.

Once upon a time I was controlled by my swimming pool controller a Goldline P4. It was an expensive piece of hardware that I got when the pool was originally installed. That thing was said to be the pinnacle of pool control, but in about 20 minutes of working with it I saw shortcomings that were a bit annoying. But, I kept using and working with it. 

At that point I started trying to overcome the various short comings. I installed an RF remote control from the manufacturer that turned out to have limited range and was constantly having to be moved to another spot to communicate with the pool. I broke into the control system and set up my own input to the communication protocol between the pool components <Link>. I installed a fancy variable speed DC motor trying to save on power, the list went on and on and on. Search for swimming pool here on the blog to see how much work went into trying to control that device.

That's why the first line says, "controlled by."

Well, the expensive variable speed motor was a bust and got replaced by a two speed pump <Link>. The much touted chlorinator got turned off and only used to keep the pipes connected. Basically, all the frills and fluff that I succumbed to were removed and I was back to a basic pool that has a chlorine float and periodic shocks because all that fancy hardware just didn't work on high calcium well water and climate extremes of the desert. So, one day I asked myself, "Self, why are you putting up with this controller?"

The immediate answer was that if I replaced it with something else, how would I get the 24 volt power I needed and the high current relays that ran the motor and lights? When I discussed this with a friend, he looked at me like I was stupid and said, you have those already in that controller. A light went on, and a journey began.

Above is the architectural drawing of what I wound up implementing to solve all the annoyances of the Goldline controller. I just gave up on controlling the controller and disconnected its relays from the big board inside; I would use them myself some other way. The 24vac supply in the controller would provide the power for the pool valves, all I had to do was come up with a way to connect them. Plus, using Zigbee kept me from having to dig trenches or cut wall board. In my previous post I mentioned that I have an extensive Zigbee network that has a router right inside the wall where the device was going. Heck, Zigbee keeps the devices off wifi, and is really fault tolerant. Perfect way to go.  

Well, the answer was obvious to me, use a couple of those Zigbee relay devices and connect them through Zigbee2mqtt into Home Assistant where I could create dashboards, automations, etc. to my heart's content. 

I started off getting the relay devices; I used two different ones because I had one type on hand from some experiments I ran a year or two ago, and I ordered another one with a temperature sensor in it to get the water temperature. 

Now, I needed a box to put the stuff in because this was going to be outside in the sun and rain. That's a big deal in Arizona because the sun will rot almost anything over time. Plus, there had to be protection from the rodents that infest the desert. It certainly wouldn't do for a pack rat to steal all the shiny things and take them home to its den.

I picked this enclosure on Temu. It turns out they have a big selection of this kind of enclosure, and the price was way less that something similar from Home Depot. Now, how to arrange the stuff on the side of the house where the pool equipment is? I came up with this because it is really simple and wound up high enough that I wasn't sitting on the ground to work on it.

Just try to get into that you packrat !! 

\

Now, to put the parts and pieces into the box, wire it up and see if it generates magic smoke or actually works.

Plenty of room for more stuff

The beauty of a nice enclosure is that it hides the wires inside where no one knows how sloppy one is when a project is getting near to the 'live stage, you're all excited to try it out and just slap things together. Someday I'll get back to it and organize the wires better (yea, right). I did realize that I was missing something though. Yes, all the planning and ordering and waiting and I still forgot something. In the Arduino mega that this was replacing, I had the air temperature and a contact sensor for my septic tank level switch. Shucks, that means I have to look into those.

I got an Xiao ESP32C6 from Seeed Studio along with it's little expansion board, an 18B20 out of a box of them I have from something I never completed and made my very first Zigbee sensor. It has two sensors exposed in it one for the temperature and the other for the septic tank float contacts. This was a coding experience that I had never done before and was a bit of a challenge. But I got it going and even have the tiny little display scrolling with updates that I seldom look at. 

Yes, someday, I'll clean up the wiring...

A note on using the 18B20: Yes it's waterproof, and you can actually drop it into water and it will work just fine. However, how do you keep it in place? If the water is flowing, how do you keep it from leaking? If you're putting it into an environment where shiny stuff gets stolen by packrats, how do you protect it? The answer, Thermowells. 

I used them in two places for this project. One is in the picture above; it sticks out the bottom of the box in the shade to measure the air temperature, and the other is inserted into the pool water flow to measure the water temperature when the pool motor is running. 

This works great to keep leaks from driving me nuts and the rats from stealing the sensor. They solve a lot of mounting problems, and the threads allow commercial piping to work. Take a closer look at the way I did that. The T fitting and the plug are standard PVC fittings I picked up at ACE Hardware. I just threaded the thermowell into the hole with a little teflon tape. Leak free seal against the water pressure of the pool pump. I wish I had known about them years ago. Now about the wrapping I have on the wires. This stuff is new to me, and an attempt to keep the various rodents (squirrels, rabbits, rats, mice, etc) from tearing the wires apart. 

It claims it keeps cats from chewing the wires. I don't worry about cats (except the occasional bobcat or mountain lion), but I'm trying this to keep the rodents out. So far so good (months) because it's wire that wraps around the various cables exposed outside the boxes and conduit. I picked it up on Amazon. A warning though, wear gloves when you put it on, the wires can be damaging to fingertips.

But what about power to the new stuff? Well, there's 24VAC, 120VAC, probably 5VDC somewhere in the Goldline; I didn't use any of those. Instead, I used a tried and true technique (to me anyway). I used a small cell phone charger.

These little things cost roughly a dollar (US), and work fine for low current applications. They won't fast charge a cell phone, but they supply enough 5vdc to power this kind of electronics. Plus, you can get cables to adapt them to any device you happen to have. I plugged one of these into a plug on the side of the Goldline, ran it up into the new enclosure, and the split the power to each of the devices according to whatever kind of plug it had. In case you haven't run into that idea yet, there are USB power (not data) cables that split into two or more other plugs. Using the appropriate adapter, you can power anything, and not have a special buck converter. These little plugs have a wide input range 90-250, and actually work. I have them all over the place. I bought ten of them a few years back, and just used the last of them on another project (another 10 on order right now). I mean a power supply for a dollar?? Heck, I've used them to get 5vdc from a 240vac circuit and it held up well. Yes, occasionally they fail, but for a dollar, I don't care. 

Well, I had it physically constructed, and had wired the little relays in the left Zigbee relay controller to the big relays in the Goldline controller, and the right Zigbee relay controller handled the 24vac valves, so let's join this thing to Z2M and see what happens. Basically, it worked really well. Some of the usual messing around with naming the various Home Assistant devices and entities and then some dashboard work, and I had a control system for my pool that could do any darn thing I wanted. 

Victory !!

Now, if I was to do this over again with a new house, and a new pool, I would never even think about all the bells and whistles that I originally suckered into. A basic two speed motor, and some Zigbee devices hooked into Home Assistant would be all I need. Hayward could just find some other dude to sell their expensive, limited, failure prone hardware to. 

(Note: If you’re wondering why the septic sensor isn't live yet, let’s just say while I was moving gravel, I gave the local packrats a clear shot at the wires. They took it. Round one goes to the rodents.)

The little zigbee sensor using the ESPC6 was a great little project that you might be interested in; details on that project are upcoming. Stay tuned.

I'm back ! So, Let's Revisit the Hubitat

 A long while back, I touted how nice the Hubitat device was, and it really was great. I moved my entire home control system over to it. I added devices and was quite happy with the system.

Then, they came out with new hardware that was really compelling and I upgraded to it. The system offered a nice transition path from the old hardware to the new, and greater range and capabilities. I was stoked until some of my devices refused to work with it. All my old Iris stuff quit working, and I couldn't get my expensive Enbrighten Zigbee light switches to join. I contacted Hubitat about it, and of course, no one else was having the problems I was.

I even sent the device to them and they, of course, couldn't find any problem. But, I had an expensive piece of hardware that just wouldn't do the job for me. I was pissed when I found out that they intentionally excluded the Iris devices from being able to join the network! That was the important thing to me at that time; I had a lot of them from years past. What REALLY annoyed me was the very latest (at the time) Hubitat has a button to join the old Iris devices right on the page to join Zigbee devices. Of course that button only wasted my time trying to make it work. On the Hubitat forum, I wasn't the only one that complained. See the old link to the discussion here <Link>. My complaints are in there as well where people came rushing to Hubitat's defence. You know the drill on forums, pick at the question until the person gets mad and leaves. 

So, it was set up the old Hubitat to handle this Iris devices as well as the Enbrighten switches, and use the new Hubitat to handle new stuff as I expanded. The idea of holding on to an old controller just to support devices that I was already using didn't sit well. 

There was also the possibility of replacing the devices that wouldn't join the Zigbee network, but that was expensive, and I have not found anything that worked and looked as nice as the Enbrighten wall switches. The Iris devices could be easily replaced with less expensive devices and new technology, so that wasn't a factor. 

Or, I could drop the Hubitat and continue developing software under the system I brought up originally. That didn't set well with me because "Rolling my own" is a lonely path. It was great back when I started because IOT was still in its infancy and I was plowing new ground with every project. I had a lot of fun developing my own monitoring system for house power from the power company. It was also a lot of fun to get my air conditioning system under my complete control. However, that path diverged too far from what other people were doing. 

What did I do? I took the old Hubitat and used it as a Zigbee controller and brought up Home Assistant on an Intel NUC that I picked up on Ebay for a fraction of the price of the newest Hubitat. Then started to work setting up controls and monitors all over again.

That actually worked pretty well as a system, but I had a LOT of devices that I built up using the Digi XBee that was pre-Zigbee, and I wanted them in the system also. So, I interfaced them into Home Assistant using MQTT. Eventually, I wound up with two Intel NUCs, the old network and the new all Zigbee network. 

Everything worked, and I had a new system that supported everything and anything new I wanted to try. I went a little nuts and spent many days on dashboards, automations, helpers and such until I really had the house fine tuned to what I liked and how I wanted things to work here in Arizona. Then, using the mqtt interface in Home Assistant, I moved my code to the second NUC and got rid of the Hubitat entirely.

The Hubitats are sitting on a shelf somewhere totally ignored now. I'll never go back to that route again. See, what had happened was that I had missed out on the incredible progress of Home Assistant. It has Zigbee to MQTT integrated, so new Zigbee devices can be integrated into Home Assistant with a little work, and that is an awesome feature for automation and monitoring the house. Then, I tried out the ESP32 devices; that gave me the ability to create something that didn't exist on the market, and I was back in the heaven of exploring new ideas and implementing new ways of controlling things. Different platforms, but way cheaper and much more capable than the Arduino, Raspberry Pi evolution of the IOT field.

Thus, the last four or so years has been an exploration and implementation of catching up with IOT, and doing my own thing around the house. I decided to wake this blog up again and share my exploits and failures because there is an audience out there that wants to see what the reality is and what is possible without spending their entire life pawing through a forum being insulted for asking stupid questions or watching endless out of date videos enduring ad after ad to see how something could be done under some old release. 

I hope you enjoy my experiences as much as I have. You do your own thing for your environment, and I'll keep poking and experimenting, sharing what I've already done, and what I'm working on now. 

Oh, and I'll probably throw in a tractor, excavator, skid steer or rock moving story along the way.

Dave.