Hacking Into The Iris Door Sensor

Several people have asked me about the Iris Door Sensor, and I haven't had much interest. My battery operated switch can do this and is totally programmable, but I figured, "What the heck." I went ahead and got one when I went to Lowe's to get another smart switch.

Now, this is one of those projects that will go on and on as I try to get it to work. Meanwhile, these are the first steps.

This is what one looks like:

I don't think Lowe's will mind that I stole the picture from their site. First thing I did was open it up and look inside:

The spot on the left is for the battery and there's a switch for joining it to a network. I tried to get it to join with my Iris network, but it didn't work, so being me, I took it apart. Here's the circuit board removed from the casing:

Not much on this side, so flipping it over:

This is a side view of the magnet it comes with:

Remember those little extrusions on the magnet case; I'll talk about them later. I got out the magnifying glass and took a look at the active components, the list isn't very long:

Battery - CR2 Lithium, 3V
Magnetic Reed Switch - Hamlin 59050-1-S-00-0 Normally Open
Magnet off center to be near the reed switch.
Antenna - Antenova a6150
Processor - Silicon Labs Ember 250
Temperature Sensor - Microchip MCP9801M

Nice set of high quality components. this thing is put together pretty well. The important part is the Ember 250 SOC (system on a chip) that runs the device. These are made specifically for ZigBee and are very nice devices; to bad the product comes from Alertme. They always ignore the ZigBee spec and go their own way in critical areas. This is going to make it tough to hack into.

I put together some code and turned off my normal network to keep down interference and gave it a try. I couldn't get it to join properly, but I was able to see it trying. Since this is an Alertme device, it has significant API differences from the ZigBee spec, so I started trying things. I got it to join eventually by a non-reproducible accident and saw it running.

The problem is that I can't get that to happen again. When it was working, I noticed that it was really sensitive to where the magnet was, so I looked at the reed switch specs and it's only got a 7 mm range. That means that the magnet needs to be right next to the sensor for it to close. The FAQ for the switch on the Lowe's site even talks about that. They blame it on polarity when it's really the fact that the magnet needs to be so close to the sensor; like polarity would matter on a reed switch. A neo magnet might make the range longer, but it wouldn't be as pretty.

Some of us may have problems with that on installation; seven millimeters is not very much, especially when it's about 4 mm from the switch to the case. Do you see why they have those little protusions on the case now? It gives some spacing and a little place for the magnet to rub on the sensor case when the door closes. Strange way to build it, but I'm not an engineer.

About how it works: When the battery is first inserted, it starts sending Device Announce messages and continues this until I answer it with an Active Endpoint  Request (which it hasn't answered yet). When it gets the Active Endpoint, it sends a Match Descriptor request and I answer it with a Match Descriptor Response tailored to it. Then, it sends one of the odd Alertme messages to profile 0xC216 that I've seen before from Alertme devices.

This is the point where, with the smart switches, I send a set of canned responses back and the device would join. The door switch doesn't join, It will stay around for a while trying, but will eventually give up and move on to another channel to look for another controller.

The time I got it to join, it started sending messages to cluster 0x500, the security device cluster. While it was doing that I could actually see the switch change state because the values it sent back were changing based on magnet location. However, it was erratic and extremely slow at sensing the magnet. There was some LED flashing on the device that seemed to correspond to the position of the magnet, but I couldn't be sure.

I'm going to post the code I'm working with as it is right now, but it's not pretty. I hacked up a piece of code I developed to break into another ZigBee device and am making experimental changes to it trying to understand this door switch.

If you grab it, there's an 'if' statement up at the top where I get the packets from the XBee, I check the incoming source address and skip anything that isn't coming from the switch. I have a number of the Iris devices and they were messing up the experiment, so I just ignore them in the code. You'll have to change that to your own devices address.

#! /usr/bin/python
# Have fun

from xbee import ZigBee 
import logging
import datetime
import time
import serial
import sys, traceback
import shlex
from struct import *
'''
Before we get started there's a piece of this that drove me nuts.  Each message to a 
Zigbee cluster has a length and a header.  The length isn't talked about at all in the 
Zigbee documentation (that I could find) and the header byte is drawn backwards to
everything I've ever dealt with.  So, I redrew the header byte so I could understand
and use it:

7   6   5   4   3   2   1   0
            X                   Disable Default Response 1 = return default message
                X               Direction 1 = server to client, 0 = client to server
                    X           Manufacturer Specific 
                            X   Frame Type 1 = cluster specific, 0 = entire profile
                                    
So, to send a cluster command, set bit zero, and to get an attribute from a cluster
set bit 4.  If you want to be sure you get a reply, set the default response.  I haven't
needed the manufacturer specific bit yet.

'''

def printData(data):
    print "********** Message Contents"
    for key, value in data.iteritems():
        if key == "id":
            print key, value
        else:
            print key, "".join("%02x " % ord(b) for b in data[key])
    print

def getAttributes(data, thisOne):
    ''' OK, now that I've listed the clusters, I'm going to see about 
    getting the attributes for one of them by sending a Discover
    attributes command.  This is not a ZDO command, it's a ZCL command.
    ZDO = ZigBee device object - the actual device
    ZCL = Zigbee device cluster - the collection of routines to control it.
        Frame control field, bit field,  first byte
            bits 0-1, frame type field
                00 entire profile
                01 specific to a particular cluster
                10-11 reserved (don't use)
                Note, if you're sending commands, this should be 01
                if you're reading attributes, it should be 00
            bit 2, manufacturer specific, if this bit is set, include
                   the manufacturer code (below)
            bit 3 direction, this determines if it is from a client to 
                   server.  
                 1 server to client
                 0 client to server
                 Note, server and client are specific to zigbee, not
                 the purpose of the machine, so think about this.  For
                 example to turn an on/off switch on, you have to be the
                 server so this bit will be 01
            bit 4 disable default response
            bits 5-7 reserved (set to zero)
        Manufacturer code,  either 2 bytes or not there 
        Transaction sequence number, byte
        Command identifier, byte
        Frame payload,  variable, the command bytes to do something
        
        frame control bits = 0b00 (this means a BINARY 00)
        manufacturer specific bit = 0, for normal, or one for manufacturer
        So, the frame control will be 0000000
        discover attributes command identifier = 0x0c
        
        then a zero to indicate the first attribute to be returned
        and a 0x0f to indicate the maximum number of attributes to 
        return.
    '''
    print "Sending Discover Attributes"
    zb.send('tx_explicit',
        dest_addr_long = data['source_addr_long'],
        dest_addr = data['source_addr'],
        src_endpoint = '\x00',
        dest_endpoint = '\x01',
        cluster = thisOne, # cluster I want to know about
        profile = '\x01\x04', # home automation profile
        # means: frame control 0, sequence number 0xaa, command 0c,
        # start at 0x0000 for a length of 0x0f
        data = '\x00' + '\xaa' + '\x0c'+ '\x00' + '\x00'+ '\x0f'
        )

# this is a call back function.  When a message
# comes in this function will get the data
def messageReceived(data):
    
    try:
        # This is the long address of my door switch device
        # since I have several other devices and they are transmitting
        # all the time, I'm excluding them and only allowing the
        # door switch in
        if data['source_addr_long'] != '\x00\x0d\x6f\x00\x03\xc2\x71\xcc':
            return
            
        print 'gotta packet',
        #print data
        if (data['id'] == 'rx_explicit'):
            print "RF Explicit"
            #printData(data)
            clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1])
            print 'Cluster ID:', hex(clusterId),
            print "profile id:", repr(data['profile'])
            if (data['profile']=='\x01\x04'): # Home Automation Profile
                # This has to be handled differently than the general profile
                # each response if from a cluster that is actually doing something
                # so there are attributes and commands to think about.
                #
                # Since this is a ZCL message; which actually means this message is 
                # is supposed to use the ZigBee cluster library to actually do something
                # like turn on a light or check to see if it's on, the command way down
                # in the rf_data is important.  So, the commands may be repeated in
                # each cluster and do slightly different things
                #
                # I'm going to grab the cluster command out of the rf_data first so 
                # I don't have to code it into each cluster
                #print "take this apart"
                #print repr(data['rf_data'])
                if (data['rf_data'][0] == '\x08'): # was it successful?
                    #should have a bit check to see if manufacturer data is here
                    cCommand = data['rf_data'][2]
                    print "Cluster command: ", hex(ord(cCommand))
                else:
                    print "Cluster command failed"
                    return
                # grab the payload data to make it easier to work with
                payload = data['rf_data'][3:] #from index 3 on is the payload for the command
                datatypes={'\x00':'no data',
                            '\x10':'boolean',
                            '\x18':'8 bit bitmap',
                            '\x20':'unsigned 8 bit integer',
                            '\x21':'unsigned 24 bit integer',
                            '\x30':'8 bit enumeration',
                            '\x42':'character string'}
                print "Raw payload:",repr(payload)
                # handle these first commands in a general way
                if (cCommand == '\x0d'): # Discover Attributes
                    # This tells you all the attributes for a particular cluster
                    # and their datatypes
                    print "Discover attributes response"
                    if (payload[0] == '\x01'):
                        print "All attributes returned"
                    else:
                        print "Didn't get all the attributes on one try"
                    i = 1
                    if (len(payload) == 1): # no actual attributes returned
                        print "No attributes"
                        return
                    while (i < len(payload)-1):
                        print "    Attribute = ", hex(ord(payload[i+1])) , hex(ord(payload[i])),
                        try:
                            print datatypes[payload[i+2]]
                            i += 3
                        except:
                            print "I don't have an entry for datatype:", hex(ord(payload[i+2]))
                            return
                            
                if (clusterId == 0x0000): # Under HA this is the 'Basic' Cluster
                    pass
                elif (clusterId == 0x0003): # 'identify' should make it flash a light or something 
                    pass
                elif (clusterId == 0x0004): # 'Groups'
                    pass
                elif (clusterId == 0x0005): # 'Scenes'  
                    pass
                elif (clusterId == 0x0006): # 'On/Off' this is for switching or checking on and off  
                    print "inside cluster 6"
                
                elif (clusterId == 0x0008): # 'Level'  
                    pass
                else:
                    print("Haven't implemented this yet")
            elif (data['profile']=='\x00\x00'): # The General Profile
                if (clusterId == 0x0000):
                    print ("Network (16-bit) Address Request")
                    #printData(data)
                elif (clusterId == 0x0004):
                    # Simple Descriptor Request, 
                    print("Simple Descriptor Request")
                    #printData(data)
                elif (clusterId == 0x0008):
                    # I couldn't find a definition for this 
                    print("This was probably sent to the wrong profile")
                elif (clusterId == 0x0013):
                    # This is the device announce message.
                    print 'Device Announce Message'
                    printData(data)
                    
                    print "sending Active Endpoint Request "
                    zb.send('tx_explicit',
                        dest_addr_long = data['source_addr_long'],
                        dest_addr = data['source_addr'],
                        src_endpoint = '\x00',
                        dest_endpoint = '\x00',
                        cluster = '\x05\x00',
                        profile = '\xc2\x16',
                        # The first item is a number to identify the message
                        # The next two are the short address of the device
                        data = '\x12' + data['source_addr'][1]+ data['source_addr'][0]
                    )

                    
                elif (clusterId == 0x8000):
                    print("Network (16-bit) Address Response")
                    #printData(data)
                elif (clusterId == 0x8038):
                    print("Management Network Update Request");
                elif (clusterId == 0x8005):
                    # this is the Active Endpoint Response This message tells you
                    # what the device can do
                    print 'Active Endpoint Response'
                    printData(data)
                    if (ord(data['rf_data'][1]) == 0): # this means success
                        print "Active Endpoint reported back is: {0:02x}".format(ord(data['rf_data'][5]))
                        
                    print("Now trying simple descriptor request on an endpoint")
                    zb.send('tx_explicit',
                        dest_addr_long = data['source_addr_long'],
                        dest_addr = data['source_addr'],
                        src_endpoint = '\x00',
                        dest_endpoint = '\x00', # This has to go to endpoint 0 !
                        cluster = '\x05\x00', #simple descriptor request'
                        profile = '\xc2\x16',
                        data = '\x13' + data['source_addr'][1] + data['source_addr'][0] + '\x01'
                    )
                elif (clusterId == 0x8004):
                    print "simple descriptor response"
                    try:
                        clustersFound = []
                        r = data['rf_data']
                        if (ord(r[1]) == 0): # means success
                            #take apart the simple descriptor returned
                            endpoint, profileId, deviceId, version, inCount = \
                                unpack('<BHHBB',r[5:12])
                            print "    endpoint reported is: {0:02x}".format(endpoint)
                            print "    profile id:  {0:04x}".format(profileId)
                            print "    device id: {0:04x}".format(deviceId)
                            print "    device version: {0:02x}".format(version)
                            print "    input cluster count: {0:02x}".format(inCount)
                            position = 12
                            # input cluster list (16 bit words)
                            for x in range (0,inCount):
                                thisOne, = unpack("<H",r[position : position+2])
                                clustersFound.append(r[position+1] + r[position])
                                position += 2
                                print "        input cluster {0:04x}".format(thisOne)
                            outCount, = unpack("<B",r[position])
                            position += 1
                            print "    output cluster count: {0:02x}".format(outCount)
                            #output cluster list (16 bit words)
                            for x in range (0,outCount):
                                thisOne, = unpack("<H",r[position : position+2])
                                clustersFound.append(r[position+1] + r[position])
                                position += 2
                                print "        output cluster {0:04x}".format(thisOne)
                            clustersFound.append('\x0b\x04')
                            print "added special cluster"
                            print "Completed Cluster List"
                    except:
                        print "error parsing Simple Descriptor"
                        printData(data)
                    print repr(clustersFound)
                    for c in clustersFound:
                        getAttributes(data, c) # Now, go get the attribute list for the cluster
                elif (clusterId == 0x0006):
                    print "Match Descriptor Request"
                    # Match Descriptor Request
                    printData(data)
                    # Now the Match Descriptor Response
                    print "Sending match descriptor response"
                    zb.send('tx_explicit',
                        dest_addr_long = data['source_addr_long'],
                        dest_addr = data['source_addr'],
                        src_endpoint = '\x00',
                        dest_endpoint = '\x00',
                        cluster = '\x80\x06',
                        profile = '\x00\x00',
                        #seq #, status, address,num endp, list
                        data = '\x00\x00\x00\x00\x01\x02')
                else:
                    print ("Unimplemented Cluster ID", hex(clusterId))
                    print
            elif (data['profile']=='\xc2\x16'): # Alertme Specific
                printData(data)
                '''print "Sending weird messages"
                if (clusterId == 0x00f6):
                    payload3 = '\x11\x01\x01'
                    zb.send('tx_explicit',
                        dest_addr_long = data['source_addr_long'],
                        dest_addr = data['source_addr'],
                        src_endpoint = '\x00',
                        dest_endpoint = data['source_endpoint'],
                        cluster = data['cluster'],
                        profile = '\xc2\x16',
                        data = payload3
                        )
                    payload4 = '\x19\x01\xfa\x00\x01'
                    zb.send('tx_explicit',
                       dest_addr_long = data['source_addr_long'],
                       dest_addr = data['source_addr'],
                       src_endpoint = '\x00',
                       dest_endpoint = data['source_endpoint'],
                       cluster = data['cluster'],
                       profile = '\xc2\x16',
                       data = payload4
                    )'''
                    
                if (clusterId == 0x00ef):
                    pass
                elif (clusterId == 0x00f0):
                    pass
            else:
                print ("Unimplemented Profile ID")
        elif(data['id'] == 'route_record_indicator'):
            print("Route Record Indicator")
        else:
            print("some other type of packet")
            print(data)
    except:
        print "I didn't expect this error:", sys.exc_info()[0]
        traceback.print_exc()
        
def sendSwitch(whereLong, whereShort, srcEndpoint, destEndpoint, 
                clusterId, profileId, clusterCmd, databytes):
    
    payload = '\x11\x00' + clusterCmd + databytes
    # print 'payload',
    # for c in payload:
        # print hex(ord(c)),
    # print
    # print 'long address:',
    # for c in whereLong:
        # print hex(ord(c)),
    # print
        
    zb.send('tx_explicit',
        dest_addr_long = whereLong,
        dest_addr = whereShort,
        src_endpoint = srcEndpoint,
        dest_endpoint = destEndpoint,
        cluster = clusterId,
        profile = profileId,
        data = payload
        )

#------------ XBee Stuff -------------------------
# this is the /dev/serial/by-id device for the USB card that holds the XBee
ZIGBEEPORT = "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A901QL3F-if00-port0"
ZIGBEEBAUD_RATE = 9600
# Open serial port for use by the XBee
ser = serial.Serial(ZIGBEEPORT, ZIGBEEBAUD_RATE)


# The XBee addresses I'm dealing with
BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff'
theSwitch = '\x00\x0d\x6f\x00\x03\x58\x05\xc2'
UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address

#-------------------------------------------------
logging.basicConfig()

    
# Create XBee library API object, which spawns a new thread
zb = ZigBee(ser, callback=messageReceived)

print ("started")
notYet = True;
firstTime = True;
while True:
    try:
        time.sleep(5)
        if (firstTime):
#           sendSwitch(whereLong=theSwitch, whereShort=UNKNOWN, srcEndpoint='\x00', 
#                   destEndpoint='\x00', clusterId='\x00\x00', profileId='\x00\x00', 
#                   clusterCmd='\x00', databytes='\x00')
#            print "sending Active Endpoint Request "
#            zb.send('tx_explicit',
#                dest_addr_long = theSwitch,
#                dest_addr = '\x94\x65',
#                src_endpoint = '\x00',
#                dest_endpoint = '\x00',
#                cluster = '\x00\x05',
#                profile = '\x00\x00',
#                # The first item is a number to identify the message
#                # The next two are the short address of the device
#                data = '\x12' + '\x65' + '\x94'
#            )
#            time.sleep(10)
#            print "sending 'configure reporting'"
#            zb.send('tx_explicit',
#                dest_addr_long = theSwitch,
#                dest_addr = '\x94\x65',
#                src_endpoint = '\x00',
#                dest_endpoint = '\x01',
#                cluster = '\x00\x06', # cluster I want to deal with
#                profile = '\x01\x04', # home automation profile
#                data = '\x00' + '\xaa' + '\x06' + '\x00' + '\x00' + '\x00' + '\x10' + '\x00' + '\x00' + '\x00' + '\x40' + '\x00' + '\x00'
#            )

            firstTime = False
        print ("tick")
        
        sys.stdout.flush() # if you're running non interactive, do this

    except KeyboardInterrupt:
        print ("Keyboard interrupt")
        break
    except:
        print ("I didn't expect this error:", sys.exc_info()[0])
        traceback.print_exc()
        break

print ("After the while loop")
# halt() must be called before closing the serial
# port in order to ensure proper thread shutdown
zb.halt()
ser.close()

I told you it wasn't pretty, so stop complaining.

I'm continuing to play with this switch as time, and new ideas permit, so I think the door switch can be conquered. However, if is as flaky as the initial results would indicate, I may not be using it. Having to have the magnet right up against the sensor and having so little elbow room would stop the switch from being useful on things like garage doors, doors that move due to weather or settling, wooden windows that may not shut all the way, that kind of thing. If someone has one that works well in an Iris setup can tell us about this switch, it would be great.

If you want, grab the code, experiment, play, and let me know the results you get.

The next post on this project is here <link>.

My New Network Attached Storage

Well, I did it. I finally broke down and bought a NAS (network attached Storage). I researched these things for days looking for something that would fit my needs and couldn't make up my mind. I was tempted to just take a Raspberry Pi and add a disk drive, but I wanted more capability in the device than that would have given me. I asked other folk, and it came down to two manufacturers of this kind of device: Synology and QNAP. Frankly, they both sound great and have more good reviews than bad.

I settled on QNAP, not because it won out in the specification, but because I caught a nice sale. TigerDirect had the QNAP TS-231 on sale for $30 bucks off, so I ordered it. I got two 3TB WD red drives from Amazon and set it up. It's up in my attic data center working its little heart out.

When I was looking at what could be run on it, MySQL was listed. That meant that I could put my house databases up there and get them off the Pi's I have running. It also meant that I could expand the database and actually log data locally for my house.

Wow, that would mean all my griping about cloud services would end.

First step, set up a MySQL database on the server; that took about 10 minutes. Then understand how to put data on it; that took a day. Then convert my code to use the new databases on the NAS; that took a stinking week. The very basic use of MySQL is different enough from the way to use SQLite3 that I had to learn a whole lot to get it working.

But, when I did, it really sped things up. I'm not clear on why exactly; I'm making data base transactions across an Ethernet link instead of locally, but it runs faster. Since the NAS is set up Raid 1, I have 3TB to play with and I expect the databases to use a hunk of that so it may slow down over time as the data bases grow.

My impetus for finally biting the bullet on one of these devices was storage failures on the Raspberry Pi. That's the most significant problem I've had with the Pi, the SD cards wear out, and I had worse luck using a USB stick. Well, I won't be wearing the cards out as fast, and I don't need the extra space of a USB stick anymore. However, the SD cards will still wear out because all the system logs and such are beating on them.

To this end, I'm going to put the operating system on the NAS as well, but I haven't gotten that far yet. The idea is to put everything I can on the NAS and only boot the Pi from the SD card. That should make it last a while. Sort of the best of both worlds; the separate processor and hardware of the Pi combined with the storage and reliability of a NAS. I'll inherit the possibility of things quitting if the Ethernet fails, but my house already relies on Ethernet anyway. I have the various Arduinos set up to work in a default mode, so that problem hasn't gotten any worse ... I think.

One of the side-effects (affects?? I can never remember which one to use) is that I cleaned up a bunch of code, removed things I really wasn't using any more and restructured a lot of my processes. So, most of the processes that wrote data to the various cloud services I looked into are gone now. The code is still out there on GitHub (the internet is forever), but it's not in the latest update. If you need it, just look at a previous commit.

Another thing that changed is since I'm not updating the various clouds, the live graphs I have scattered in earlier posts may not work any more. I replaced some of them with pictures to keep it somewhat coherent, but gave up after a few. Also, the architecture of the overall house control and monitoring is a bit different. I guess I'll delete the page on that and try to create a new picture and description soon.

I'm keeping my legacy feeds on Xively running for now since the majority of my data is there. I wouldn't mind losing that data, I'll make more, but I want to annoy them as long a possible with a personal feed. They've been hinting that they may remove the personal feeds for a while now, and I just plan on staying there until they kick me off.

But, this thing is so cool. It just works and accepts transactions without complaining at all. MySOL is way capable and can do things I hadn't even thought of.

So, Dave's place has Arduinos running things like the garage, pool and air conditioning that talk to a Raspberry Pi that saves it's status to a NAS. There's also a weather head that sends RF data to another Raspberry Pi that saves weather data cumulatively to the NAS as well.

The code for the new way of doing things is out on GitHub, if you want to grab something.

Unfortunately, this exercise has given me a bunch of ideas on things I can do to improve the operation. Now that I can essentially have unlimited Raspberry Pi devices doing things and coordinating activities of other devices, I can put the house configuration file on the NAS and read it from any Ethernet connected device, I can separate the web services from everything else and increase house Ethernet security. I can save data on each room using the new temperature sensor in various ways. I can ... I can go completely nuts,

If I haven't already.

Battery Operated Temperature Sensor: OK, Let's Build This Thing

The previous post in this series is here <link>.

Well, I've been testing, trying new ideas and even added a button to this thing for the last several weeks. I don't think I'm going to learn anything new in the short term, so I decided to solder everything down and turn it into a package that I can actually use.

I got some of these:

Those of you that have been following this project understand now why I wanted to confine the parts to half a breadboard. I can put all the parts on this and then have something I can put in some kind of enclosure. I stripped all the parts off the breadboard and put them on the new board.

Yep, I got the regulator, XBee, Arduino Mini Pro, sensor, and a push button on there. I mounted the active components in sockets so they'd be easy to change if I needed to. I also put a plug on the battery pack so I can unplug it if needed. I still have the monitor circuit for watching the battery level, that way I can tell when the batteries are starting to go flat.

It's a nice little package, even if I do have to say so myself. If I put the battery underneath the board, it is pretty compact as well:

I think I'll put it in a travel container for a bar of soap to play with it around the house for a few months. Here's the schematic for the final result:

I'm going to the 3V version of the MCP1700 as soon as they come in from Mouser. Those little devices are perfect for battery applications. The 3V version (opposed to the 3.3V) worked well with this combination of components because the battery drops to just under 2.9V before it shuts off. That way I've dropped the batteries down to a volt each, and there's essentially nothing left in there.

Notice the voltage divider on the battery input? That combination of values works well to monitor the battery voltage, but I had to calibrate it in the code. It's pretty easy, get your meter, measure the battery voltage and then adjust the code until you get the same reading ... done. This will hold essentially forever, but if you want to check it yearly or so, it's easy.

This particular device (there WILL be others) is going beside my bed. I'll have the temperature of my bedroom as well as a button to turn the light off eliminating the X10 controller I have there now. Remember, this is a smart device, I can program it to do other things over time, and probably will. I think having one in the guest room with a door sensor attached would be fun. I may put one in the attic to monitor the heat up there, and one hooked to a moisture sensor out in the garage would be great for keeping track of the water heater.

How long will the batteries last? I don't know exactly, but it should be many months, a little less than a year. I can't measure the power usage, it's too low for anything I have. I removed the led from the Arduino board and feed the power behind the power regulator, so I have no losses there. I suspect it's using around 10 micro amps when asleep and not much when it's running. I'm going to set it to 5 seconds awake and 115 seconds asleep which will cut the drain in half compared to the level I ran it at in testing. If it runs down too soon, there's still several things I can do, but they won't actually gain me very much. Slower clock speed, mosfet switch for the power to the sensor and XBee, that kind of thing. The one thing that will really increase the time between battery changes is to simply use four batteries. The regulator I used can take it and it's not much more space. I'll try the three battery setup for as long as it runs to get a feel for what's going on.

My new NAS came in and is installed, so I guess I'll move on to something on it now. I'll build more of these as time permits, I have enough XBees for a bunch.

The next entry on this project is here <link>

Battery Powered Temperature Sensor: Remote Control Added

The previous post in this series is here <link>.

So, I crawled into bed the other night and reached over to turn off the light and it didn't work. Poked the X10 control button several times and nothing. Crap! I had to get out of bed, go to another room and turn off the light from there. The next morning I replaced the button battery in the control and everything was fine again.

But this got me to thinking; this is the only X10 device I have left from years ago and it's time to think about doing something different. X10 failed completely in my house except for the room farthest away from everything else and it has been flaky for some time now. But, how would I implement a remote control that can just hang on the wall for months and get used maybe once a day?

Put a button on my battery operated temperature sensor! It's still in testing, but it appears to be working great, All I'd need to do is put a button on it and cause an XBee message to be sent to my house controller and then I can choose some switch device to put on the light circuit and have my button beside the bed. It would measure the temperature periodically and just wait for a button press to send the signal.

Sure, I could use my cell phone with the (brand spanking new) app I made for the house, A tablet, or a web browser. I could even call my neighbor and tell them to turn the light off for me, but I want a simple button on the nightstand that I can press without any hassle and turn the silly light off. But, how to do it without draining the battery?

Interrupts, yes, that's the ticket.

Remember from the previous (ton) of posts on this device, I put the device to sleep for 55 seconds at a time and wake it up for 5 seconds to transmit the battery level and temperature, all I should have to do is to hook a button to an interrupt pin and add some code to send a message to the house controller. At the controller I can add some code to do anything I have a controller for. I could open the garage door with it if I want to.

So, a few hours later after about 25 false starts, I got it working. I connected a little switch between pin 2 and ground of the Arduino board I'm using and it causes an interrupt which breaks the arduino out of sleep and then I just send the normal temperature message which now includes a new field to indicate that the button was pushed. At the house controller, I parse out the field, look at the value and do whatever I program in.

The reason it took several tries to get it working was because I was making it too hard. I thought I'd have to have a lot of special code to support the interrupt and proceeded down that path. It turned out the JeeLab routine I'm using for sleep already supports other interrupts, and all I had to do was check a variable and add a simple interrupt handler and everything worked ... well mostly.  Here's a code fragment showing how I sensed that the button had been pressed:

 if (millis() - savedmillis > AWAKETIME){
    Serial.print("was awake for ");
    Serial.println(millis() - savedmillis);
    delay(100); // delay to allow the characters to get out
    savedmillis = millis();
    digitalWrite(xbeeSleepReq, SLEEP); // put the XBee to sleep
    while (digitalRead(xbeeCTS) != SLEEP){} // Wait 'til it actually goes to sleep
    unsigned long timeSlept = 0;
    int result = 1;
    while (timeSlept < SLEEPTIME){
      attachInterrupt(0, buttonThing, LOW);
      result = Sleepy::loseSomeTime((unsigned int)7000);
      if (result == 0) // this is something other than a watchdog
        break;
      timeSlept = (millis() - savedmillis);
    }
    Serial.print("was asleep for ");
    Serial.println(millis() - savedmillis);
    savedmillis = millis();
    if (result == 0)
      Serial.println("Woke up on a button press");
    digitalWrite(xbeeSleepReq, AWAKE); // wake that boy up now
    sendStatusXbee();
  }

I simply look at the return value from loseSomeTime() and if it's 0, then some interrupt other than the watchdog timer happened. I print a message for debugging and go to the XBee send code. The trick is in the interrupt handler:

// This is for the remote control buttons
void buttonThing(){
  sleep_disable();
  buttonPressed = true;
  detachInterrupt(0);
}

The variable 'buttonPressed' is simply a global variable that I set to true and look at in the XBee send routine. If the variable is true, I set a new field in the mesage to say so, if it's false, I say 'nothing' in the field. I'll eventually work out what would be the best thing to put in the message, but that parts easy once you get it working. Notice that I attachInterrupt() for interrupt zero, which means button two on the Arduino, just before going to sleep, and detach it after the button is pressed. That prevents multiple interrupts from button bounce. The disadvantage to doing it this way is that it won't respond to a button press during the 5 seconds the board is awake. I don't currently consider this a problem, but that might change if I want to use it for something else.

So, now if I push the button, it gets an interrupt on pin 2, breaks out of sleep, sends the temperature and an indicator that the button was pushed to the house controller. The house controller records the temperature, then looks at the new field and does whatever I want.

The Xbee send code looks like this

void sendStatusXbee(){
  xbeeReadyWait(); // Make sure the XBee is ready

  char *command;
  if (buttonPressed)
    command = "toggle";
  else
    command = "nothing";
  buttonPressed = false;
  sprintf(Dbuf, "{\"%s\":{\"name\":\"%s\",\"temperature\":\"%s\",\"command\":\"%s\",\"voltage\":\"%s\"}}\n", 
            deviceType, // this happens to be a temperature sensor
            deviceName, // originally read from the XBee
            dtostrf(readTemp(), 4, 1, t),
            command,
            dtostrf(readVcc(), 5, 3, v) // This is a text conversion of a float
            );
  Serial.print(Dbuf); // notice this is only for the serial port
  sendXbee(Dbuf);     // out to the XBee
  Serial.println("Message sent");
}

Not much to it, just a field called 'command' in the JSON string that I can look at when the message comes in. Currently I'm sending the word 'toggle' if the button was pressed and 'nothing' if it wasn't; I love being able to read the messages, so that was descriptive enough for me.

There's another interrupt pin on the Arduino, pin 3, and I can use that also for two buttons. Maybe an on-off toggle. I could also multiplex it by putting a keypad on there. First button wakes up the Arduino and then watch for letters, digits, whatever and compose a more complex command. For now, It's just a button that turns off my bedside lights. 

Tomorrow though, it'll be a button that turns off the bedside light, turns off the outside lights, sets the thermostat to the temperature I use at night, closes the garage doors (if left open), and maybe turns off the water heater power. Call it my 'going to bed now' button.

Now, a bit about the sensor testing. I had to put in a voltage divider to measure the input power because it was driving me nuts monitoring the power after the regulator. To do this, use really high value resistors for the divider and then a .1 cap on the analog pin to ground to lower the impedance. The arduino has an input impedance of around 10K on the input pins, and if your external circuitry is too high, it will give the wrong readings. The easiest way to overcome this problem and still not use very much power is to use a cap. I used 10 Meg and 1 Meg, which put my quiescent current below a micro amp, that should be fine.

So watching it run on the almost dead batteries I installed, it started failing at around 3.6 volts. The regulator on the Pro Mini I'm using was junk. First, it didn't regulate well, the voltage wandered around 3.9 V and eventually dropped off to nothing when the supply went to 3.6. This is not what the spec sheet says. Fine, I have a solution to that, I'll just put a MCP1700 regulator in place and started it back up. Take a look at the specs for the MCP1700, it's a totally awesome low current, low dropout regulator designed for this kind of thing.The new regulator worked great and I've been running with 'dead' batteries for several days now.

I did kind of mess up though. I should have ordered a different MCP1700. I chose the 3.3V version, and I think I should have gone for a lower voltage. Fortunately, they cost me 37 cents each, so I haven't lost much money if I decide to go lower. I'll know more when I hit the cut off point on the 3.3V and how it performs when I bottom out the batteries.

Just for giggles, here's a picture of it right now:

I thought it was a mess of wires a few weeks ago, now you can't see some of the components for the wires connecting things. I haven't tried to condense it yet, I'll get to that later. The new switch is between the batteries and the other stuff so I can get to it. I plan on actually using it for a while to be sure it will do the job for me.

Overall though, this little bundle of wires has really performed well. This learning curve has actually been fun and gave me ideas for other things I can build around the house. Remember, it currently senses temperature, it could easily sense anything else you want to throw at it. Suppose you put a pressure pad in front of the driveway, you could tell when someone pulled in. A light sensor to tell you if the lights in the shed were left on. A moisture sensor beneath the water heater to tell you if it started leaking. An air flow sensor in one of the ducts running through the ceiling. All of these things could be at my beck and call.

One more step in my quest to conquer the world.

The next post in this series is here <link>.

Getting Back To The Weather Station

I know I've worn out my welcome about the AcuRite 5n1weather station, sorry. This time it's a bit different and I'm really, really open to input and ideas. I decided that the console that came with the weather sensor is too strange and quirky to deal with long term and set up a Raspberry Pi to act as my weather station. I used a radio and directly receive the transmissions from the weatherhead <link> instead of the USB interface I worked on forever <link>

I have a lot of work to go on it, but I'll update it over time as I figure out what I actually want. To date, I set up a little SQLite3 database and record each of the readings there on a periodic basis. Currently, I update each record either as it comes in, or every minute. The reason for the difference is that my barometer and outside temperature sensor is on my XBee network and handled by a different machine, so I only interrogate it on a minute basis.

Yes, the little Pi weather station is a separate machine and I have it talk to the Pi that controls my house to get the two readings.

Actually, let me elaborate on this a bit. Remember way back when I posted about bringing up an HTML interface for the various processes I run using CherryPy <link>? That little effort worked incredibly well. For most processes I have at least two interfaces, one that I can read and one that is formatted in JSON that a machine can read. So, to check on a particular process, I just type the IP address and a port number into the browser and the process responds so I can see whatever I set up. This interaction doesn't care what machine it's on or where the request came from. Using this I have the weather software on one Pi, then get the readings for barometric pressure and outdoor temperature from another one. It's so cool.

I may expand on this idea and have several Pi's running doing different things for me. I could put the XBee network on a machine by itself and have other things talk to it to get the data. Sure it raises the risk of failure because it increases the number of machines that can fail, but It also means I have all the power of a Pi to do a task and not have to worry about things bogging it down. I may move all the upload processes to a single machine so that stuff is separate as well.

But enough of that, I'm here to talk about the weather station. I keep data on a periodic basis for: Rain counter, Outside Temperature on the fence, Outside Temperature at the weatherhead, Barometric Pressure, Wind Speed, Wind Direction, and Humidity. I also keep a daily record, written at midnight that hold the Barometric pressure at that moment, the high and low temperature for the day, and the rain counter at that moment. I create this record once a day, so I can use it as a daily data point on a graph.

From this I can calculate the daily rainfall, and when I have enough data, the weekly, monthly, etc for my location. I have enough data to graph the temperature on a daily basis, and even use the midnight record to go longer term. That stuff, of course, will depend on gathering the data over time. Or, I could download the data from Xively and fill it in for the past if it becomes relevant.

This looks like it will eat up about a megabyte a week in space, and I have roughly 4 Gig available on the Pi, so I have some time to get an NAS online to hold the data long term.

But, as usual, I have no idea what the heck I'm doing. I don't have a clue what I may want next month, or what is the best way to gather the stuff that I don't know about yet. The station is running right now, but it isn't hooked into the web yet so I can't show it off. I'll get to that in a week or so.

I looked into loading the stuff up to Weather Underground, but that didn't excite me very much. I've been fighting cloud providers for a few years now and I'm not impressed too much anymore. With the terabytes of NAS storage available these days and the ease of bringing up a significant little computer, who needs the headaches of specialized protocols and changing terms of service?

So, you folk that have weather stations and such, what am I missing?

Oh, the code isn't on GitHub yet, I'm not sure how I want to arrange it there. I may set up another repository for this instead of adding it to what's already there. I need to think about this a while. But, if anyone wants to take a look, let me know and I'll move more quickly and make it available.

The next post on this project is here <link>.

Battery Operated Temperature Sensor, yet another revision.

Previous post on this project is here <link>.

This little project has certainly taken on a life of its own. I've been at this for several weeks now watching what goes on with batteries and various components and software. My latest set of revisions is almost a start-over.

I removed the Ardweeny I was using and replaced it with an Arduino Pro Mini. The little Pro Mini is designed for folk that want the power of an Arduino without the bells and whistles. There's no place to plug in wires, no screw holes to mount it, take a look:

The disadvantage is that it has an LED and a voltage regulator on it. Voltage regulators waste power by having a ground reference and a small amount of current through it that can run my batteries down. However, this regulator is a SE5509BALG and it has a ground current of  only 21 uA, so that may not be a problem. I've run across folk that pull the regulator out to decrease the current draw, but I need some kind of regulator to supply the XBee with 3 volts. More on that later.

The LED is no problem, I can just pull it off the board if I can see it well enough.

I also switched from the TMP36 temperature sensor to an 18B20 one wire device. The reason I didn't start with the 18B20 was that I didn't have one to try. During the testing of battery operation, I ordered some and decided that now was the time to try them out. My decision to switch was partly curiosity and partly laziness. When I tested my code on the Pro Mini, I found a problem with the analog to digital converters. It seems when you switch them around to read the processor voltage, they don't settle out correctly on the different version of the 328 chip on the Pro Mini. I was going to research the problem and work around it, but realized that the one-wire 18B20 didn't need analog conversion, it's all digital. Problem solved.

Then, I went to three batteries. All the experiments indicated that I just didn't have enough voltage range to get long battery life. The AA batteries supply power very well all the way down to about 0.9V, and then they drop off pretty rapidly. With two batteries, that means I can't get close to the maximum capabilities of the batteries because my circuitry drops out at just under 2.8V. I can only drag the batteries down to 1.4V each and that means I'm wasting almost half a volt of useful power.

Using three batteries and a 0.9V limit I can drag most of the power out of the AA batteries and do useful things with it.

Now do you see why I need the voltage regulator? Three AA batteries will give me 4.5 volts and a very hot XBee. With the low dropout, low current voltage regulator on the Pro Mini, I should be able to get a long period of life before I have to change the batteries and still power the XBee just fine.

This is what it looks like right now:

I tried to hide the LED so the picture would be clearer. The 18B20 is on top, bent over a bit so it can catch whatever breeze may happen along. I used the same rubber band, but it may have to be replaced soon, starting to dry out and crack. Things are really crowded on purpose, I'm trying to keep all the circuitry on a half sized protoboard. At some point I'm going to solder all this stuff down and actually use it.

So, it's in service right now measuring temperature every minute and sending it to my house controller. I loaded it with dead batteries from the previous attempts. Not only is that saving me money, it should also die more quickly from draining the batteries. Similarly, I left the LED in to consume more power to get to that point. Battery operated tests take a long time to learn what happens at the end if you don't do something like that to hurry things along.

The compromise is that the processor voltage is after the voltage regulator. That means my measured voltage is going to be level until the batteries are nearing the drop out point. I could get around that with some clever rewiring, or a separate regulator for the XBee, but this test may give me all the information I need. I'm just going to look for a level voltage reading until the end where the voltage regulator drop out happens. With the LED running, it shouldn't take too long. If it does take too long, I'll think a bit about a voltage divider from the unregulated voltage to give me a sample measurement. If anyone's interested the code will be in GitHub in about an hour or so.

More later.

The next post in this series is here <link>

Battery Operated Temperature Sensor - Still learning about the TMP36

The previous entry on this project is here <link>

After I found out about the open emitter problem in reading a TMP36 <link> and tried it out for a longer period of time I noticed there were still some glitches in the readings. Don't misunderstand, these were tiny and represented no real problem, but I wanted to settle it down as much as possible.

Yes, obsessive, compulsive about this I am.

I recorded a ton of readings from the chip and looked them over. The problem seems to come from simple outlier readings that happen from time to time.  Every sensor has outlier readings, it's just part of reality, and there are methods of fixing this problem. Naturally, I took the easy way out. I simply read the sensor 15 times in a row into an array, sort them, and average the middle five. This way the outliers wind up at the bottom or top of an array and get excluded from the sort. Here's the code:

#define READSIZE 15

float readTemp2(){
  int readings[READSIZE];
  int reading=0;
  
  for (int i = 0; i < READSIZE; i++){
    readings[i] = analogRead(tmpInput);
  }
  // Now sort the list to put the outliers at the beginning and end
  sort(readings, READSIZE);
  // grab the middle 5 and average them
  for (int i = READSIZE / 2 - 2; i < READSIZE / 2 + 3 ; i++){
    reading +=(readings[i]);
  }
  reading /= 5;
  //reading = analogRead(tmpInput);
  float voltage =  (reading * 1.1) / 1024;
  float tempC = (voltage - 0.5) * 100;
  float tempF = (tempC * 9.0 / 5.0) + 32.0;
  return(tempF);
}

void sort(int a[], int size) {
  for(int i=0; i<(size-1); i++) {
    for(int o=0; o<(size-(i+1)); o++) {
      if(a[o] > a[o+1]) {
        int t = a[o];
        a[o] = a[o+1];
        a[o+1] = t;
      }
    }
  }
}

This smoothed the reading right out and made for a very nice graph. In the fragment of readings charted below you can see how well this works:

This little piece of data is when I had the sensor outside, brought it in the warmer house, then took it back outside. Notice the shape of the curve? Classic ramp up and back down similar to the response shape of an inductor. This is the thermal resistance of the plastic case on the sensor, wires, etc, and is a far cry from the erratic reading I used to get.

The TMP36 turn out to be a really nice little device when used correctly with my new filtering. I'll eventually look at removing the power from the sensor when I put the Arduino to sleep, but I don't expect much of an increase in battery life since the quiescent current of the sensor is claimed at 50uA. Every little bit helps since I plan on running these things forever.

The next entry on this project is here <link>