Monday, February 23, 2015

Battery Operated Temperature Sensor - Smarter TMP36

Last post <link> about my project to create a useful battery operated temperature sensor I whined a while about the flaky behavior of the TMP36 sensor.  I'm not the only person that has had this problem.  A simple search with Google will turn up person after person that has fought this problem. People have tried intricate filtering techniques, averaging, continuous averaging, everything they can think of to settle this device down. In the post I used the average of ten readings to get mine.

Even with that, I had some trouble.  This is on a battery supply, so there's no ripple or chatter from a buck supply, very low noise. I thought it could be the XBee transmission, so I added a little time delay to keep the XBee from transmitting at the same time as the reading. Nothing seemed to help Then, it occurred to me that I'd had a TMP36 setting outside hooked to an XBee using a pretty noisy wall wart for a supply continuously for a couple of years and never noticed a problem with it.

What's up with that? Why did it work on a fence post and not in my house?

I drug out the remains of the fence post temperature sensor from the box I'd tossed it into and thought about it a bit. I had put a voltage divider between the output of the TMP35 and ground to extend the range of the reading. Since I was using a tiny reference voltage, the max temperature I could read would be 120F and I needed more than that to have an outside sensor here in AZ (Don't try telling me to move, winters here are wonderful). Could the device need a load on the output?

I looked up a datasheet on the TMP36 and looked at the schematic on the device; the device has an open emitter output. I had never noticed that before because I was looking at tutorials and such and they all showed the TMP36 feeding directly into the input of an Arduino or other MPU.

I had to try it.  I dragged out a bag of resistors and took a look at the maximums of the TMP36 and settled for a 47K resistor, and grabbed a .01ufd capacitor and added them to the circuitry of the board. Everything worked fine, but I still had the averaging in the code, so I removed that and put the board back in service.

That seems to do it. The little temperature sensor is chugging away giving me nice stable readings. It hasn't been in service for very long, but I usually got a spike in the reading in that amount of time. Just to be clear on what I did, here's a schematic:


The .01ufd cap is as close to the chip as I can get it as is the resistor. I simply plugged them into the breadboard.

Could it be that all the tutorials out there are wrong ? I went looking again and never found a place where someone pointed this out. There were long discussions on the time it takes the input cap on the A/D converter to charge, a myriad of algorithms on smoothing (some of them mine), etc. Then, I stumbled across another person that came up with this <link>, Dr. Monk found the exact same thing. I was a little embarassed until I noticed that he posted on the 16th of this month, after I started this particular project and well after my other attempts, so I was right in line with his discovery. Great minds think alike (slowly).

I'm going to watch this over time, but it brings this little temperature sensor back as a very real possibility. I'm also going to try another sensor the DS18B20 since it comes in a cool waterproof housing as well as the TO-92.  I even want to experiment with a bimetal sensor that is heat resistant for monitoring food.

I can't believe I missed this for YEARS.

Wednesday, February 18, 2015

Battery Operated Temperature Sensor

This project will take a long time to complete, but I wanted to post about it because it's a nice project for a person starting off in Home Automation.  I do a number of things in this project that can lead to even more projects later; this could be the first step for a person that wants to take control of their house without the huge expense of one of the ready made systems out there.

The story begins after I installed my new barometer out on the fence <link>, I was looking at the temperature sensor that I removed <link> and thought, "It would be cool to have temperature sensors that I could put anywhere around the house."  It would be nice to see the temperature in the hot parts of the house and maybe move some air in there when company is around. If I used a sensor from one of those oven temperature probes, I could watch a roast from anywhere; this could be useful ... so I built one.

I wanted to be able to power it with batteries and be able to read the output on a monitoring XBee, so that would mean a processor of some kind and a lot of learning about low power techniques. Low power pretty much leaves out a normal Arduino, they have power regulators on them and those things constantly consume power; they also get warm and that would mess with the temperature readings. I decided to use one of my Ardweenys.  If you're not familiar with these, here's what they look like:
They're not the tiniest Arduino derivative, but they are really close.  The beauty of these devices is that they have a tiny footprint and no unnecessary components to use power. I don't have to come up with a design for supporting the processor and can just use the device.

Since the new device was going to be battery powered, I would need to somehow monitor the battery level so I could change them when needed.  Did you know that the atmega328p can monitor its own power?  Yep, it can, you just have to be good with google to find out how.  There's also a temperature monitor in there so you can keep track of the heat generated.  I don't actually need that, but it would be fun to do it. Here's the code I used to capture the processor temperature and supply voltage:

/*
These two routines use the atmega328 built in thermometer and VCC measurement techniques
This way I can get the supply voltage to see how the batteries are doing, as well as grab
the temperature of the chip.  It may need calibration, but it's free
*/
float readTemp(){
  long result; // Read temperature sensor against 1.1V reference
  ADMUX = _BV(REFS1) | _BV(REFS0) | _BV(MUX3);
  delay(20); // Wait for Vref to settle - 2 was inadequate
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
  result = ADCL;
  result |= ADCH<<8;
  result = (result - 125) * 1075;
  return (result / 10000.0) * 1.8 + 32.0; // I want degrees F
}

float readVcc(){
  long resultVcc; // Read 1.1V reference against AVcc
  ADMUX = _BV(REFS0) | _BV(MUX3) | _BV(MUX2) | _BV(MUX1);
  delay(20); // Wait for Vref to settle
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
  resultVcc = ADCL;
  resultVcc |= ADCH<<8;
  resultVcc = 1126400L / resultVcc; // calculate AVcc in mV
  return (resultVcc / 1000.0); // but return volts
}


The units (Fahrenheit and Volts) are just my choice, Feel free to use degrees C and millivolts if you want to.  These routines ONLY work on certain of the Atmel processsors, but I'm only going to be using the 328, so that's not a problem.

Next I need a temperature sensor. I've already worked with the tmp36 a lot, and I have one of them on hand, so I went with it. I really can't recommend the tmp36 though; the things are skittish and have to be catered to a bit.  I've found with a lot of experimenting that stabilizing the power supply a lot and reading them several times in a row and taking an average will give a good reading.  Since I can read it 10 times and take an average in a few microseconds, that should work just fine. The power supply noise will only be a problem while I debug it because it will run on batteries.

/*
Reading a tmp35 chip to get temperature.
These devices are a little flakey. If the power has even a tiny
bit of noise, they give erratic readings. So, filter them as
best you can and then take several readings and average them.
*/
float readTemp2(){
  int total = 0;
  for (int i = 0; i < 10; i++){
    total += analogRead(0);
  }
  int reading = total / 10;
  float voltage =  (reading * 1.1) / 1024;
  float tempC = (voltage - 0.5) * 100;
  float tempF = (tempC * 9.0 / 5.0) + 32.0;
  return(tempF);
}

I've used this type of code several times for various experiments and it seems to work pretty well. There's other techniques, but they're much more complex. I'll try this for a while and see how it holds up in actual operation over time.

Of course, since it's me, there will be an XBee in there to transmit to my House Controller. I've never tried to get the power down low on an XBee, so this should be interesting. Notice how 'interesting' and 'frustrating' often mean the same thing? This time, there really wasn't any problem. I wanted the processor to control the XBee, so all I needed to do was set the XBee up for a pin controlled sleep and the processor could control it through a digital pin.

One of the cool things about sleep and processors is that the GPIO pins retain their state when you put them to sleep.  So, you set a pin high and put the processor to sleep, the pin stays high. That was a bit of a surprise at first, but it makes sense when you think about it a bit. This makes it really simple to put the XBee to sleep, just:

#define SLEEP HIGH

      digitalWrite(xbeeSleepReq, SLEEP); // put the XBee to sleep

I used a #define for SLEEP because I kept forgetting which state I needed to set the pin to.

The idea is that the processor starts, initializes the XBee and sets up anything else needed, then goes to sleep.  Sometime later it wakes up, takes a temperature reading, sends it, then goes back to sleep. Repeat forever, or at least until the batteries die.

Now, all I have to do is learn how to put the 328 to sleep. Yep, there's thousands of blog posts and tutorials out there on how to do this. Tons of examples of setting registers and shifting bits to control internal counters and such. Yuck! I really don't want to spend a few days learning the ins and outs (pun intended) of the 328P registers, I want to use them. JeeLabs to the rescue. Jean-Claude Wippler of JeeLab has created a nice library that includes a function to handle sleep without having to learn a ton of marginally useful register manipulation. However, I need to wake it up also; it's not very useful to put it to sleep and just leave it there. I can't do my usual technique of asynchronous timers in this instance because the timers won't run when the 328 is asleep, so I have to rely on the milli() function. Here's the code I came up with to control the sleep-awake cycle including putting the 328 to sleep.

  if (millis() - savedmillis > AWAKETIME){
    Serial.print("was awake for ");
    Serial.println(millis() - savedmillis);
    delay(100); // delay to allow the characters to get out
    savedmillis = millis();
    unsigned long timeSlept = 0;
    while (timeSlept < SLEEPTIME){
      digitalWrite(xbeeSleepReq, SLEEP); // put the XBee to sleep
      Sleepy::loseSomeTime((unsigned int)1000);
      timeSlept = (millis() - savedmillis);
      digitalWrite(xbeeSleepReq, AWAKE); // wake that boy up now
    }
    Serial.print("was asleep for ");
    Serial.println(millis() - savedmillis);
    savedmillis = millis();
    sendStatusXbee();
  }

AWAKETIME and SLEEPTIME can be changed to limit battery usage while still leaving enough time for the XBee to stabilize and complete the transaction. I'll have to experiment with this over time to find some balance based on real-life experience.  But notice how easy it was to put the board to sleep for a timed period?  One litle call to loseSomeTime() with the length of time I want to sleep was all there was to it.

A problem crept in this technique though.  Since the board is asleep, it isn't counting milliseconds, so the JeeLab library estimates the duration and bumps the millisecond timer to get it close to being correct. The problem comes when an interrupt happens. When something interrupts, the board wakes up to handle the interrupt, and when using SoftwareSerial to get an additional emulated serial port, you get interrupts. That's why I keep my own count and repeat the loop to sleep some more when this happens. In practice, I lose time this way, but it's been less than a second in the testing I've done so far. The timing doesn't have to be perfect, just good enough and this seems to fit the bill nicely.

So, I do some initialization, go to sleep, wake up, send the status and go back to sleep. I based the code on the XBee example I gave in a previous post for the barometer <link> I have outside. I had to take out some stuff and add others, but the code is sorta, mostly the same. I still have the command handling in there in case I need it.

Now, armed with enough hardware to get this project started I cobbled together a setup on a protoboard and started putting it together.


Yes, that messy lump of wires is my temperature sensor.  It takes up less than half a full sized breadboard, and has a few locations to spare.  There were a number of things that crept into the project that after getting it going I didn't expect.  For example, when the board is asleep too long, the XBee coordinator forgets about it. What happens is that the coordinator expects some interaction with every XBee on the network periodically, and if it doesn't get it, it removes the entry from its table of devices. So, on the first few tries, the temperature sensor couldn't connect when it woke up and I got error 22 or 24 depending on what was going on at the time.  A significant amount of time later I had a pretty good solution; just change the timeout on the coordinator for this item to be significantly larger than the sleep time on the temperature sensor.  I decided to make it three minutes and give it a try since I can change it later. The complication to this is that you have to change it for the coordinator and all the routers on the network.  Fine, I just fired up XCTU and used the remote command capability to update the various XBees one at a time.  The values I used in this case were:

SP = 7D0
SN = 9

The SP parameter evaluates to 20 seconds and the SN parameter means do it 9 times. Three minutes on a device that reports once a minute should be fine.  If extended testing indicated that I needed to change it, I'd just have to do the change over again. Not too bad.

Another thing was that I tried sending too soon after waking up the XBee.  The XBee takes a small amout of time to get itself going well enough to respond to send requests or commands, so I had to enable the CTS (clear to send) capability and hook it up to a 328 pin.  After doing that all I had to do was hang in a loop until the CTS signal changed and then I could send to my hearts content.  To set the parameter use the value:

D7 = 1

And my simple solution to monitoring it before sending was:

void sendXbee(const char* command){
  ZBTxRequest zbtx = ZBTxRequest(Destination, (uint8_t *)command, strlen(command));
  while (digitalRead(xbeeCTS) == SLEEP){} // just hang up and wait 
  xbee.send(zbtx);
}

I just hang in a while loop until the pin goes low.  It was nice how the #define I came up with worked in this case.

I decided to play around with it hooked to a battery and rewired the mess until it looked a little better and this is the device as it exists right now:


Maybe it isn't any less messy after all.  I did get to use my rubber band though.  And yes, I use cheap batteries. No, I don't expect you to be able to trace the wires from this.  Here's a Fritzing image of it:


Note that the black thing is NOT a 328 chip, it's an Ardweeny, I couldn't find a Fritzing part for the Ardweeny that worked.  And, for my 'more advanced' readers:


Once again, it isn't a 328, it's an Ardweeny, and of course I'll give you the code, it's in GitHub already with my other Arduino projects and examples <link> it's called RoomTemp; I also added Jeelib so I wouldn't have to keep track of changes to it. But remember, I'm not done with this yet.

I'll have to monitor the battery usage over time and make adjustments or changes as necessary, bugs may show up, or I may get a brilliant idea. The eventual goal is to have them in several of my rooms as well as a couple to play with.  I may even carry out my threat to hook one to the barbecue for monitoring steaks. I'm monitoring this on my Raspberry Pi house controller, but not doing anything with the readings yet.  Here's a sample of the output I'm using:

{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.9', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.9', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.7', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.2', u'temperature': u'73.7', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.7', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.9', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.7', u'voltage': u'3.0'}}

I'm using a JSON string and the 'ptemperature' value is the processor temperature I mentioned above, the voltage is taken from the processor also. Of course you can change the code and use any format you want to in the message.

Have fun.



Saturday, February 14, 2015

Acurite Console Barometer: OK, This is it.


Back in a previous post I tried to decode the barometer reading from the AcuRite console.  I just couldn't get it, but one of the readers, Play Kube, stepped up and nailed it for us. The post is here <link> and his findings are down in the comments.  He emailed them to me and I posted them for him. Then I added code to my stuff and recorded the readings for a few days and this is the correlation to my (very accurate) fence post barometer. I spent quite a bit of time making sure my barometer compared to the weather stations around me and it was dead on.  The decoded AcuRite readings follow mine really, really well.  Take a look:


Isn't that great?  There's a couple of caveats though: The console barometer is NOT temperature or altitude corrected.  That's because we don't have access to the temperature or altitude corrections that the chip manufacturer supplies inside the chip. These values can be read and the math applied to get the reading dead on, but we don't have them to work with.

The way I corrected the readings was to simply adjust the algorithm to compensate.  This means that if I move the console up or down by a significant amount (I move to sea level) or the temperature changes (put it in the sun) the readings will change.  However, in most of our houses the temperature stays pretty stable and it's unlikely we'll be moving soon.  If one of these does happen, just readjust the algorithm and you'll be all set for the next time.

Looking at the chart, the difference in the readings is less than a mbar and tracks exactly.  Here's the code I used to decode the reading:

                    float bar = 6.23 * (R2[23] << 8 | R2[24]) - 20402;
                    weatherData.barometer = bar / 100; // convert to mbar from pascals
                    weatherData.barometer += 81.1; //adjust for altitude

It's a simple linear equation like a line graph from high school algebra.  Take the last two bytes and form a single number from them, then use a slope of 6.23 and an offset of -20402.  This will get you the barometric pressure reading if you were at the same altitude and temperature as the person that figured it out. So, to adjust it to your location, just go look up the barometric pressure reading for your location at the same time and add the difference in.  That's what the 81.1 number above means, I just added it in as the last step. The changes will be in GitHub soon.

Don't think this is too simple a solution.  The manufacturer of the chip put a lot of work into getting the sensor to follow pressure changes, then they put the adjustments for temperature and fixed altitude in the chip so they can be read and applied.  There was a lot of work done to get it to work. The trick was to get the slope correct, then the offset was just the method of getting it to line up vertically.  The altitude offset is simply necessary to adjust for the specific location.

Slick, but remember, change altitude or temperature and you may have to revisit the algorithm and change the values. We owe this discovery to Play Kube for sending me the mail a few days ago.

I still haven't found the battery level from the weather head though.

Wednesday, February 4, 2015

Reading the AcuRite 5n1 Sensor Set, This time with RF

I've managed to read the RF signal coming from the AcuRite weather head.  It wasn't too hard to do since I had some significant help from other folk that have done the same thing.  The problem was only that there wasn't anyplace out there that I could just look at for an answer that was somewhat coherent to someone that wasn't conversant in the latest digital tuner techniques.  It turns out the answer is quite simple to do, once you work your way through the confusion.  I hope I can clear it up for folk because this is the last piece of the AcuRite puzzle that I think is worth pursuing. 

First, get one of these:


This the tuner chip that is used in digital TV that has been hooked to a USB interface and enclosed in a plastic case.  The chip is a RTL2832 and has a huge bandwidth; it can be used to receive AM, FM, SSB, etc all the way up past the 900MHz range that the AcuRite transmits on.  It's a fun device in and of itself and is the subject of blog posts all over the web. You can listen in to commercial flights, the cops parked at the corner watching your garage, even the doorbell transmitter in your neighbors house.  I didn't want it for boring things like this, I wanted to get the rainfall count out of my 5n1 sensor set without using the console.

So, I downloaded a great tool for doing this kind of thing called SDRsharp and installed it on my laptop.  After visiting several web sites that described how to use its features, and listening to the five or six FM stations I can receive out here, I went looking for the AcuRite sensor.  I found it.


That's it, the spike that is showing on the capture.  It took me a little while prowling around the spectrum to find it, but there it is, Oh ... wait ...


Notice in the bottom panel that there are three different signals that are on almost exactly the same frequency?  Yep, I'm picking up three devices up in the 433MHz area and any one of them could be what I'm looking for.  After a bunch of looking, it turns out the first one I found is the one I want, but how to I get the data out of it? Heck, how do I separate the one I want from the others?

This has already been solved for us under Linux by a tool called rtl_433.  But before we talk about that, use SDRsharp to zero in on the frequency you need to use.  There's a little drift between devices both at the radio and at the transmitter, getting the exact frequency reading will help you later.  Write it down somewhere.

OK, I'm getting too technical without explaining what is going on.  These little tuner chips were designed for digital TV and the description (stolen from here <link> is:
What is RTL-SDR?
RTL-SDR is a very cheap software defined radio that uses a DVB-T TV tuner dongle based on the RTL2832U chipset. With the combined efforts of Antti Palosaari, Eric Fry and Osmocom it was found that the signal I/Q data could be accessed directly, which allowed the DVB-T TV tuner to be converted into a wideband software defined radio via a new software driver.
Essentially, this means that a cheap $20 TV tuner USB dongle with the RTL2832U chip can be used as a computer based radio scanner. This sort of scanner capability would have cost hundreds or even thousands just a few years ago. The RTL-SDR is also often referred to as RTL2832U, DVB-T SDR, or the “$20 Software Defined Radio”.
There's an entire web site dedicated to these little devices at http://www.rtl-sdr.com/ and another at http://www.rtlsdr.com that describes many of the uses people have come up with, so I won't go into it much more deeply.

Next, there's a Linux library that supports this and a really cool tool you can run on your PC to play around. The tool is called SDRsharp and has a dedicated website <link>; just go there and select the download link to get it.  The screenshots above are taken from this tool

Now, before I get into describing how to use this on a Raspberry Pi, you should be aware that these little radio receivers pull a lot of power.  When you plug them into the Pi, it will suck so much power that the Pi will reset and have to boot back up.  If your power supply is not up to it, the radio won't work properly.  A powered hub will help isolate the Pi from the high current, but isn't absolutely necessary if you power supply is strong enough and you don't mind it rebooting when you plug the receiver in.

The Linux library is called  rtl_sdr and is at http://sdr.osmocom.org/trac/, but don't download it yet. Don't misunderstand, this is a nice tool, but not exactly what you want to read the AcuRite weather head. What you want is a tool called rtl_433 that was developed specifically to decode the various devices that operate in the same unlicensed frequency range.  Things like temperature sensors, doorbells, full blown weather stations, etc.

There are a lot of blog posts out there on how to install the rtl_433 tool, but none of them worked for me.  I had to actually follow the instructions as written in the readme file that came with it because the authors had recently added the general purpose library for SDR (software defined radio) when they found out it needed to be changed to help their tool work better. When I downloaded the zip file, I got it all.  It took me about three days to figure that out and get it to compile and work.  Then, when I went back later in preparation for this post, they had changed rtl_433 a bunch. I guess someone got a burst of energy and they updated everything in sight. They had gone back to installing rtl_sdr as a separate step and that would make things much harder.

I just created a new one under my login on GitHub.  I really didn't like the idea of things changing every time I wanted to do something and using it for my weather station would depend on two projects out there that can change at any time.  I tried to use the Git fork and other tools and got totally confused with the various methods of backing up in a respository; I guess my very own version isn't a bad solution.

There is a dependency on libusb though.  If you've already installed my USB code that previous posts described, you've got that requirement already.  If you haven't, you'll need to install libusb. Go to my post on this <link> and follow the part about installing libusb and libudev-dev.  This is a USB device (of course) and is very, very similar to the code I created to read the console device.

Now you're ready to work on rtl_433, so just go to github and grab it <link> by clicking on the 'Download ZIP' button on the lower right.  To build it though, they use a tool called 'cmake' that isn't on the Pi by default.  Fortunately, it can be installed easily

sudo apt-get install cmake

Now you can (finally) just follow the instructions in the readme that comes with the library to build it.  Don't do like I did and think I knew everything and install the rtl_sdr library and try to get it to link up properly with rtl_433; that is a path to almost terminal frustration.  Just build rtl_433 from my copy. You'll probably want to rename it once you get it unzipped to something easier to type.  Here's the commands I used to build mine:
cd Desert-Home-rtl_433-master
mkdir build
cd build
cmake ../
make
Now, you are almost ready to actually read the RF from the weatherstation, but (another one of those) the decoder included with rtl_433 is wrong. I don't know if the author of that particular piece had the same kind of problems I did originally, but he made some of the same mistakes I did.  So, I put together my own version that decodes my weatherhead and corresponds with the console. These changes are part of the zip file you downloaded; you don't have to do anything special.

The great thing about doing it this way is the weatherhead provides a checksum on the data.  That means no more undetected short packets or weird data because something in the AcuRite console is messed up. It also means you only get the 5 sensors included in the weatherhead: temperature, humidity, wind speed, wind direction, and rainfall count.  But, since I installed my own barometer <link>, and plan on building anything else I may need, that's good enough for me.

Just for completion's sake, here is sample output from my weatherhead:

pi@deserthome2:~/src/Desert-Home-rtl_433-master/build$ cd src
pi@deserthome2:~/src/Desert-Home-rtl_433-master/build/src$ rtl_433
Registering protocol[01] Rubicson Temperature Sensor
Registering protocol[02] Prologue Temperature Sensor
Registering protocol[03] Silvercrest Remote Control
Registering protocol[04] ELV EM 1000
Registering protocol[05] ELV WS 2000
Registering protocol[06] Waveman Switch Transmitter
Registering protocol[07] Steffen Switch Transmitter
Registering protocol[08] Acurite 5n1 Weather Station
Found 1 device(s):
  0:  Generic, RTL2832U, SN: 77771111153705700

Using device 0: Generic RTL2832U (e.g. hama nano)
Found Rafael Micro R820T tuner
Exact sample rate is: 250000.000414 Hz
Sample rate set to 250000.
Sample rate decimation set to 0. 250000->250000
Bit detection level set to 10000.
Tuner gain set to Auto.
Reading samples in async mode...
Tuned to 433920000 Hz.
Detected Acurite 5n1 sensor
wind speed: 2.0 mph, temp: 54.5F, humidity: 67%
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, wind direction: NE, rain counter: 344,
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, temp: 54.5F, humidity: 67%
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, wind direction: NE, rain counter: 344,

Notice how the build process put the files in the directory rtl_433/build/src/ ?  I'm not responsible for that ... that was the way I got it.  Just move them wherever you want them to reside at run time.

Remember way above I told you to write down the frequency you found the weather head transmitting on?  This is where you may have to use it.  If it drifted far enough, you won't get a good signal from the transmitter up on the roof or out in the yard.  You can simply specify the frequency you want to listen to with the '-f' parameter like this:

rtl_433 -f 433.915e6

This means to set the receiver at 433.915 MHz.  I actually use this number for my weather head.

There's still some stuff to be done from my perspective, I want to put some code in to present the data as a JSON string so it can be used exactly like I did in the other version.  I also want to wait until I have both types of messages in hand before I output anything.  I'll be doing that over time and moving to exclusively use the RF from the weather head.  That will eliminate all the little irritations that the console entails. So, if you grab the code at some point down the line, there will be changes that reflect my specific use; if you want this particular version, the commit # in Github is 5c72e32 and you can get to it by clicking on 'Commits', then finding the number in the list and clicking on the browse next to it that looks like this <> . That will get you back through any changes to the level it was when I wrote this.

This will also make the Raspberry Pi a possible replacement for their expensive bridge device that is limited in what it can do and completely closed to us changing it.

But what will I do with the console?  It's actually an attractive device and I think I'll put it in the kitchen somewhere.  It will give me a nice conversation piece in the most often used room.

Have fun.

Edit Feb 13, 2015:  One of the readers, John, had a problem with attaching to the SDR device.  He chased it down to some code needing to be added in the USB open code.  He emailed it to me and I tried out his fix and it worked fine.  The problem relates back to removing the default hardware driver when starting up like I mentioned way back in a previous post <link>.  I tested it to be sure it would work for me as well and it was great.  I've already put the change into the github repository as commit number 86c70cf .  See, it does pay off to read all the posts after all. Feel free to grab it

Tuesday, February 3, 2015

Suddenly, I'm Famous

Heh, this blog is the "Engineering Site of the Day" at EEWeb.com.  I only get this privilege for a day, so check it out quickly:

http://www.eeweb.com/websites/desert-home

I've used the site several times looking for cool devices and ideas.  You know, when you just want to step through the various advertisements and announcements to see what the new things are.  They also have a forum where you can ask questions like "How do I ...," and "what device will ..."  The nice thing is there isn't the collection of snarky replies that usually come from a question like that.

No, I have no idea why they chose me other than they recognize tenacity.  I can't be the beautiful prose and superior artistry of my blog.

So, go to their site and prowl around. Then, for folk like us, go to the embedded tab and really look around, There are some articles in the embedded section that really caught my eye.  Especially one on sequencing power supplies <link>.  I'm looking at that problem right now where I have a 5V device hooked to a 3V device and I want to put them to sleep.  How the heck does one do that?

Now, a screenshot in case they decide they made a horrible mistake and pull it.  I want proof.


I'm so proud.

Saturday, January 31, 2015

Let's Finish Off the Barometer Project.

In the last entry <link> I posted the code for my barometer; I took a simple program of a few lines and turned it into a monster that does everything imaginable.  Now, I should actually set it up to work and put it in service.  Yes, my XBee shield came in and worked fine first try, so I mounted the little breakout board on it and added four wires to connect the barometer chip to the Arduino pins and tested it.


Made a nice little package when I got it all put together, so I dismantled the Stephenson Screen on my fence and tie wrapped it inside.


I had the old temperature sensor in there for years with no problems, so I don't think I need to get any fancier than that.

One thing I did in the code that I didn't mention in the last post was how to correct the reading to adjust for sea level comparisons.  See, to make barometer reading useful around the world, weather folk adjust them for the altitude so that there is a common base for comparison.  There's some really good explanations for why and how to do this out there, so if you don't already understand this, do a quick search.

From our perspective, this is incredibly easy; just call a different routine to read the data:

float pressure = bmp.readSealevelPressure(altitude)/100.0;

This statement will get the pressure adjusted for 'altitude' and the division by 100 will convert it from Pascals to millibars.  If you want inches of mercury, just look up the conversion factor.  The Adafruit library has already got the code for the conversion, all you have to do is look up your altitude in Google Maps, or read it off a GPS at the location where you put the barometer.

I've already adjusted my house monitor code to use this device and if you look at my house display, the outside temp and barometric pressure are taken from the new set up. I was lucky enough to catch a small storm so I could compare it with the other stations in my area, and it was right on.  It's really great when a sensor does exactly what the datasheet says it will.

I really like this sensor; the temperature measurement is very stable as is the barometer.  It's a snap to read and only takes two signal wires for all of it.  Since there's room on the device for more sensors, I may think about luminance or something in the future.

All this playing with the AcuRite weatherstation has been fun, and now I'm going to look at conquering the RF signal it sends.  What I'm going for now is far, far away from what I started off to do, but isn't that how it goes?  I want to grab the RF directly and use that data along with any other sensors I need to add and bring up a separate Raspberry Pi to handle weather presentation. Remember, the code is already in GitHub for you to grab <link>.

Funny how these projects tend to take on a life of their own.

Wednesday, January 28, 2015

My Barometer, and Tips and Tricks With XBee Device Code.

In the last post <link> I put together a basic barometer to substitute for the silly one on the AcuRite console that came with my 5n1 weatherhead.  My XBee shield still hasn't come in, but I got a wild hair and decided to go ahead and put together the code while I waited.  As I was stealing pieces from various other projects I've done and combining it to implement this latest one, I realized there was a significant piece of code I haven't shared yet.

Early on I got tired of having my XBee set up wrong because I forgot something when I configured it and created a little piece of code to check the configuration to some degree at boot time on the Arduino.  By sending AT commands to the XBee, you can check things like software version, operating mode, etc.  Then if something was messed up, you can correct it easily without a long debugging session to discover some minor setting error.  I also use this to get the name of the device, and the address that it normally sends to.  Using this piece of code makes the entire device more configurable.  For example, you need to change to broadcast, instead of a specific address, to monitor the packets, just use XCTU to remotely change the DL, DH attributes and reset the Arduino so it reads the new address.  If you decide to change the device name, remotely change the NI attribute, and again, reboot the arduino.  I've been using this in one form or another now for a while to help work the kinks out of the latest device I'm working with.

Yes, you can change the configuration of the XBee that you stuck in the corner of the attic remotely,  This has been a capability of XCTU for a long time, and the latest version of XCTU makes it even easier.  To remotely reset an Arduino, I simply decode a command I send and then call location zero.

These two tiny tricks makes minor changes to the network really easy and can be done from my recliner.

The code that follows (below somewhere) is a simple thing with complete XBee interaction,  I have examples of sending formatted strings containing float data through the XBee, parsing a long unsigned integer out of an XBee message and using it, simulating the data portion of an XBee message for debugging, setting up a timer to send status periodically, reading some of the configuration out of the XBee to be sure it's OK before continuing the work.  That kind of thing.

Since the barometer code is so easy to understand, I thought that would be a perfect device to put all this stuff in and use as an illustration.  Go take a look at the barometer <link> then come back here and I'll step through some of the things that you can use in your own code to make the XBee a valuable device in do it yourself home automation.

First, just to get it out of the way, resetting the Arduino from software.  If you haven't hit it yet, you will, that time when you know a reset will cure a malfunction on the Arduino, but its up high, out in the yard, in the dark on the side of the house; somewhere hard or inconvenient to get to.  Wouldn't it be nice to just send a command to it and have it reset itself?  A software reset is easy to do.  If you need a hard reset (similar to pulling the plug) this won't do it; but those are not often needed.  Just put this line somewhere at the top of the code:

void(* resetFunc) (void) = 0; //declare reset function @ address 0

and when you need it, call it like this:

resetFunc();  //This will reset the board and start all over.

Yep, that's all there is to it.  I use this a lot for overcoming memory leaks in libraries that haven't been tested long enough, memory loss from using Streams in c++, that kind of problem.  It also helps overcome my own bugs from time to time.  As an example, in the code below I use it twice.  I have one software reset that is on a timer to reset the board every day around midnight.  This is to recover memory lost to whatever code does a malloc() and then doesn't remember to do a free() to release the memory back.  This is a very common problem (called memory leak) and has plagued tiny computers as long as they've existed.  When you want to run an Arduino 24/7, you have to code defensively.  The other instance is when I send a reset command over the XBee network to cause the board to reset.  I use this to test things like power failures, how long it will take to settle down, and to see if a reset will work before I get the ladder out to take a physical look at it.

Since I mentioned remotely resetting the board, I'll describe how I do that, but I need to explain how I use the IDE itself first.  Most of you already know that the Arduino IDE supports 'tabs', but the new folk haven't caught up to that yet.  Tabs are actually separate source files that the IDE stores in the same directory as the sketch.  These are extremely nice for organizing code so you don't have to scroll through a couple of hundred lines of code to get to something you want to work on.


What you're looking at above is the Arduino IDE with three tabs.  One is for the bulk of the barometer code, the next is for some utility routines I use a lot, and the one selected is the XBee code I use on a lot of my devices.  I opened the window that controls tabs so the new folk can get a look at the tool provided for working with tabs.  Remember, each tab is actually a separate file in the sketch directory.  When you compile something, the files are all concatenated together and passed to the compiler, this is just a convenience to help you organize your project.  There's a couple of tutorials out there that mention tabs, and I've heard of a book or two that do, but most of the instructions are great at telling you how to install it, but not how to use it effectively.  So, don't be afraid to poke around and experiment.

No, I'm not going to write a book about it.  If I did, they'd change it and I'd have to have a second revision.  I've got projects to work on instead.

Anyway, The picture shows how I decode the command and reset the board.  Over time I've worked out my own particular method of sending and receiving packets that works well for me.  Everything is done in a string; I don't try to send integers, floats or compact the data.  For home automation, interactions are measured in a minimum of seconds and plain text traveling between the machines is perfectly adequate.  It also makes monitoring what's going on much easier since I don't have to decode anything to read it, it's already text.  So my command to reset the barometer is:

Barometer,Reset

I can type that right into an XCTU session from my recliner and the barometer out on the fencepost (where it will eventually reside) will reset itself.  Yes, I have to construct a packet in API mode 2 format, but XCTU has a packet generator that works pretty well to help with that.  

Now, let's talk about AT commands to the XBee itself.  When you look at the various tutorials on the web, they love to type AT into the XBee console and look for the OK that comes back.  Then they teach you how to set up the XBee using these commands.  This is good, but not very useful if you're going to actually do a project that will run all the time and have to survive a lot of the normal problems such as RF noise, distance, flaky devices, etc.  For that you have to go to API mode and send fully constructed packets between machines.  I have already covered the various pitfalls and have instructions on how to set this up on my XBee page, so I won't go over it again here, go here <link> for that when you need it.  What I want to show you is how to send an AT command, get something from the XBee and then use it.  Doing this you can double check the configuration, get the device name, etc.  It's a really good feature to understand.  As an example, here are the first few lines executed on my barometer:


I know, it's a dumb screenshot.  I did this on purpose, to illustrate again how tabs can work for you.  I have selected the barometer tab and scrolled down a bit to the setup() routine.  See how you can flip between tabs to do things instead of scrolling though tons of lines of code and losing your train of thought?  I like this feature.

Anyway, I set up the XBee code, then call getDeviceParameters().  This routine sends a series of AT commands to gather various items and then returns.  Then I display the parameters so I can tell if I have the XBee set up the way I want it.  I actually store the string 'Barometer' in the NI command; this is the name of the device and I use it to check incoming packets to see if they're for this device and include it in the out going status packets I send.  This way I can change the name of the device by simply changing the parameter on the XBee.  Here's the code that does this:

void getDeviceParameters(){
  uint32_t addrl = 0;
  uint32_t addrh = 0;
  uint32_t daddrl = 0;
  uint32_t daddrh = 0;
  
  if(sendAtCommand((uint8_t *)"VR")){
    if (atResponse.getValueLength() != 2)
      Serial.println(F("Wrong length in VR response"));
    deviceFirmware = atResponse.getValue()[0] << 8;
    deviceFirmware += atResponse.getValue()[1];
  }
  if(sendAtCommand((uint8_t *)"AP")){
    if (atResponse.getValue()[0] != 0x02)
      Serial.println(F("Double Check the XBee AP setting, should be 2"));
  }
  if(sendAtCommand((uint8_t *)"AO")){
    if (atResponse.getValue()[0] != 0)
      Serial.println(F("Double Check the XBee A0 setting, should be 0"));
  }
  
  if(sendAtCommand((uint8_t *)"NI")){
    memset(deviceName, 0, sizeof(deviceName));
    for (int i = 0; i < atResponse.getValueLength(); i++) {
      deviceName[i] = (atResponse.getValue()[i]);
    }
    if (atResponse.getValueLength() == 0){
      Serial.println(F("Couldn't find a device name"));
    }
  }
  if(sendAtCommand((uint8_t *)"SH")){
    for (int i = 0; i < atResponse.getValueLength(); i++) {
      addrh = (addrh << 8) + atResponse.getValue()[i];
    }
  }
  if(sendAtCommand((uint8_t *)"SL")){
    for (int i = 0; i < atResponse.getValueLength(); i++) {
      addrl = (addrl << 8) + atResponse.getValue()[i];
    }
  }
  ThisDevice=XBeeAddress64(addrh,addrl);
  if(sendAtCommand((uint8_t *)"DH")){
    for (int i = 0; i < atResponse.getValueLength(); i++) {
      daddrh = (daddrh << 8) + atResponse.getValue()[i];
    }
  }
  if(sendAtCommand((uint8_t *)"DL")){
    for (int i = 0; i < atResponse.getValueLength(); i++) {
      daddrl = (daddrl << 8) + atResponse.getValue()[i];
    }
  }
  Controller=XBeeAddress64(daddrh,daddrl);
}

uint8_t frameID = 12;

boolean sendAtCommand(uint8_t *command) {
  while(1){
    frameID++;
    atRequest.setFrameId(frameID);
    atRequest.setCommand(command);
    // send the command
    xbee.send(atRequest);
    //Serial.println("sent command");
    // now wait a half second for the status response
    if (xbee.readPacket(500)) {
      // got a response!
  
      // should be an AT command response
      if (xbee.getResponse().getApiId() == AT_COMMAND_RESPONSE) {
        xbee.getResponse().getAtCommandResponse(atResponse);
  
        if (atResponse.isOk()) {
          //Serial.println("response OK");
          if (atResponse.getFrameId() == frameID){
            //Serial.print("Frame ID matched: ");
            //Serial.println(atResponse.getFrameId());
            return(true);
          }
          else {
            Serial.println("Frame Id did not match");
          }
        }
        else {
          Serial.print(F("Command returned error code: "));
          Serial.println(atResponse.getStatus(), HEX);
        }
      }
      else {
        Serial.print(F("Expected AT response but got "));
        Serial.println(xbee.getResponse().getApiId(), HEX);
      }
    } 
    else {
      // at command failed
      if (xbee.getResponse().isError()) {
        Serial.print(F("Error reading packet.  Error code: "));  
        Serial.println(xbee.getResponse().getErrorCode());
      } 
      else {
        Serial.println(F("No response from radio"));
      }
    }
  }
}

It's not perfect code, and it doesn't serve the 'general case', but it is easy to understand and does the job.  Notice in the routine sendAtCommand() that I hang up in an infinite loop waiting for a response to each AT comand I send?  Yes, that's on purpose.

On a network like mine where there are a number of XBees sending data at essentially random intervals, traffic can be mixed in with the responses to the AT commands.  I send a request for the software version at the same time as a time update is received from my house clock.  The XBee will send the packet from the clock first and I could miss the response to the AT command.  Also, the receive for the packets is interrupt driven and characters can be missed by simultaneous sends and receives.  To overcome these kinds of problems, I wait for the response about a half second, if it doesn't come back, I try it again.  This guarantees that I will accumulate the items I need during initialization regardless of traffic conditions, it just may take a couple of tries to do it.  It's actually kind of fun to watch it during high traffic conditions; it tries each command as many times as necessary to gather the data and then returns to let setup() print them out.  Here's what the output looks like when I run it:

Barometer Version 1 Init...
This is device Barometer
Device Firmware version 2370
This Device Address is 0x0013A200 0x4089D915
Default Destination Device Address is 0x0013A200 0x406F7F8C
Setup Complete
Mem = 276
Barometer: temperature: 77.5, uncorrected pressure: 937.8
See how the name of the device, long address, version, destination long address can be gathered directly from the XBee?  This is nice to guarantee that you set things up correctly when you try your code out.

Now, let's look at sending the actual data that I gathered from the barometer chip.  I've learned to like JSON encoding as a method of passing data and am converting all my devices to send data using this format.  I haven't gotten very far on that project, but the barometer is new, so I use it here.  I use this line to construct the status text that is sent:

void sendStatusXbee(){
  float temperature = bmp.readTemperature()* 9/5 +32;
  float pressure = bmp.readSealevelPressure(altitude)/100.0;
  sprintf(Dbuf, "{\"%s\":{\"temperature\":\"%s\",\"pressure\":\"%s\",\"utime\":\"%ld\"}}\n", 
            deviceName, // originally read from the XBee
            dtostrf(temperature, 1, 1, t), // This is a text conversion of a float
            dtostrf(pressure, 1, 1, p),  // same as above
            now());  // current time in seconds
  Serial.print(Dbuf); // notice this is only for the serial port
  sendXbee(Dbuf);     // out to the XBee
  Serial.println("Pressure message sent");
}

I use sprintf() because the documentation is all over the web and there are tons of examples out there for people to follow.  The problem is that the Arduino sprintf() doesn't support floating point, but does have a handy routine dtostrf() that will turn a float variable into a nice little string that can be used.  Pay particular attention to the format string I used; you don't have to do it this way.  You can assemble any string you want to and send it instead.  Like I said though, JSON is already documented on the web and I don't have to invent anything this way.

Now, I want to show you how I control where the message is sent.  I read the default address from the XBee DH and DL parameters during initialization and put it into the variable 'Destination', now I get to use the variable:

void sendXbee(const char* command){
  ZBTxRequest zbtx = ZBTxRequest(Destination, (uint8_t *)command, strlen(command));
  xbee.send(zbtx);
}

This will take the string I just constructed and put it into an XBee packet, set the address I mentioned and send it.  To change the address such that the message is broadcast to all devices, I decode a command and just change the value of Destination:

    // The ability to broadcast is sometimes used to see what it going on
    // using an XBee plugged into a laptop to monitor the data.
    else if (strcmp(nxtToken, "SendBroadcast") == 0){
      Serial.println(F("Switching Address to Broadcast"));
      Destination = Broadcast;     // set XBee destination address to Broadcast for now
    }
    else if (strcmp(nxtToken, "SendController") == 0){
      Serial.println(F("Switching Address to Controller"));
      Destination = Controller;     // set XBee destination address to Broadcast for now
    }

This is just like the Reset command above; I send

Barometer,SendBroadcast

and it switches the Destination variable to the broadcast address and from then on every XBee in the network will see the message.  To put it back to normal:

Barometer,SendController

And, this brings up another feature I like to have on my remote devices, a nice way to test the commands without having to drag out an XBee breakout board every time I want to add or try something.  Wouldn't it be nice to be able to simulate the receipt of a command through the IDE serial display?  I worked up this code to allow that very thing:

  // This allows emulation of an incoming XBee command from the serial console
  // It's not a command interpreter, it's limited to the same command string as
  // the XBee.  Can be used for testing
  if(Serial.available()){  // 
    char c = Serial.read();
    if(requestIdx < sizeof(requestBuffer)){
      requestBuffer[requestIdx] = c;
      requestIdx++;
    }
    else{  //the buffer filled up with garbage
      memset(requestBuffer,0,sizeof(requestBuffer));  //start off empty
      requestIdx = 0; // and at the beginning
      c = 0; // clear away the just-read character
    }
    if (c == '\r'){
      requestBuffer[requestIdx] = '\0';
//      Serial.println(requestBuffer);
      // now do something with the request string
      processXbee(requestBuffer,strlen(requestBuffer));
      //start collecting the next request string
      memset(requestBuffer,0,sizeof(requestBuffer));  //start off empty
      requestIdx = 0; // and at the beginning
    }
  }

This fragment will allow me to type a command that would normally come in over the XBee right into the IDE serial monitor and send it to the code.  It does this by just gathering the characters and calling the same routine I use to decode XBee commands.  This cuts short the time it takes to add another command to the device.

Now, normally, I would post the code of the entire module, but I bet you've had enough code samples in windows for one session.  Instead, I finally got the Arduino code I'm using for the various things into GitHub.  This entire module is in the 'Barometer' directory so you can grab it and modify away.  It's been running for a few days and hasn't blown up yet, but I'll certainly be adding to it over time.

If you go looking in the other directories don't expect all of these features to be in each thing.  Like I said, I developed these tricks over time and only recently started to go back and update the older devices.  Heck, I haven't changed the thermostats at all in months.  One of the modules hasn't even been compiled under an IDE version in a couple of years.  I'll work through the devices one at a time as I have the chance, and probably come up with another trick that I'll want to fit into the others.  It never ends.

Here's the link to GitHub  <link>. Have fun.