Saturday, March 14, 2015

Battery Operated Temperature Sensor - Still learning about the TMP36

After I found out about the open emitter problem in reading a TMP36 <link> and tried it out for a longer period of time I noticed there were still some glitches in the readings. Don't misunderstand, these were tiny and represented no real problem, but I wanted to settle it down as much as possible.

Yes, obsessive, compulsive about this I am.

I recorded a ton of readings from the chip and looked them over. The problem seems to come from simple outlier readings that happen from time to time.  Every sensor has outlier readings, it's just part of reality, and there are methods of fixing this problem. Naturally, I took the easy way out. I simply read the sensor 15 times in a row into an array, sort them, and average the middle five. This way the outliers wind up at the bottom or top of an array and get excluded from the sort. Here's the code:

#define READSIZE 15

float readTemp2(){
  int readings[READSIZE];
  int reading=0;
  
  for (int i = 0; i < READSIZE; i++){
    readings[i] = analogRead(tmpInput);
  }
  // Now sort the list to put the outliers at the beginning and end
  sort(readings, READSIZE);
  // grab the middle 5 and average them
  for (int i = READSIZE / 2 - 2; i < READSIZE / 2 + 3 ; i++){
    reading +=(readings[i]);
  }
  reading /= 5;
  //reading = analogRead(tmpInput);
  float voltage =  (reading * 1.1) / 1024;
  float tempC = (voltage - 0.5) * 100;
  float tempF = (tempC * 9.0 / 5.0) + 32.0;
  return(tempF);
}

void sort(int a[], int size) {
  for(int i=0; i<(size-1); i++) {
    for(int o=0; o<(size-(i+1)); o++) {
      if(a[o] > a[o+1]) {
        int t = a[o];
        a[o] = a[o+1];
        a[o+1] = t;
      }
    }
  }
}

This smoothed the reading right out and made for a very nice graph. In the fragment of readings charted below you can see how well this works:


This little piece of data is when I had the sensor outside, brought it in the warmer house, then took it back outside. Notice the shape of the curve? Classic ramp up and back down similar to the response shape of an inductor. This is the thermal resistance of the plastic case on the sensor, wires, etc, and is a far cry from the erratic reading I used to get.

The TMP36 turn out to be a really nice little device when used correctly with my new filtering. I'll eventually look at removing the power from the sensor when I put the Arduino to sleep, but I don't expect much of an increase in battery life since the quiescent current of the sensor is claimed at 50uA. Every little bit helps since I plan on running these things forever.

Friday, March 13, 2015

Raspberry Pi - Read Only USB Stick - Pain in the bottom

Remember back last year when I added a USB stick to my Pi to increase reliability <link>? Serious mistake, it died. Not only did it die, I reacted wrongly to every single thing and messed myself up and had to rebuild the Pi from almost the ground up.

The story goes like this, but I don't have nice pretty examples of the output of the various efforts to fix and recover from this. I wasn't thinking about blogging it, I was trying to get my Pi house controller back up.

I tried to open the garage doors with a remote command it didn't work. I walked out to the garage, pushed the button and the door opened. OK, the door opener was working, so I took a quick look at the garage controller, it looked fine, all the little blinking lights were working. I took a look at the various house control logs and they were weird. I was getting a message that said that the USB stick was a read-only file system. It's important to note that all the devices were still controlling things perfectly. Over the years I made sure that their default operation didn't depend on the house controller, The house controller is a convenience and logging device, nothing totally depends on it. This is when I made my most serious mistake, I tried rebooting the device.

Naturally, it wouldn't boot. I took the whole mess of wires and components to the kitchen table and started trying to understand what was going on. I couldn't read the file system on a laptop since it was a Linux file system, so I went looking for something that could help, but instead I found a number of tools that wouldn't work with a USB stick. I didn't want to spend hours trying things out, so I just edited my boot SD card back the way it was before I added the USB stick. The little Pi booted first try with software a year old. Of course this messed up my data logging because things have changed a lot in the last months.

I searched the web for solutions to the read-only-USB-stick and found lots of people that had also encountered the problem, but nothing that would actually solve it. Time to look at my backups...I didn't have any! Yep, I relied on the stick to keep working and my source repositories on GitHub as my backup plan.

Fine, I'll just rebuild it. The problem was that I no longer had my source repository on the Pi, it was gone with the USB stick. If I had been coherent while the Pi was running read only, I could have just copied it off and gotten on with life. The source was there in GitHub, all I had to do was get it. So, I wiped my really old source directory from the SD card and downloaded a zip file from GitHub and installed it.

Now, I had the source, but the various libraries and packages that I've accumulated over the months weren't there. It became a chore of try something, get the missing stuff, try it again - repeat for each and every process I'd put together and modified over time.

My notes here on the blog were extremely valuable. I kept going back to see what I did in a particular instance which gave me clues to what I needed to add and modify. As I stepped through things, I even noticed a couple of bugs that hadn't given me problems before. Basically, I recreated my house controller from notes on this blog. But, my configuration file 'houserc' was a total loss and I built it up from scratch one line at a time, rediscovering the various entries I needed as processes printed error messages. I can't put that one on GitHub, it has the keys and passwords used for all the cloud servers. Eventually though, the pieces came together and everything started working again.

I managed to recreate my source repository so I could get back in sync with GitHub and update the minor changes I made to the code. That turned out to relatively easy. My objections to GitHub from the past seem really silly now. You folk should consider using it.

The USB stick is a total loss. I can't format it because it's read only, and I can't read it on a laptop because I can't format it back to DOS. Guess I'll take it apart and see what's in there.

So, did I learn anything? I learned not to trust USB sticks, they just aren't meant to be the primary device on a system. How about making backups more often? Well, I'm backed up now, but when life starts interfering, I'm sure I'll get lax about it. Automate the backup process? There's a possibility, but it costs money in the form of somewhere to back it up to, and time rotating the backups so they stay relevant. There's no really good solution, just a bunch of compromises.

I guess I'll start saving for a network appliance, but I still think the 3D printer is more important.

Sunday, March 1, 2015

Battery Operated Temperature Sensor - Now About Battery Life

I'm still working on my battery operated sensor and have been testing it for battery life and stability. I've learned a thing or two. First, this is absolutely possible, but not like the various tutorials out there lead you to believe; I'll never be able to get years of battery life like they imply, Second, I'll never be able run a battery completely down.

There's a number of reasons for this, but it will take some explaining. I've been saving the data I'm gathering and some things are noteworthy. Here's a graph of the data I've accumulated so far:


For this, only pay attention to the black line, I'll address the temperature (blue) later. Notice that it starts off around 2.9 volts, that's because I didn't get to uploading the data until the battery had dropped some, then there's some chatter where it started to drop to 2.8 volts and a long string of 2.8 volts until it jumps up. What happened was I only kept one digit to the right of the decimal and the chatter is where it was close to the rounding point, 2.85 volts. Then, because it quit transmitting just after noon on the 26th, I have a straight line until I changed the batteries on the 28th.

So the jump and relatively rapid decline is where I changed the battery and the code to a 50 second on versus 10 second off rate. I wanted data faster than a week at a time. I also added code to sense that it stopped transmitting and drop the reading. This way the battery getting too low would be readily apparent. I learned two things from this: use more precision, and heavier drain to get experience in a shorter amount of time.

Now about the weird temperature readings. After a couple of days of watching the temperature inside, I got bored with the flat line and took it outside. That's what the cyclic activity is from the 20th to the 23rd is.  The drops are experiments I did with the temperature sensor. Things settled down in the temperature sensor before the 26th, but I did still move it around the house some.

Now, take a look at this chart. This is the period after I changed the batteries and was using 50sec on, 10 sec off timing:


No, the batteries didn't last long, but that was my objective for this test. Notice that the device quit when the batteries dropped to 2.754 volts; that makes perfect sense when you think about it a bit. All electronics have some cut off point where they just quit working, for this combination of components, that's the cut off point. The temperature sensor is only good down to 2.7 volts and the processor is over-clocked as it is since I'm running it at the full 16Mhz, and lowered voltage.

What this means is that the rating of the battery doesn't mean much. These things have roughly 1600-1800 mah, but since the device dies at 2.75 volts, I'm only getting a portion of that. Don't make the mistake of thinking you can get all the power out of a battery, the lower voltage will step up and bite you.

There's a lot I can do to extend the battery's usefulness now that I know where it dies. First, I can adjust the sleep-awake factor. It turns out I need less than 10 seconds to take a reading and send it. I could probably drop it as low as 2 seconds. That will mean 58 seconds of sleep where it is drawing microamps of power to keep the timer running. I could add a battery and make it a three battery device. With three batteries I would start out with close to 4.65 volts, giving me almost two volts of range before it quit. This would mean a voltage regulator, but there are low current versions of them that would work. Ideas like a boost power supply aren't going to work because they take a lot of power just to run them. I'd get more out of the batteries, but it would mostly go to supporting the boost supply. I could lower the clock speed on the processor, but that makes timing and other things much more complex.

The test also told me that the battery voltage declines relatively rapidly from 3.1 down to 2.8 where it hangs in for a long period of time. Most of the current I got out of the battery was at this voltage.

Now, I'm going to set it up again for a shorter awake time and keep the code in that indicates when it dies and see what happens while I wait for a nice regulator to come in the mail. Then I'll test it with three batteries and regulation to see how that works out. I don't really want to use three batteries in each device, but I'll take the best road when I know more. Batteries are cheap in bulk, and I'd rather use more if it means the device will last longer.

Some of you will wonder, "Other devices can last a year or more on two batteries, what's the big deal?" Well, that's true, but they're activated by something. The smoke detector you have is asleep most of the time, and awakened by an external signal. For something like a door sensor, I could put it to sleep for several minutes at a time and wake up to send a message saying it was still alive. It would also wake up on a change in level of the actual sensor; I could probably get a year out of something like that. It's a different ball game. I want to monitor the temperature often enough to be of value, but not constantly. Once a minute is fine and maybe once every couple of minutes, but longer is something I'll have to think about.

More later as I get more experience.

Monday, February 23, 2015

Battery Operated Temperature Sensor - Smarter TMP36

Last post <link> about my project to create a useful battery operated temperature sensor I whined a while about the flaky behavior of the TMP36 sensor.  I'm not the only person that has had this problem.  A simple search with Google will turn up person after person that has fought this problem. People have tried intricate filtering techniques, averaging, continuous averaging, everything they can think of to settle this device down. In the post I used the average of ten readings to get mine.

Even with that, I had some trouble.  This is on a battery supply, so there's no ripple or chatter from a buck supply, very low noise. I thought it could be the XBee transmission, so I added a little time delay to keep the XBee from transmitting at the same time as the reading. Nothing seemed to help Then, it occurred to me that I'd had a TMP36 setting outside hooked to an XBee using a pretty noisy wall wart for a supply continuously for a couple of years and never noticed a problem with it.

What's up with that? Why did it work on a fence post and not in my house?

I drug out the remains of the fence post temperature sensor from the box I'd tossed it into and thought about it a bit. I had put a voltage divider between the output of the TMP36 and ground to extend the range of the reading. Since I was using a tiny reference voltage, the max temperature I could read would be 120F and I needed more than that to have an outside sensor here in AZ (Don't try telling me to move, winters here are wonderful). Could the device need a load on the output?

I looked up a datasheet on the TMP36 and looked at the schematic on the device; the device has an open emitter output. I had never noticed that before because I was looking at tutorials and such and they all showed the TMP36 feeding directly into the input of an Arduino or other MPU.

I had to try it.  I dragged out a bag of resistors and took a look at the maximums of the TMP36 and settled for a 47K resistor, and grabbed a .01ufd capacitor and added them to the circuitry of the board. Everything worked fine, but I still had the averaging in the code, so I removed that and put the board back in service.

That seems to do it. The little temperature sensor is chugging away giving me nice stable readings. It hasn't been in service for very long, but I usually got a spike in the reading in that amount of time. Just to be clear on what I did, here's a schematic:


The .01ufd cap is as close to the chip as I can get it as is the resistor. I simply plugged them into the breadboard.

Could it be that all the tutorials out there are wrong ? I went looking again and never found a place where someone pointed this out. There were long discussions on the time it takes the input cap on the A/D converter to charge, a myriad of algorithms on smoothing (some of them mine), etc. Then, I stumbled across another person that came up with this <link>, Dr. Monk found the exact same thing. I was a little embarassed until I noticed that he posted on the 16th of this month, after I started this particular project and well after my other attempts, so I was right in line with his discovery. Great minds think alike (slowly).

I'm going to watch this over time, but it brings this little temperature sensor back as a very real possibility. I'm also going to try another sensor the DS18B20 since it comes in a cool waterproof housing as well as the TO-92.  I even want to experiment with a bimetal sensor that is heat resistant for monitoring food.

I can't believe I missed this for YEARS.

Edit: I finally got this sensor under complete control with some additional work on the software detailed here <link>.

Wednesday, February 18, 2015

Battery Operated Temperature Sensor

This project will take a long time to complete, but I wanted to post about it because it's a nice project for a person starting off in Home Automation.  I do a number of things in this project that can lead to even more projects later; this could be the first step for a person that wants to take control of their house without the huge expense of one of the ready made systems out there.

The story begins after I installed my new barometer out on the fence <link>, I was looking at the temperature sensor that I removed <link> and thought, "It would be cool to have temperature sensors that I could put anywhere around the house."  It would be nice to see the temperature in the hot parts of the house and maybe move some air in there when company is around. If I used a sensor from one of those oven temperature probes, I could watch a roast from anywhere; this could be useful ... so I built one.

I wanted to be able to power it with batteries and be able to read the output on a monitoring XBee, so that would mean a processor of some kind and a lot of learning about low power techniques. Low power pretty much leaves out a normal Arduino, they have power regulators on them and those things constantly consume power; they also get warm and that would mess with the temperature readings. I decided to use one of my Ardweenys.  If you're not familiar with these, here's what they look like:
They're not the tiniest Arduino derivative, but they are really close.  The beauty of these devices is that they have a tiny footprint and no unnecessary components to use power. I don't have to come up with a design for supporting the processor and can just use the device.

Since the new device was going to be battery powered, I would need to somehow monitor the battery level so I could change them when needed.  Did you know that the atmega328p can monitor its own power?  Yep, it can, you just have to be good with google to find out how.  There's also a temperature monitor in there so you can keep track of the heat generated.  I don't actually need that, but it would be fun to do it. Here's the code I used to capture the processor temperature and supply voltage:

/*
These two routines use the atmega328 built in thermometer and VCC measurement techniques
This way I can get the supply voltage to see how the batteries are doing, as well as grab
the temperature of the chip.  It may need calibration, but it's free
*/
float readTemp(){
  long result; // Read temperature sensor against 1.1V reference
  ADMUX = _BV(REFS1) | _BV(REFS0) | _BV(MUX3);
  delay(20); // Wait for Vref to settle - 2 was inadequate
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
  result = ADCL;
  result |= ADCH<<8;
  result = (result - 125) * 1075;
  return (result / 10000.0) * 1.8 + 32.0; // I want degrees F
}

float readVcc(){
  long resultVcc; // Read 1.1V reference against AVcc
  ADMUX = _BV(REFS0) | _BV(MUX3) | _BV(MUX2) | _BV(MUX1);
  delay(20); // Wait for Vref to settle
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
  resultVcc = ADCL;
  resultVcc |= ADCH<<8;
  resultVcc = 1126400L / resultVcc; // calculate AVcc in mV
  return (resultVcc / 1000.0); // but return volts
}


The units (Fahrenheit and Volts) are just my choice, Feel free to use degrees C and millivolts if you want to.  These routines ONLY work on certain of the Atmel processsors, but I'm only going to be using the 328, so that's not a problem.

Next I need a temperature sensor. I've already worked with the tmp36 a lot, and I have one of them on hand, so I went with it. I really can't recommend the tmp36 though; the things are skittish and have to be catered to a bit.  I've found with a lot of experimenting that stabilizing the power supply a lot and reading them several times in a row and taking an average will give a good reading.  Since I can read it 10 times and take an average in a few microseconds, that should work just fine. The power supply noise will only be a problem while I debug it because it will run on batteries.

Edit: I beat the interaction with this sensor. I can get nice readings that seem to be very accurate without too much trouble, the solution is both hardware and software and is described here <link> and here <link>.

/*
Reading a tmp35 chip to get temperature.
These devices are a little flakey. If the power has even a tiny
bit of noise, they give erratic readings. So, filter them as
best you can and then take several readings and average them.
*/
float readTemp2(){
  int total = 0;
  for (int i = 0; i < 10; i++){
    total += analogRead(0);
  }
  int reading = total / 10;
  float voltage =  (reading * 1.1) / 1024;
  float tempC = (voltage - 0.5) * 100;
  float tempF = (tempC * 9.0 / 5.0) + 32.0;
  return(tempF);
}

I've used this type of code several times for various experiments and it seems to work pretty well. There's other techniques, but they're much more complex. I'll try this for a while and see how it holds up in actual operation over time.

Of course, since it's me, there will be an XBee in there to transmit to my House Controller. I've never tried to get the power down low on an XBee, so this should be interesting. Notice how 'interesting' and 'frustrating' often mean the same thing? This time, there really wasn't any problem. I wanted the processor to control the XBee, so all I needed to do was set the XBee up for a pin controlled sleep and the processor could control it through a digital pin.

One of the cool things about sleep and processors is that the GPIO pins retain their state when you put them to sleep.  So, you set a pin high and put the processor to sleep, the pin stays high. That was a bit of a surprise at first, but it makes sense when you think about it a bit. This makes it really simple to put the XBee to sleep, just:

#define SLEEP HIGH

      digitalWrite(xbeeSleepReq, SLEEP); // put the XBee to sleep

I used a #define for SLEEP because I kept forgetting which state I needed to set the pin to.

The idea is that the processor starts, initializes the XBee and sets up anything else needed, then goes to sleep.  Sometime later it wakes up, takes a temperature reading, sends it, then goes back to sleep. Repeat forever, or at least until the batteries die.

Now, all I have to do is learn how to put the 328 to sleep. Yep, there's thousands of blog posts and tutorials out there on how to do this. Tons of examples of setting registers and shifting bits to control internal counters and such. Yuck! I really don't want to spend a few days learning the ins and outs (pun intended) of the 328P registers, I want to use them. JeeLabs to the rescue. Jean-Claude Wippler of JeeLab has created a nice library that includes a function to handle sleep without having to learn a ton of marginally useful register manipulation. However, I need to wake it up also; it's not very useful to put it to sleep and just leave it there. I can't do my usual technique of asynchronous timers in this instance because the timers won't run when the 328 is asleep, so I have to rely on the milli() function. Here's the code I came up with to control the sleep-awake cycle including putting the 328 to sleep.

  if (millis() - savedmillis > AWAKETIME){
    Serial.print("was awake for ");
    Serial.println(millis() - savedmillis);
    delay(100); // delay to allow the characters to get out
    savedmillis = millis();
    unsigned long timeSlept = 0;
    while (timeSlept < SLEEPTIME){
      digitalWrite(xbeeSleepReq, SLEEP); // put the XBee to sleep
      Sleepy::loseSomeTime((unsigned int)1000);
      timeSlept = (millis() - savedmillis);
      digitalWrite(xbeeSleepReq, AWAKE); // wake that boy up now
    }
    Serial.print("was asleep for ");
    Serial.println(millis() - savedmillis);
    savedmillis = millis();
    sendStatusXbee();
  }

AWAKETIME and SLEEPTIME can be changed to limit battery usage while still leaving enough time for the XBee to stabilize and complete the transaction. I'll have to experiment with this over time to find some balance based on real-life experience.  But notice how easy it was to put the board to sleep for a timed period?  One litle call to loseSomeTime() with the length of time I want to sleep was all there was to it.

A problem crept in this technique though.  Since the board is asleep, it isn't counting milliseconds, so the JeeLab library estimates the duration and bumps the millisecond timer to get it close to being correct. The problem comes when an interrupt happens. When something interrupts, the board wakes up to handle the interrupt, and when using SoftwareSerial to get an additional emulated serial port, you get interrupts. That's why I keep my own count and repeat the loop to sleep some more when this happens. In practice, I lose time this way, but it's been less than a second in the testing I've done so far. The timing doesn't have to be perfect, just good enough and this seems to fit the bill nicely.

So, I do some initialization, go to sleep, wake up, send the status and go back to sleep. I based the code on the XBee example I gave in a previous post for the barometer <link> I have outside. I had to take out some stuff and add others, but the code is sorta, mostly the same. I still have the command handling in there in case I need it.

Now, armed with enough hardware to get this project started I cobbled together a setup on a protoboard and started putting it together.


Yes, that messy lump of wires is my temperature sensor.  It takes up less than half a full sized breadboard, and has a few locations to spare.  There were a number of things that crept into the project that after getting it going I didn't expect.  For example, when the board is asleep too long, the XBee coordinator forgets about it. What happens is that the coordinator expects some interaction with every XBee on the network periodically, and if it doesn't get it, it removes the entry from its table of devices. So, on the first few tries, the temperature sensor couldn't connect when it woke up and I got error 22 or 24 depending on what was going on at the time.  A significant amount of time later I had a pretty good solution; just change the timeout on the coordinator for this item to be significantly larger than the sleep time on the temperature sensor.  I decided to make it three minutes and give it a try since I can change it later. The complication to this is that you have to change it for the coordinator and all the routers on the network.  Fine, I just fired up XCTU and used the remote command capability to update the various XBees one at a time.  The values I used in this case were:

SP = 7D0
SN = 9

The SP parameter evaluates to 20 seconds and the SN parameter means do it 9 times. Three minutes on a device that reports once a minute should be fine.  If extended testing indicated that I needed to change it, I'd just have to do the change over again. Not too bad.

Another thing was that I tried sending too soon after waking up the XBee.  The XBee takes a small amout of time to get itself going well enough to respond to send requests or commands, so I had to enable the CTS (clear to send) capability and hook it up to a 328 pin.  After doing that all I had to do was hang in a loop until the CTS signal changed and then I could send to my hearts content.  To set the parameter use the value:

D7 = 1

And my simple solution to monitoring it before sending was:

void sendXbee(const char* command){
  ZBTxRequest zbtx = ZBTxRequest(Destination, (uint8_t *)command, strlen(command));
  while (digitalRead(xbeeCTS) == SLEEP){} // just hang up and wait 
  xbee.send(zbtx);
}

I just hang in a while loop until the pin goes low.  It was nice how the #define I came up with worked in this case.

I decided to play around with it hooked to a battery and rewired the mess until it looked a little better and this is the device as it exists right now:


Maybe it isn't any less messy after all.  I did get to use my rubber band though.  And yes, I use cheap batteries. No, I don't expect you to be able to trace the wires from this.  Here's a Fritzing image of it:


Note that the black thing is NOT a 328 chip, it's an Ardweeny, I couldn't find a Fritzing part for the Ardweeny that worked.  And, for my 'more advanced' readers:


Once again, it isn't a 328, it's an Ardweeny, and of course I'll give you the code, it's in GitHub already with my other Arduino projects and examples <link> it's called RoomTemp; I also added Jeelib so I wouldn't have to keep track of changes to it. But remember, I'm not done with this yet.

I'll have to monitor the battery usage over time and make adjustments or changes as necessary, bugs may show up, or I may get a brilliant idea. The eventual goal is to have them in several of my rooms as well as a couple to play with.  I may even carry out my threat to hook one to the barbecue for monitoring steaks. I'm monitoring this on my Raspberry Pi house controller, but not doing anything with the readings yet.  Here's a sample of the output I'm using:

{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.9', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.9', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.7', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.2', u'temperature': u'73.7', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.7', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.9', u'voltage': u'3.0'}}
{u'Temp1': {u'ptemperature': u'79.4', u'temperature': u'73.7', u'voltage': u'3.0'}}

I'm using a JSON string and the 'ptemperature' value is the processor temperature I mentioned above, the voltage is taken from the processor also. Of course you can change the code and use any format you want to in the message.

Have fun.



Saturday, February 14, 2015

Acurite Console Barometer: OK, This is it.


Back in a previous post I tried to decode the barometer reading from the AcuRite console.  I just couldn't get it, but one of the readers, Play Kube, stepped up and nailed it for us. The post is here <link> and his findings are down in the comments.  He emailed them to me and I posted them for him. Then I added code to my stuff and recorded the readings for a few days and this is the correlation to my (very accurate) fence post barometer. I spent quite a bit of time making sure my barometer compared to the weather stations around me and it was dead on.  The decoded AcuRite readings follow mine really, really well.  Take a look:


Isn't that great?  There's a couple of caveats though: The console barometer is NOT temperature or altitude corrected.  That's because we don't have access to the temperature or altitude corrections that the chip manufacturer supplies inside the chip. These values can be read and the math applied to get the reading dead on, but we don't have them to work with.

The way I corrected the readings was to simply adjust the algorithm to compensate.  This means that if I move the console up or down by a significant amount (I move to sea level) or the temperature changes (put it in the sun) the readings will change.  However, in most of our houses the temperature stays pretty stable and it's unlikely we'll be moving soon.  If one of these does happen, just readjust the algorithm and you'll be all set for the next time.

Looking at the chart, the difference in the readings is less than a mbar and tracks exactly.  Here's the code I used to decode the reading:

                    float bar = 6.23 * (R2[23] << 8 | R2[24]) - 20402;
                    weatherData.barometer = bar / 100; // convert to mbar from pascals
                    weatherData.barometer += 81.1; //adjust for altitude

It's a simple linear equation like a line graph from high school algebra.  Take the last two bytes and form a single number from them, then use a slope of 6.23 and an offset of -20402.  This will get you the barometric pressure reading if you were at the same altitude and temperature as the person that figured it out. So, to adjust it to your location, just go look up the barometric pressure reading for your location at the same time and add the difference in.  That's what the 81.1 number above means, I just added it in as the last step. The changes will be in GitHub soon.

Don't think this is too simple a solution.  The manufacturer of the chip put a lot of work into getting the sensor to follow pressure changes, then they put the adjustments for temperature and fixed altitude in the chip so they can be read and applied.  There was a lot of work done to get it to work. The trick was to get the slope correct, then the offset was just the method of getting it to line up vertically.  The altitude offset is simply necessary to adjust for the specific location.

Slick, but remember, change altitude or temperature and you may have to revisit the algorithm and change the values. We owe this discovery to Play Kube for sending me the mail a few days ago.

I still haven't found the battery level from the weather head though.

Wednesday, February 4, 2015

Reading the AcuRite 5n1 Sensor Set, This time with RF

I've managed to read the RF signal coming from the AcuRite weather head.  It wasn't too hard to do since I had some significant help from other folk that have done the same thing.  The problem was only that there wasn't anyplace out there that I could just look at for an answer that was somewhat coherent to someone that wasn't conversant in the latest digital tuner techniques.  It turns out the answer is quite simple to do, once you work your way through the confusion.  I hope I can clear it up for folk because this is the last piece of the AcuRite puzzle that I think is worth pursuing. 

First, get one of these:


This the tuner chip that is used in digital TV that has been hooked to a USB interface and enclosed in a plastic case.  The chip is a RTL2832 and has a huge bandwidth; it can be used to receive AM, FM, SSB, etc all the way up past the 900MHz range that the AcuRite transmits on.  It's a fun device in and of itself and is the subject of blog posts all over the web. You can listen in to commercial flights, the cops parked at the corner watching your garage, even the doorbell transmitter in your neighbors house.  I didn't want it for boring things like this, I wanted to get the rainfall count out of my 5n1 sensor set without using the console.

So, I downloaded a great tool for doing this kind of thing called SDRsharp and installed it on my laptop.  After visiting several web sites that described how to use its features, and listening to the five or six FM stations I can receive out here, I went looking for the AcuRite sensor.  I found it.


That's it, the spike that is showing on the capture.  It took me a little while prowling around the spectrum to find it, but there it is, Oh ... wait ...


Notice in the bottom panel that there are three different signals that are on almost exactly the same frequency?  Yep, I'm picking up three devices up in the 433MHz area and any one of them could be what I'm looking for.  After a bunch of looking, it turns out the first one I found is the one I want, but how to I get the data out of it? Heck, how do I separate the one I want from the others?

This has already been solved for us under Linux by a tool called rtl_433.  But before we talk about that, use SDRsharp to zero in on the frequency you need to use.  There's a little drift between devices both at the radio and at the transmitter, getting the exact frequency reading will help you later.  Write it down somewhere.

OK, I'm getting too technical without explaining what is going on.  These little tuner chips were designed for digital TV and the description (stolen from here <link> is:
What is RTL-SDR?
RTL-SDR is a very cheap software defined radio that uses a DVB-T TV tuner dongle based on the RTL2832U chipset. With the combined efforts of Antti Palosaari, Eric Fry and Osmocom it was found that the signal I/Q data could be accessed directly, which allowed the DVB-T TV tuner to be converted into a wideband software defined radio via a new software driver.
Essentially, this means that a cheap $20 TV tuner USB dongle with the RTL2832U chip can be used as a computer based radio scanner. This sort of scanner capability would have cost hundreds or even thousands just a few years ago. The RTL-SDR is also often referred to as RTL2832U, DVB-T SDR, or the “$20 Software Defined Radio”.
There's an entire web site dedicated to these little devices at http://www.rtl-sdr.com/ and another at http://www.rtlsdr.com that describes many of the uses people have come up with, so I won't go into it much more deeply.

Next, there's a Linux library that supports this and a really cool tool you can run on your PC to play around. The tool is called SDRsharp and has a dedicated website <link>; just go there and select the download link to get it.  The screenshots above are taken from this tool

Now, before I get into describing how to use this on a Raspberry Pi, you should be aware that these little radio receivers pull a lot of power.  When you plug them into the Pi, it will suck so much power that the Pi will reset and have to boot back up.  If your power supply is not up to it, the radio won't work properly.  A powered hub will help isolate the Pi from the high current, but isn't absolutely necessary if you power supply is strong enough and you don't mind it rebooting when you plug the receiver in.

The Linux library is called  rtl_sdr and is at http://sdr.osmocom.org/trac/, but don't download it yet. Don't misunderstand, this is a nice tool, but not exactly what you want to read the AcuRite weather head. What you want is a tool called rtl_433 that was developed specifically to decode the various devices that operate in the same unlicensed frequency range.  Things like temperature sensors, doorbells, full blown weather stations, etc.

There are a lot of blog posts out there on how to install the rtl_433 tool, but none of them worked for me.  I had to actually follow the instructions as written in the readme file that came with it because the authors had recently added the general purpose library for SDR (software defined radio) when they found out it needed to be changed to help their tool work better. When I downloaded the zip file, I got it all.  It took me about three days to figure that out and get it to compile and work.  Then, when I went back later in preparation for this post, they had changed rtl_433 a bunch. I guess someone got a burst of energy and they updated everything in sight. They had gone back to installing rtl_sdr as a separate step and that would make things much harder.

I just created a new one under my login on GitHub.  I really didn't like the idea of things changing every time I wanted to do something and using it for my weather station would depend on two projects out there that can change at any time.  I tried to use the Git fork and other tools and got totally confused with the various methods of backing up in a respository; I guess my very own version isn't a bad solution.

There is a dependency on libusb though.  If you've already installed my USB code that previous posts described, you've got that requirement already.  If you haven't, you'll need to install libusb. Go to my post on this <link> and follow the part about installing libusb and libudev-dev.  This is a USB device (of course) and is very, very similar to the code I created to read the console device.

Now you're ready to work on rtl_433, so just go to github and grab it <link> by clicking on the 'Download ZIP' button on the lower right.  To build it though, they use a tool called 'cmake' that isn't on the Pi by default.  Fortunately, it can be installed easily

sudo apt-get install cmake

Now you can (finally) just follow the instructions in the readme that comes with the library to build it.  Don't do like I did and think I knew everything and install the rtl_sdr library and try to get it to link up properly with rtl_433; that is a path to almost terminal frustration.  Just build rtl_433 from my copy. You'll probably want to rename it once you get it unzipped to something easier to type.  Here's the commands I used to build mine:
cd Desert-Home-rtl_433-master
mkdir build
cd build
cmake ../
make
Now, you are almost ready to actually read the RF from the weatherstation, but (another one of those) the decoder included with rtl_433 is wrong. I don't know if the author of that particular piece had the same kind of problems I did originally, but he made some of the same mistakes I did.  So, I put together my own version that decodes my weatherhead and corresponds with the console. These changes are part of the zip file you downloaded; you don't have to do anything special.

The great thing about doing it this way is the weatherhead provides a checksum on the data.  That means no more undetected short packets or weird data because something in the AcuRite console is messed up. It also means you only get the 5 sensors included in the weatherhead: temperature, humidity, wind speed, wind direction, and rainfall count.  But, since I installed my own barometer <link>, and plan on building anything else I may need, that's good enough for me.

Just for completion's sake, here is sample output from my weatherhead:

pi@deserthome2:~/src/Desert-Home-rtl_433-master/build$ cd src
pi@deserthome2:~/src/Desert-Home-rtl_433-master/build/src$ rtl_433
Registering protocol[01] Rubicson Temperature Sensor
Registering protocol[02] Prologue Temperature Sensor
Registering protocol[03] Silvercrest Remote Control
Registering protocol[04] ELV EM 1000
Registering protocol[05] ELV WS 2000
Registering protocol[06] Waveman Switch Transmitter
Registering protocol[07] Steffen Switch Transmitter
Registering protocol[08] Acurite 5n1 Weather Station
Found 1 device(s):
  0:  Generic, RTL2832U, SN: 77771111153705700

Using device 0: Generic RTL2832U (e.g. hama nano)
Found Rafael Micro R820T tuner
Exact sample rate is: 250000.000414 Hz
Sample rate set to 250000.
Sample rate decimation set to 0. 250000->250000
Bit detection level set to 10000.
Tuner gain set to Auto.
Reading samples in async mode...
Tuned to 433920000 Hz.
Detected Acurite 5n1 sensor
wind speed: 2.0 mph, temp: 54.5F, humidity: 67%
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, wind direction: NE, rain counter: 344,
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, temp: 54.5F, humidity: 67%
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, wind direction: NE, rain counter: 344,

Notice how the build process put the files in the directory rtl_433/build/src/ ?  I'm not responsible for that ... that was the way I got it.  Just move them wherever you want them to reside at run time.

Remember way above I told you to write down the frequency you found the weather head transmitting on?  This is where you may have to use it.  If it drifted far enough, you won't get a good signal from the transmitter up on the roof or out in the yard.  You can simply specify the frequency you want to listen to with the '-f' parameter like this:

rtl_433 -f 433.915e6

This means to set the receiver at 433.915 MHz.  I actually use this number for my weather head.

There's still some stuff to be done from my perspective, I want to put some code in to present the data as a JSON string so it can be used exactly like I did in the other version.  I also want to wait until I have both types of messages in hand before I output anything.  I'll be doing that over time and moving to exclusively use the RF from the weather head.  That will eliminate all the little irritations that the console entails. So, if you grab the code at some point down the line, there will be changes that reflect my specific use; if you want this particular version, the commit # in Github is 5c72e32 and you can get to it by clicking on 'Commits', then finding the number in the list and clicking on the browse next to it that looks like this <> . That will get you back through any changes to the level it was when I wrote this.

This will also make the Raspberry Pi a possible replacement for their expensive bridge device that is limited in what it can do and completely closed to us changing it.

But what will I do with the console?  It's actually an attractive device and I think I'll put it in the kitchen somewhere.  It will give me a nice conversation piece in the most often used room.

Have fun.

Edit Feb 13, 2015:  One of the readers, John, had a problem with attaching to the SDR device.  He chased it down to some code needing to be added in the USB open code.  He emailed it to me and I tried out his fix and it worked fine.  The problem relates back to removing the default hardware driver when starting up like I mentioned way back in a previous post <link>.  I tested it to be sure it would work for me as well and it was great.  I've already put the change into the github repository as commit number 86c70cf .  See, it does pay off to read all the posts after all. Feel free to grab it