Wednesday, February 4, 2015

Reading the AcuRite 5n1 Sensor Set, This time with RF

I've managed to read the RF signal coming from the AcuRite weather head.  It wasn't too hard to do since I had some significant help from other folk that have done the same thing.  The problem was only that there wasn't anyplace out there that I could just look at for an answer that was somewhat coherent to someone that wasn't conversant in the latest digital tuner techniques.  It turns out the answer is quite simple to do, once you work your way through the confusion.  I hope I can clear it up for folk because this is the last piece of the AcuRite puzzle that I think is worth pursuing. 

First, get one of these:


This the tuner chip that is used in digital TV that has been hooked to a USB interface and enclosed in a plastic case.  The chip is a RTL2832 and has a huge bandwidth; it can be used to receive AM, FM, SSB, etc all the way up past the 900MHz range that the AcuRite transmits on.  It's a fun device in and of itself and is the subject of blog posts all over the web. You can listen in to commercial flights, the cops parked at the corner watching your garage, even the doorbell transmitter in your neighbors house.  I didn't want it for boring things like this, I wanted to get the rainfall count out of my 5n1 sensor set without using the console.

So, I downloaded a great tool for doing this kind of thing called SDRsharp and installed it on my laptop.  After visiting several web sites that described how to use its features, and listening to the five or six FM stations I can receive out here, I went looking for the AcuRite sensor.  I found it.


That's it, the spike that is showing on the capture.  It took me a little while prowling around the spectrum to find it, but there it is, Oh ... wait ...


Notice in the bottom panel that there are three different signals that are on almost exactly the same frequency?  Yep, I'm picking up three devices up in the 433MHz area and any one of them could be what I'm looking for.  After a bunch of looking, it turns out the first one I found is the one I want, but how to I get the data out of it? Heck, how do I separate the one I want from the others?

This has already been solved for us under Linux by a tool called rtl_433.  But before we talk about that, use SDRsharp to zero in on the frequency you need to use.  There's a little drift between devices both at the radio and at the transmitter, getting the exact frequency reading will help you later.  Write it down somewhere.

OK, I'm getting too technical without explaining what is going on.  These little tuner chips were designed for digital TV and the description (stolen from here <link> is:
What is RTL-SDR?
RTL-SDR is a very cheap software defined radio that uses a DVB-T TV tuner dongle based on the RTL2832U chipset. With the combined efforts of Antti Palosaari, Eric Fry and Osmocom it was found that the signal I/Q data could be accessed directly, which allowed the DVB-T TV tuner to be converted into a wideband software defined radio via a new software driver.
Essentially, this means that a cheap $20 TV tuner USB dongle with the RTL2832U chip can be used as a computer based radio scanner. This sort of scanner capability would have cost hundreds or even thousands just a few years ago. The RTL-SDR is also often referred to as RTL2832U, DVB-T SDR, or the “$20 Software Defined Radio”.
There's an entire web site dedicated to these little devices at http://www.rtl-sdr.com/ and another at http://www.rtlsdr.com that describes many of the uses people have come up with, so I won't go into it much more deeply.

Next, there's a Linux library that supports this and a really cool tool you can run on your PC to play around. The tool is called SDRsharp and has a dedicated website <link>; just go there and select the download link to get it.  The screenshots above are taken from this tool

Now, before I get into describing how to use this on a Raspberry Pi, you should be aware that these little radio receivers pull a lot of power.  When you plug them into the Pi, it will suck so much power that the Pi will reset and have to boot back up.  If your power supply is not up to it, the radio won't work properly.  A powered hub will help isolate the Pi from the high current, but isn't absolutely necessary if you power supply is strong enough and you don't mind it rebooting when you plug the receiver in.

The Linux library is called  rtl_sdr and is at http://sdr.osmocom.org/trac/, but don't download it yet. Don't misunderstand, this is a nice tool, but not exactly what you want to read the AcuRite weather head. What you want is a tool called rtl_433 that was developed specifically to decode the various devices that operate in the same unlicensed frequency range.  Things like temperature sensors, doorbells, full blown weather stations, etc.

There are a lot of blog posts out there on how to install the rtl_433 tool, but none of them worked for me.  I had to actually follow the instructions as written in the readme file that came with it because the authors had recently added the general purpose library for SDR (software defined radio) when they found out it needed to be changed to help their tool work better. When I downloaded the zip file, I got it all.  It took me about three days to figure that out and get it to compile and work.  Then, when I went back later in preparation for this post, they had changed rtl_433 a bunch. I guess someone got a burst of energy and they updated everything in sight. They had gone back to installing rtl_sdr as a separate step and that would make things much harder.

I just created a new one under my login on GitHub.  I really didn't like the idea of things changing every time I wanted to do something and using it for my weather station would depend on two projects out there that can change at any time.  I tried to use the Git fork and other tools and got totally confused with the various methods of backing up in a respository; I guess my very own version isn't a bad solution.

There is a dependency on libusb though.  If you've already installed my USB code that previous posts described, you've got that requirement already.  If you haven't, you'll need to install libusb. Go to my post on this <link> and follow the part about installing libusb and libudev-dev.  This is a USB device (of course) and is very, very similar to the code I created to read the console device.

Now you're ready to work on rtl_433, so just go to github and grab it <link> by clicking on the 'Download ZIP' button on the lower right.  To build it though, they use a tool called 'cmake' that isn't on the Pi by default.  Fortunately, it can be installed easily

sudo apt-get install cmake

Now you can (finally) just follow the instructions in the readme that comes with the library to build it.  Don't do like I did and think I knew everything and install the rtl_sdr library and try to get it to link up properly with rtl_433; that is a path to almost terminal frustration.  Just build rtl_433 from my copy. You'll probably want to rename it once you get it unzipped to something easier to type.  Here's the commands I used to build mine:
cd Desert-Home-rtl_433-master
mkdir build
cd build
cmake ../
make
Now, you are almost ready to actually read the RF from the weatherstation, but (another one of those) the decoder included with rtl_433 is wrong. I don't know if the author of that particular piece had the same kind of problems I did originally, but he made some of the same mistakes I did.  So, I put together my own version that decodes my weatherhead and corresponds with the console. These changes are part of the zip file you downloaded; you don't have to do anything special.

The great thing about doing it this way is the weatherhead provides a checksum on the data.  That means no more undetected short packets or weird data because something in the AcuRite console is messed up. It also means you only get the 5 sensors included in the weatherhead: temperature, humidity, wind speed, wind direction, and rainfall count.  But, since I installed my own barometer <link>, and plan on building anything else I may need, that's good enough for me.

Just for completion's sake, here is sample output from my weatherhead:

pi@deserthome2:~/src/Desert-Home-rtl_433-master/build$ cd src
pi@deserthome2:~/src/Desert-Home-rtl_433-master/build/src$ rtl_433
Registering protocol[01] Rubicson Temperature Sensor
Registering protocol[02] Prologue Temperature Sensor
Registering protocol[03] Silvercrest Remote Control
Registering protocol[04] ELV EM 1000
Registering protocol[05] ELV WS 2000
Registering protocol[06] Waveman Switch Transmitter
Registering protocol[07] Steffen Switch Transmitter
Registering protocol[08] Acurite 5n1 Weather Station
Found 1 device(s):
  0:  Generic, RTL2832U, SN: 77771111153705700

Using device 0: Generic RTL2832U (e.g. hama nano)
Found Rafael Micro R820T tuner
Exact sample rate is: 250000.000414 Hz
Sample rate set to 250000.
Sample rate decimation set to 0. 250000->250000
Bit detection level set to 10000.
Tuner gain set to Auto.
Reading samples in async mode...
Tuned to 433920000 Hz.
Detected Acurite 5n1 sensor
wind speed: 2.0 mph, temp: 54.5F, humidity: 67%
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, wind direction: NE, rain counter: 344,
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, temp: 54.5F, humidity: 67%
Detected Acurite 5n1 sensor
wind speed: 1.5 mph, wind direction: NE, rain counter: 344,

Notice how the build process put the files in the directory rtl_433/build/src/ ?  I'm not responsible for that ... that was the way I got it.  Just move them wherever you want them to reside at run time.

Remember way above I told you to write down the frequency you found the weather head transmitting on?  This is where you may have to use it.  If it drifted far enough, you won't get a good signal from the transmitter up on the roof or out in the yard.  You can simply specify the frequency you want to listen to with the '-f' parameter like this:

rtl_433 -f 433.915e6

This means to set the receiver at 433.915 MHz.  I actually use this number for my weather head.

There's still some stuff to be done from my perspective, I want to put some code in to present the data as a JSON string so it can be used exactly like I did in the other version.  I also want to wait until I have both types of messages in hand before I output anything.  I'll be doing that over time and moving to exclusively use the RF from the weather head.  That will eliminate all the little irritations that the console entails. So, if you grab the code at some point down the line, there will be changes that reflect my specific use; if you want this particular version, the commit # in Github is 5c72e32 and you can get to it by clicking on 'Commits', then finding the number in the list and clicking on the browse next to it that looks like this <> . That will get you back through any changes to the level it was when I wrote this.

This will also make the Raspberry Pi a possible replacement for their expensive bridge device that is limited in what it can do and completely closed to us changing it.

But what will I do with the console?  It's actually an attractive device and I think I'll put it in the kitchen somewhere.  It will give me a nice conversation piece in the most often used room.

Have fun.

Edit Feb 13, 2015:  One of the readers, John, had a problem with attaching to the SDR device.  He chased it down to some code needing to be added in the USB open code.  He emailed it to me and I tried out his fix and it worked fine.  The problem relates back to removing the default hardware driver when starting up like I mentioned way back in a previous post <link>.  I tested it to be sure it would work for me as well and it was great.  I've already put the change into the github repository as commit number 86c70cf .  See, it does pay off to read all the posts after all. Feel free to grab it

31 comments:

  1. Back in Dec I ordered a 433mhz receiver and transmitter from e-Bay that I was going to hook to my Raspi to read my Acurite 5-in-1. After about 6 weeks I finally received it from China. I paid $1.56 for it and when it arrived the Philippine Postal Service charged me PHP100 which is about USD2.27 or almost 1.5% more than I originally paid.

    I haven't had a chance to work with it yet as I haven't the faintest clue on how to get started except for a lot Googling.

    Maybe your way will be the better way to go.

    ReplyDelete
    Replies
    1. I looked at the little inexpensive radios and they're cool, but I wanted to play with one of these. Sure, they cost more, I think I paid a little over 8 bucks for it (free shipping), but I got to play with the scanning software for days before I even tried to look at the sensors.

      I'm trying to think of another project to do with it, but haven't come up with one ... yet.

      Delete
  2. I've had a chance to play with these SDR "dongles" for awhile now off and on. For those truly into it, there's a software package called GNU Radio that's worth looking into. Back to the dongle, be sure to "calibrate" it first. Listen to a known frequency, for example the weather. In my area, it's found strongest on 162.500 MHz. Using SDR# or GQRX or whatever general software package you use, there's an adjustment window you open and a text box you click up and down in until the highlighted cursor aligns with the peak of the FFT display. Record that number (mine is 56 ppm). That same number is used in other software packages too. The calibration represents differences in the local oscillator on each of the dongles. There are plenty of YouTube channels that describe how this is done. The $20 dongle is great... but then when you look around, there are devices like the HackRF and Ettus Research B200 that start to look very interesting...

    ReplyDelete
    Replies
    1. Yes, for the radio folk out there, those devices could be a serious temptation.

      Delete
  3. I printed this last night, I prefer to read on dead trees and old blue jeans, i.e. paper and read it this AM.

    A couple of questions came up:

    1. In your data shot it shows the humidity as 67%. I thought the AZ desert is dry...
    2. The rain count shows 344. What does this number represent? Is that the number of times the buckets have tripped? Is there a way to convert this to inches/part of inches of rain?
    3. Have you considered saving the data to a database file? I have MySQL running on my Raspi with no problem. If the database gets too big it could be stored on an external hard drive.

    BTW, I just ordered a RTL-SDR dongle from Adafruit tonight. The cost was $22.50 and the shipping was $14.75. Will probably have to pay Philippine customs when it arrives.

    ReplyDelete
    Replies
    1. Man, you got ripped for the shipping, but I don't understand shipping outside my tiny little area of the world.

      Arizona is dry, but we seem to be having a reasonably wet winter this year. Most of the time the humidity is less than 20%. Wait for summer an take a look again.

      The rain counter represents .01 inches of rainfall per count. So, I've gotten 3.44 inches of rain since I turned it on. I'm not sure how to reset the counter, butt pulling the batteries will probably do it.

      I'll get around to putting this data into a database at some point, but just being able to get it was my goal.

      Delete
  4. If I remember from one of your previous post you had to pay more for postage than you did for the item. At least in my case the shipping is less than the item.

    Received a notice today that it has been shipped. Now will have to see how long it takes to get here to the Philippines. Before when we sent packages it would take 2-3 MONTHS. There have been times that a letter takes 2 weeks from the Post Office which is 5 km from the house.

    Sure is different from the US, but we have been living here over 10 years so just have to accept it as that is it.

    As far as a database goes. This am I did some calculations and saving a 7 column record every 3 minutes for a year would take only about 4mb of disk space, if my calculations are correct!!!

    ReplyDelete
  5. I finally received my DVB-T Stick and started to work with it following this post. The first snag I ran into is that the SDRsharp software available only for Windows. As I don't want to contaminate my Linux system with Windoze garbage I am now stuck until I can find a similar application for Linux.

    Any ideas of what might be available?

    Thanks.

    ReplyDelete
    Replies
    1. There's at least one for linux, take a look around Gnu Radio,

      Delete
    2. Did some more searching and found a program called "gqrx". It looks similar to your screen shot above. I then ran the rtl_433 program. It ran to the line "Tuned to 433920000 Hz." and then just hung. I moved the antenna and it finally recognized the 5in1 and gave me the data. One difference I noticed between your data and mine is it showed the rain gauge as 0.00 in instead of rain counter. Then it showed "AcuRite Rain Gauge Total Rain is 136.0mm". It then displayed "Rain Gauge is 64.0mm"

      I am running this on my desktop before I transfer everything to my Raspi, so I am getting closer step by step.

      Thanks for your great articles and help.

      Delete
    3. Remember, I modified rtl_433 a bit to fit my purposes. I wanted inches, millibars, etc. I also mucked around a bit with the readings to fix a couple of mistakes. So you will see some differences.

      Delete
  6. Got this the first time I tried it:
    Found 1 device(s):
    0: Realtek, RTL2838UHIDIR, SN: 00000001

    Using device 0: Generic RTL2832U OEM

    Kernel driver is active, or device is claimed by second instance of librtlsdr.
    In the first case, please either detach or blacklist the kernel module
    (dvb_usb_rtl28xxu), or enable automatic detaching at compile time.

    usb_claim_interface error -6
    Failed to open rtlsdr device #0.

    Fixed from the info here: https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=88653

    ReplyDelete
  7. Do you need one receiver/antenna per station, or can this actually pick up multiple stations at the same time?

    ReplyDelete
    Replies
    1. You can pick up multiple transmitters at the same time. Well, sort of. If two devices transmit at exactly the same time, they mess each other up, but since the devices are mostly battery operated, they seldom do that. The bandwidth is pretty wide also, so that means they can wander a bit due to heat and cold.

      Delete
    2. Awesome. And thanks again for putting all of this up (I'm sure it's under-said). It's a great source of info for people like me that have just enough knowledge to start but not enough time to really dig deep in to this (i.e. when it came to the RF portion with me).

      Delete
    3. Don't sweat the RF too much, it's literally like operating radio with a dial. Well, actually a bunch of dials. Well, some dials, some buttons, so arcane names for things that don't even come close to being understandable.

      I should shut up now.

      Delete
    4. Tuning's the easy part, reading the bursts with the RDS code would've taken me a few day or two of solid work :)

      Delete
  8. Thank you very much for this research. I just got this to work on my 5n1 bought at Costco late last year. rtl_433 is running on a PC with 64 bit Gentoo Linux.

    I did have to back out the change introduced by commit f9fcb6f in the acurite_crc() function - byte 1 was always failing the parity check even though the CRC passed.

    Also, in line 205 of acurite.c I changed bits_per_row[1] to bits_per_row[i] to get the number of bits on the actual matched line.

    ReplyDelete
    Replies
    1. Thanks for the update, I'll check it out in my running code as well.

      Delete
    2. Another thing I did was change the acurite_detect() code to not always flip the bits before calling acurite_crc(). Upon looking at the data received by using the -a option, I saw that I seemed to get alternate flipped and non-flipped messages. So now in acurite_detect.c I check for the crc on the non-flipped bits, and if that fails, I flip the bits and try again. This seems to help get more packets:

      static int acurite_detect(uint8_t *pRow) {
      int i;

      if (pRow[0] != 0x00)
      {
      if (acurite_crc(pRow, 7))
      return 1;

      // If the first attempt to get the checksum failed, it could be
      // because the bits are inverted. Invert them back and try again
      for (i = 0; i < 8; i++)
      pRow[i] = ~pRow[i] & 0xFF;
      //pRow[0] |= pRow[8]; // fix first byte that has mashed leading bit

      if (acurite_crc(pRow, 7))
      return 1; // passes crc check
      }

      return 0;
      }

      That, combined with setting the gain to the maximum (49.6) is reliably giving me a usable message every 20 seconds.

      Delete
  9. Good Read. My dongle arrives today and I have a 5 in 1 so here we go.

    ReplyDelete
  10. Hello,
    Great write up. after going through all the different pages landed on the SDR and went straight for it. It was the most simplest thing I have done on a while. I got it running on a PI3, bought the NooElec NESDR Mini 2 for about $20 and once there I got rtf_433 via apt-get.

    this is the command I am using to run the program and create a JSON file with the wether:

    rtl_433 -G -q -F json > test.json

    With the output of that file I wrote a quick perl script that decodes the JSON and uploads every 5 minutes to weather underground and writes the data into a database just in case one of these days I have nothing else to do and decided to take it to the next level.

    if anybody is interested, I can post the perl script. But Thanks, I get the signal from the station easily 100 ft from where the station is located and deep within the building.

    Thanks,

    Ken.

    ReplyDelete
    Replies
    1. Congratulations. I use my own database in house on a NAS and love it.

      Have fun.

      Delete
  11. I have gone through the steps, but still can't get it to recognize my Acurite 5n1 that I got at costco just before christmas. Any good debugging tips?

    ReplyDelete
    Replies
    1. Just to add some more info, it just sits on Tuned to "433499260 Hz." this is the frequency I get from gqrx

      Delete
  12. I've only had this problem when the signal from the 5n1 was too weak, but the way you can troubleshoot it is to take a look at the code for rtl_433 and start playing with the options.

    I had the 5n1 go silent on me a couple of times and had to reposition the antenna to get the signal back. This is probably not the case for you, so look at the debugging built into rtl_433.

    ReplyDelete
  13. Thanks for the quick response I was able to get it working. For me a used the standard rtl_433 it seemed to work a little better in my situation, and a bonus was that I was able to get support for one of our other sensors that we use as well. Thanks for having such a great blog post about this. Half the time it's understanding the problem well enough to get more help. This blog post got me going in the correct direction.

    ReplyDelete
  14. I have an emonPi with a RFM69Pi at 433Mhz and the La Crosse outdoor temperature and humidity sensor (TX141TH).
    And was wondering if I could read the temp and RH% using the RFM69Pi.
    I used minicom at 38400 on /dev/ttyAMA0 and get this every 10 seconds
    > 0,11,20,0,0s
    -> 4 b
    > 0,11,20,0,0s
    -> 4 b
    Which does not matchup with the actuals temp and RH. Plus the data comes in every 10 seconds, and the LaC sensor reports every 58-60mseconds.

    Do you think I can read the sensor data using the RFM69Pi on my emonPi?

    ReplyDelete
    Replies
    1. You might be able to, but it's hard to say with the hardware setup you have. If it were I, I'd get an SDR of some kind and run RTL433 on the Pi to see if you can see it with that radio. If you can, it may give you a hint on how to proceed.

      Delete
  15. Hi Dave -- do you have a link to your git repo for this project? I'm at the point where it would be useful.

    ReplyDelete
  16. https://github.com/draythomp/Desert-Home-rtl_433

    ReplyDelete