Wednesday, December 3, 2014

AcuRite Weather Station, Raspberry Pi and a USB interface; Part 3

Part 2 of this is here <link>, and part 1 is here <link>.

Well, we've talked about the weather station, how to create a usb device for it and how to get a device driver that can talk to it.  Now we have to get the data off the usb and do something with it.

Edit:  I forgot a couple of items and one of the comments brought it to my attention.  Run the weather station in USB mode 4.  Two reasons, modes 3 and 4 are the modes that send output to the USB port, and there's bug in the station that can mess you up.  Other folk out there have reported to AcuRite that the device fills up and doesn't empty.  When you're running and saving data on the console, some bug causes the device to stop saving in a couple of weeks and you get an error message on the scrolling display that won't go away.  The message is something like, "Datalogger full.'  Naturally, I found this out AFTER I bought it.

The usb device has two reports it provides, they're called Report 1 and Report 2 (duh).  R1 is a few bytes long and R2 is longer.  R1 has all the data from the weather head in it while R2 appears to have data gathered by the console.  For my purposes, I don't need anything from the console except the barometric pressure.  The sensor for that is inside the console piece and I haven't figured out how to get that part yet.

Yes, let me admit right up front I haven't figured out the barometric pressure reading.  The way you break into these things is to read the data, change something, look at the change, repeat until you have it figured out.  Barometric pressure doesn't change fast or far enough for me to have that ... yet.  I'll get it at some point.

However, wind speed, direction, a rain counter, temperature and humidity are all available and we can get that stuff from the device that's up on the roof.  It's really cool to be able to see that stuff on the Pi, so let's deal with R1 and save R2 for sometime later.  Here's the two reports that we can get out of the device:

R1 - 01 C0 5C 71 00 05 00 0C 03 FF
R1 - 01 C0 5C 78 00 08 1F 53 03 FF
R2 - 02 00 00 80 00 00 00 00 00 04 00 10 00 00 00 09 60 01 01 01 01 90 01 47 68

R1 is 10 bytes long and R2 is 25 bytes long.  In each message the first byte is simply the report number and the last byte is the end marker, so the actual data for the two R1's are:

C0 5C 71 00 05 00 0C 03
C0 5C 78 00 08 1F 53 03

It's complicated by the fact that there are two types of R1 messages.  Which type each one is, is determined by the low order half of byte 2 -- remember, we start counting from 0.  In the first case, the lower 4 bits of byte 2 are 0001, this message contains wind speed, wind direction and rain counter.  For the second example, the lower 4 bits of byte 2 are 1000 and this message contains wind speed, temp and relative humidity.  To test for it in code:

if ((data[2] & 0x0f) == 8){ // this has wind speed, temp and relative humidity

and

if ((data[2] & 0x0f) == 1){ // this has wind speed, direction and rainfall

We've just obtained the readings of all five of the sensors in the 5 in 1 sensor head for the weatherstation.  Next we have to pick the bits out of each of them corresponding to the sensors reported.  So, I'll decode an item; the wind direction is the low order half of byte 4 (counting from 0) in message type 1, so the value would be 5.  Look above in the data, byte 2 is 71 so the low order half is 1 which means it holds wind speed, direction and rainfall; see how it works?

The 5 is one of the sixteen cardinal directions starting with NNW being 0 counting counter-clockwise around the compass and ending with N being 15.  That makes this SW, and 6 would be SSW and so on.  To decode this into something readable, just set up an array and index into it using the value you get from byte 4.

char *Direction[] = {"NNW", "NW", "WNW", "W", "WSW", "SW", "SSW", "S",
                                   "SSE", "SE",  "ESE", "E", "ENE", "NE", "NNE", "N" };

And then:

char *direction = Direction[data[4] & 0x0f];

Would give you the string "SW".  Not too hard, so let's look at something a bit harder to decode, the temperature.  Temperature is held in bytes 4 and 5 of message type 8.  Looking above at the data, the one with byte 2 = 78 (78 & 0x0f = 8) bytes 4 and five have the values 08 1F.  With these we have to combine the lower order 4 bits of byte 4 with the 7 lowest bits of byte 5.  The code for this looks like this:

    int highpart = (data[4] & 0x0f) << 7;
    int lowpart = data[5] & 0x7f;
    float combined = highpart | lowpart;

Notice how we can get a floating point number out of this?  We have to use the conversion capabilities of the compiler to make it easy, but it works nicely.  So in this case we get 08 & 0f which give us 0x08 then we shift it left 7 bits to result in 0x400.  Then we take the 0x1F & 0x7F to get 0x1F. Combining them 0x400 | 0x1f gets us 0x41F which is decimal 1055.  That seems a bit high, even for Arizona, so subtract 400 and divide by 10 to give a float value of 65.5 degrees Fahrenheit.  Fortunately, the compiler will do this by:

float temperature = (combined - 400) / 10.0;

Are you starting to see how this works?  I decoded the rest of them and the code that comes later will give you all the details for the other sensor data.  Never mind that every single thing is carefully designed to be confusing.  The compass cardinals count counter-clockwise; all the high order bits of the sensor data are for parity, not values; The readings have an offset of 400; etc.  This stuff is just details that, once figured out and coded, aren't even relevant any more.  You only have to be confused once, and I did that for you already.

We have the ability to decode the sensors, but how did I get the data?  Remember in the last episode I opened and closed the USB device but didn't read it?  It's time for me to show you how to read it.  With a USB device, you don't read it, you 'transfer' it; this is what it looks like:

actual = libusb_control_transfer(weatherStation.handle,
                    LIBUSB_REQUEST_TYPE_CLASS |
                    LIBUSB_RECIPIENT_INTERFACE |
                    LIBUSB_ENDPOINT_IN,
                    //These bytes were stolen with a USB sniffer
                    0x01,0x0100+whichOne,0,
                    data, 50, 10000);

We had already found the device, opened it and reserved it use for ourselves, so this transfer is the last piece.  The bits above are documented with libusb and the numeric values were totally stolen by sniffing the USB interaction and doing some copy and paste.  The very last three items in the call are the pointer to where the data is going to wind up, the maximum length acceptable and a maximum wait time of 10 seconds (in milliseconds).  Yep, that's all there is to it getting the data.  I didn't want to hang up in a hard loop banging the USB port as fast as I could because it wouldn't leave the weather station any time to actually process the incoming data from the sensor head.  I set up a really simple timer to read the USB port periodically to get the report R1 and process it into usable data.

I'm not real proud of the way I timed it, but it works.  I put the code in to read both of the reports, R1 and R2 because I haven't given up on the barometric pressure; I totally will decode that someday.  Once I got the code to read the data, convert it and print it, I decided to fancy it up a bit.  I like working on this kind of stuff in python, so I made the 'c' code send its output to stderr and stdout.  Stderr gets the informational, and error messages while stdout gets a ... you guessed it ... JSON ... message that can be read into python and further processed.  The idea is that you pipe one process into another using the Linux model.  First the code that I'm currently running to get a feel for what I want to do in the future:

/*
    Experimentation with a USB interface to the Acu-Rite 5 in 1 Weatherstation
    specifically for the Raspberry Pi.  The code may work on other systems, but I 
    don't have one of those.  Contrary to other folk's thinking, I don't care if 
    this ever runs on a Windows PC or an Apple anything.
    
    I specifically used a model 2032 display with the sensor that I picked
    up at one of those warehouse stores.  The display has a usb plug on it and 
    I thought it might be possible to read the usb port and massage the data myself.
    
    This code represents the result of that effort.
    
    I gathered ideas from all over the web.  I use the latest (for this second)
    libusb and about a thousand examples of various things that other people have 
    done and posted about.  Frankly, I simply can't remember all of them, so please,
    don't be offended if you see your ideas somewhere in here and it isn't attributed.
    
    I simply lost track of where I found what.
    
    This module relies on libusb version 1.0.19 which, at this time, can only be
    compiled from source on the raspberry pi.
    
    Because there likely to be a version of libusb and the associated header file
    on a Pi, use the command line below to build it since the build of libusb-1.0.19
    places things in /usr/local
    
    cc -o weatherstation  weatherstation.c -L/usr/local/lib -lusb-1.0    
    use ldd weatherstation to check which libraries are linked in.
    If you still have trouble with compilation, remember that cc has a -v
    parameter that can help you unwind what is happening.
*/

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <time.h>
#include <unistd.h>
#include <sys/time.h>
#include <libusb-1.0/libusb.h>

// The vendor id and product number for the AcuRite 5 in 1 weather head.
#define VENDOR 0x24c0
#define PRODUCT 0x0003

// I store things about the weather device USB connection here.
struct {
    libusb_device *device;
    libusb_device_handle *handle;
    int verbose;
} weatherStation;
// These are the sensors the the 5 in 1 weather head provides
struct {
    float   windSpeed;
    time_t  wsTime;
    int     windDirection;
    time_t  wdTime;
    float   temperature;
    time_t  tTime;
    int     humidity;
    time_t  hTime;
    int     rainCounter;
    time_t  rcTime;
} weatherData;

// This is just a function prototype for the compiler
void closeUpAndLeave();

// I want to catch control-C and close down gracefully
void sig_handler(int signo)
{
  if (signo == SIGINT)
    fprintf(stderr,"Shutting down ...\n");
    closeUpAndLeave();
}

/*
This tiny thing simply takes the data and prints it so we can see it
*/
// Array to translate the integer direction provided to text
char *Direction[] = {
    "NNW",
    "NW",
    "WNW",
    "W",
    "WSW",
    "SW",
    "SSW",
    "S",
    "SSE",
    "SE",
    "ESE",
    "E",
    "ENE",
    "NE",
    "NNE",
    "N"  };

void showit(){

    fprintf(stdout, "{\"windSpeed\":{\"WS\":\"%.1f\",\"t\":\"%d\"},"
                    "\"windDirection\":{\"WD\":\"%s\",\"t\":\"%d\"},"
                    "\"temperature\":{\"T\":\"%.1f\",\"t\":\"%d\"},"
                    "\"humidity\":{\"H\":\"%d\",\"t\":\"%d\"},"
                    "\"rainCounter\":{\"RC\":\"%d\",\"t\":\"%d\"}}\n",
            weatherData.windSpeed, weatherData.wsTime,
            Direction[weatherData.windDirection],weatherData.wdTime,
            weatherData.temperature, weatherData.tTime,
            weatherData.humidity, weatherData.hTime,
            weatherData.rainCounter, weatherData.rcTime);
    fflush(stdout);
}
/* 
This code translates the data from the 5 in 1 sensors to something 
that can be used by a human.
*/
float getWindSpeed(char *data){
    int leftSide = (data[3] & 0x1f) << 3;
    int rightSide = data[4] & 0x70 >> 4;
    // Yes, I use mph, never got used to kilometers.
    return((float)(leftSide | rightSide) * 0.62);
}
int getWindDirection(char *data){
    return(data[4] & 0x0f);
}
float getTemp(char *data){
    // This item spans bytes, have to reconstruct it
    int leftSide = (data[4] & 0x0f) << 7;
    int rightSide = data[5] & 0x7f;
    float combined = leftSide | rightSide;
    return((combined - 400) / 10.0);
}
int getHumidity(char *data){
    int howWet = data[6] &0x7f;
    return(howWet);
}
int getRainCount(char *data){
    int count = data[6] &0x7f;
    return(count);
}
// Now that I have the data from the station, do something useful with it.
void decode(char *data, int length, int noisy){
    //int i;
    //for(i=0; i<length; i++){
    //    fprintf(stderr,"%0.2X ",data[i]);
    //}
    //fprintf(stderr,"\n"); */
    time_t seconds = time (NULL);
    //There are two varieties of data, both of them have wind speed
    // first variety of the data
    if ((data[2] & 0x0f) == 1){ // this has wind speed, direction and rainfall
        if(noisy)
            fprintf(stderr,"Wind Speed: %.1f ",getWindSpeed(data));
        weatherData.windSpeed = getWindSpeed(data);
        weatherData.wsTime = seconds;
        if(noisy)
            fprintf(stderr,"Wind Direction: %s ",Direction[getWindDirection(data)]);
        weatherData.wdTime = seconds;
        weatherData.windDirection = getWindDirection(data);
        if(noisy){
            fprintf(stderr,"Rain Counter: %d ",getRainCount(data));
            fprintf(stderr,"\n");
        }
        weatherData.rainCounter = getRainCount(data);
        weatherData.rcTime = seconds;
    }
    // this is the other variety
    if ((data[2] & 0x0f) == 8){ // this has wind speed, temp and relative humidity
        if(noisy)
            fprintf(stderr,"Wind Speed: %.1f ",getWindSpeed(data));
        weatherData.windSpeed = getWindSpeed(data);
        weatherData.wsTime = seconds;
        if(noisy)
            fprintf(stderr,"Temperature: %.1f ",getTemp(data));
        weatherData.temperature = getTemp(data);
        weatherData.tTime = seconds;
        if(noisy){
            fprintf(stderr,"Humidity: %d ", getHumidity(data));
            fprintf(stderr,"\n");
        }
        weatherData.humidity = getHumidity(data);
        weatherData.hTime = seconds;
    }
}
/*
This code is related to dealing with the USB device
*/
// This searches the USB bus tree to find the device
int findDevice(libusb_device **devs)
{
    libusb_device *dev;
    int err = 0, i = 0, j = 0;
    uint8_t path[8]; 
    
    while ((dev = devs[i++]) != NULL) {
        struct libusb_device_descriptor desc;
        int r = libusb_get_device_descriptor(dev, &desc);
        if (r < 0) {
            fprintf(stderr,"Couldn't get device descriptor, %s\n", libusb_strerror(err));
            return(1);
        }

        fprintf(stderr,"%04x:%04x (bus %d, device %d)",
            desc.idVendor, desc.idProduct,
            libusb_get_bus_number(dev), libusb_get_device_address(dev));

        //r = libusb_get_port_numbers(dev, path, sizeof(path));
        //if (r > 0) {
        //  fprintf(stderr," path: %d", path[0]);
        //  for (j = 1; j < r; j++)
        //      fprintf(stderr,".%d", path[j]);
        //}
        fprintf(stderr,"\n");
        
        if (desc.idVendor == VENDOR && desc.idProduct == PRODUCT){
            fprintf(stderr,"Found the one I want\n");
            weatherStation.device = dev;
            return (1);
        }
    }
    return(0);
}

// to handle testing and try to be clean about closing the USB device,
// I'll catch the signal and close off.
void closeUpAndLeave(){
    //OK, done with it, close off and let it go.
    fprintf(stderr,"Done with device, release and close it\n");
    int err = libusb_release_interface(weatherStation.handle, 0); //release the claimed interface
    if(err) {
        fprintf(stderr,"Couldn't release interface, %s\n", libusb_strerror(err));
        exit(1);
    }
    libusb_close(weatherStation.handle);
    libusb_exit(NULL);
    exit(0);
}

// This is where I read the USB device to get the latest data.
unsigned char data[50]; // where we want the data to go
int getit(int whichOne, int noisy){
    int actual; // how many bytes were actually read
    
    // The second parameter is bmRequestType and is a bitfield
    // See http://www.beyondlogic.org/usbnutshell/usb6.shtml
    // for the definitions of the various bits.  With libusb, the 
    // #defines for these are at:
    // http://libusb.sourceforge.net/api-1.0/group__misc.html#gga0b0933ae70744726cde11254c39fac91a20eca62c34d2d25be7e1776510184209
    actual = libusb_control_transfer(weatherStation.handle, 
                    LIBUSB_REQUEST_TYPE_CLASS | LIBUSB_RECIPIENT_INTERFACE | LIBUSB_ENDPOINT_IN,
                    //These bytes were stolen with a USB sniffer
                    0x01,0x0100+whichOne,0,
                    data, 50, 10000);
    if (actual < 0){
        fprintf(stderr,"Read didn't work for report %d, %s\n", whichOne, libusb_strerror(actual));
    }
    else {
        // If you want both of the reports that the station provides,
        // just allow for it.  Right this second, I've found every thing
        // I need in report 1.  When I look further at report 2, this will
        // change
        //fprintf(stderr,"R%d:%d:", whichOne, actual);
        //int i;
        //for(i=0; i<actual; i++){
        //    fprintf(stderr,"%0.2X ",data[i]);
        //}
        //fprintf(stderr,"\n");
        if (whichOne == 1)
            // The actual data starts after the first byte
            // The first byte is the report number returned by 
            // the usb read.
            decode(&data[1], actual-1, noisy);
    }
}
// I do several things here that aren't strictly necessary.  As I learned about
// libusb, I tried things and also used various techniques to learn about the 
// weatherstation's implementation.  I left a lot of it in here in case I needed to
// use it later.  Someone may find it useful to hack into some other device.
int main(int argc, char **argv)
{
    char *usage = {"usage: %s -u -n\n"};
    int libusbDebug = 0; //This will turn on the DEBUG for libusb
    int noisy = 0;       //This will print the packets as they come in
    libusb_device **devs;
    int r, err, c;
    ssize_t cnt;
    
    while ((c = getopt (argc, argv, "unh")) != -1)
        switch (c){
            case 'u':
                libusbDebug = 1;
                break;
            case 'n':
                noisy = 1;
                break;
            case 'h':
                fprintf(stderr, usage, argv[0]);
            case '?':
                exit(1);
            default:
                exit(1);
       }
    fprintf (stderr,"libusbDebug = %d, noisy = %d\n", libusbDebug, noisy);

    if (signal(SIGINT, sig_handler) == SIG_ERR)
        fprintf(stderr,"Couldn't set up signal handler\n"); 
    err = libusb_init(NULL);
    if (err < 0){
        fprintf(stderr,"Couldn't init usblib, %s\n", libusb_strerror(err));
        exit(1);
    }
    // This is where you can get debug output from libusb.
    // just set it to LIBUSB_LOG_LEVEL_DEBUG
    if (libusbDebug)
        libusb_set_debug(NULL, LIBUSB_LOG_LEVEL_DEBUG);
    else
        libusb_set_debug(NULL, LIBUSB_LOG_LEVEL_INFO);

    
    cnt = libusb_get_device_list(NULL, &devs);
    if (cnt < 0){
        fprintf(stderr,"Couldn't get device list, %s\n", libusb_strerror(err));
        exit(1);
    }
    // got get the device; the device handle is saved in weatherStation struct.
    if (!findDevice(devs)){
        fprintf(stderr,"Couldn't find the device\n");
        exit(1);
    }
    // Now I've found the weather station and can start to try stuff
    // So, I'll get the device descriptor
    struct libusb_device_descriptor deviceDesc;
    err = libusb_get_device_descriptor(weatherStation.device, &deviceDesc);
    if (err){
        fprintf(stderr,"Couldn't get device descriptor, %s\n", libusb_strerror(err));
        exit(1);
    }
    fprintf(stderr,"got the device descriptor back\n");
    
    // Open the device and save the handle in the weatherStation struct
    err = libusb_open(weatherStation.device, &weatherStation.handle);
    if (err){
        fprintf(stderr,"Open failed, %s\n", libusb_strerror(err));
        exit(1);
    }
    fprintf(stderr,"I was able to open it\n");
    // There's a bug in either the usb library, the linux driver or the 
    // device itself.  I suspect the usb driver, but don't know for sure.
    // If you plug and unplug the weather station a few times, it will stop
    // responding to reads.  It also exhibits some strange behaviour to 
    // getting the configuration.  I found out after a couple of days of
    // experimenting that doing a clear-halt on the device while before it
    // was opened it would clear the problem.  So, I have one here and a 
    // little further down after it has been opened.
    fprintf(stderr,"trying clear halt on endpoint %X ... ", 0x81);
    err = libusb_clear_halt(weatherStation.handle, 0x81);
    if (err){
        fprintf(stderr,"clear halt crapped, %s  Bug Detector\n", libusb_strerror(err));;
    }
    else {
        fprintf(stderr,"OK\n");
    }
    
    // Now that it's opened, I can free the list of all devices
    libusb_free_device_list(devs, 1); // Documentation says to get rid of the list
                                      // Once I have the device I need
    fprintf(stderr,"Released the device list\n");
    // Now I have to check to see if the kernal using udev has attached
    // a driver to the device.  If it has, it has to be detached so I can
    // use the device.
    if(libusb_kernel_driver_active(weatherStation.handle, 0) == 1) { //find out if kernel driver is attached
        fprintf(stderr,"Kernal driver active\n");
        if(libusb_detach_kernel_driver(weatherStation.handle, 0) == 0) //detach it
            fprintf(stderr,"Kernel Driver Detached!\n");
    }

    int activeConfig;
    err =libusb_get_configuration   (weatherStation.handle, &activeConfig);
    if (err){
        fprintf(stderr,"Can't get current active configuration, %s\n", libusb_strerror(err));;
        exit(1);
    }
    fprintf(stderr,"Currently active configuration is %d\n", activeConfig);

    if(activeConfig != 1){
        err = libusb_set_configuration  (weatherStation.handle, 1);
        if (err){
            fprintf(stderr,"Cannot set configuration, %s\n", libusb_strerror(err));;
            exit(1);
        }
    fprintf(stderr,"Just did the set configuration\n");
    }
    
    err = libusb_claim_interface(weatherStation.handle, 0); //claim interface 0 (the first) of device (mine had jsut 1)
    if(err) {
        fprintf(stderr,"Cannot claim interface, %s\n", libusb_strerror(err));
        exit(1);
    }
    fprintf(stderr,"Claimed Interface\n");
    fprintf(stderr,"Number of configurations: %d\n",deviceDesc.bNumConfigurations);
    struct libusb_config_descriptor *config;
    libusb_get_config_descriptor(weatherStation.device, 0, &config);
    fprintf(stderr,"Number of Interfaces: %d\n",(int)config->bNumInterfaces);
    // I know, the device only has one interface, but I wanted this code
    // to serve as a reference for some future hack into some other device,
    // so I put this loop to show the other interfaces that may
    // be there.  And, like most of this module, I stole the ideas from
    // somewhere, but I can't remember where (I guess it's google overload)
    const struct libusb_interface *inter;
    const struct libusb_interface_descriptor *interdesc;
    const struct libusb_endpoint_descriptor *epdesc;
    int i, j, k;
    for(i=0; i<(int)config->bNumInterfaces; i++) {
        inter = &config->interface[i];
        fprintf(stderr,"Number of alternate settings: %d\n", inter->num_altsetting);
        for(j=0; j < inter->num_altsetting; j++) {
            interdesc = &inter->altsetting[j];
            fprintf(stderr,"Interface Number: %d\n", (int)interdesc->bInterfaceNumber);
            fprintf(stderr,"Number of endpoints: %d\n", (int)interdesc->bNumEndpoints);
            for(k=0; k < (int)interdesc->bNumEndpoints; k++) {
                epdesc = &interdesc->endpoint[k];
                fprintf(stderr,"Descriptor Type: %d\n",(int)epdesc->bDescriptorType);
                fprintf(stderr,"Endpoint Address: 0x%0.2X\n",(int)epdesc->bEndpointAddress);
                // Below is how to tell which direction the 
                // endpoint is supposed to work.  It's the high order bit
                // in the endpoint address.  I guess they wanted to hide it.
                fprintf(stderr," Direction is ");
                if ((int)epdesc->bEndpointAddress & LIBUSB_ENDPOINT_IN != 0)
                    fprintf(stderr," In (device to host)");
                else
                    fprintf(stderr," Out (host to device)");
                fprintf(stderr,"\n");
            }
        }
    }
    fprintf(stderr,"trying clear halt on endpoint %X ... ", (int)epdesc->bEndpointAddress);
    err = libusb_clear_halt(weatherStation.handle, (int)epdesc->bEndpointAddress);
    if (err){
        fprintf(stderr,"clear halt crapped, %s  SHUCKS\n", libusb_strerror(err));;
        closeUpAndLeave();
    }
    else {
        fprintf(stderr,"OK\n");
    }

    // So, for the weather station we now know it has one endpoint and it is set to
    // send data to the host.  Now we can experiment with that.
    //
    // I don't want to just hang up and read the reports as fast as I can, so
    // I'll space them out a bit.  It's weather, and it doesn't change very fast.
    int tickcounter= 0;
    while(1){
        sleep(1);
        if(tickcounter++ % 10 == 0){
            getit(1, noisy);
        }
        if(tickcounter % 30 == 0){
            getit(2, noisy);
        }
        if (tickcounter % 15 == 0){
            showit();
        }
    }
}
Yes, I'll get it up on github so you can download it, I just want to finish this series before I put it there.  This compiles like the last example where we had to specify the library location:

cc -o weatherstation  weatherstation.c -L/usr/local/lib -lusb-1.0

I called it weatherstation, if you want a shorter name, feel free.  If you take a quick glance at the code you'll notice that it accepts parameters:

weatherstation -[unh]

-u will turn on libusb debug so you can see what is happening behind the scenes.
-n (noisy) will print the various decoded sensor data as it comes in
-h simple help for the options because I won't remember them

The output of the code looks like this combining without parameters:

libusbDebug = 0, noisy = 0
24c0:0003 (bus 1, device 6)
Found the one I want
got the device descriptor back
I was able to open it
trying clear halt on endpoint 81 ... OK
Released the device list
Currently active configuration is 1
Claimed Interface
Number of configurations: 1
Number of Interfaces: 1
Number of alternate settings: 1
Interface Number: 0
Number of endpoints: 1
Descriptor Type: 5
Endpoint Address: 0x81
 Direction is  In (device to host)
trying clear halt on endpoint 81 ... OK
{"windSpeed":{"WS":"0.0","t":"1417653478"},"windDirection":{"WD":"NNW","t":"0"},"temperature":{"T":"62.6","t":"1417653478"},"humidity":{"H":"92","t":"1417653478"},"rainCounter":{"RC":"0","t":"0"}}
{"windSpeed":{"WS":"3.7","t":"1417653488"},"windDirection":{"WD":"NNE","t":"1417653488"},"temperature":{"T":"62.6","t":"1417653478"},"humidity":{"H":"92","t":"1417653478"},"rainCounter":{"RC":"12","t":"1417653488"}}
^CShutting down ...
Done with device, release and close it

Notice that you get the informational stuff and errors will show up also, but if you use a redirect to get rid of the stderr output, it will look like this:

pi@deserthome2:~/src$ weatherstation 2>/dev/null
{"windSpeed":{"WS":"0.0","t":"1417653739"},"windDirection":{"WD":"SSE","t":"1417653739"},"temperature":{"T":"62.6","t":"1417653729"},"humidity":{"H":"92","t":"1417653729"},"rainCounter":{"RC":"12","t":"1417653739"}}
{"windSpeed":{"WS":"0.0","t":"1417653749"},"windDirection":{"WD":"SSE","t":"1417653749"},"temperature":{"T":"62.6","t":"1417653729"},"humidity":{"H":"92","t":"1417653729"},"rainCounter":{"RC":"12","t":"1417653749"}}
{"windSpeed":{"WS":"0.0","t":"1417653769"},"windDirection":{"WD":"SSE","t":"1417653749"},"temperature":{"T":"62.6","t":"1417653769"},"humidity":{"H":"92","t":"1417653769"},"rainCounter":{"RC":"12","t":"1417653749"}}

Notice that I used the line:

weatherstation 2>/dev/null

This will throw away any information or error, so you may want to send it to a file instead:

weatherstation 2>somefilename

But you can read up on that if you don't already have it down pat.  The JSON string has each device, its value and the time it was read.  So, suppose you pipe this into a little piece of python code that looks like this:

#!/usr/bin/python
import sys
import json
import time

buff = ''
while True:
    try:
        buff += sys.stdin.read(1)
        if buff.endswith('\n'):
            data = json.loads(buff[:-1])
            print "On", time.strftime("%A, %B, %d at %H:%M:%S",time.localtime(float(data["windSpeed"]["t"]))),
            print "the wind was blowing from the", data["windDirection"]["WD"],
            print "at\n", data["windSpeed"]["WS"], "mph,",
            print "and it is", data["temperature"]["T"], "degrees F Outside.",
            print "The humidity is", data["humidity"]["H"], "percent",
            print "and \nthe rain counter is", data["rainCounter"]["RC"],
            print
            sys.stdout.flush()
            buff = ''
    except KeyboardInterrupt:
        sys.stdout.flush()
        sys.exit()

Then you'll get a cute little update that looks like this:

On Wednesday, December, 03 at 17:51:01 the wind was blowing from the E at
0.0 mph, and it is 62.6 degrees F Outside. The humidity is 92 percent and
the rain counter is 12
On Wednesday, December, 03 at 17:51:21 the wind was blowing from the ENE at
2.5 mph, and it is 62.6 degrees F Outside. The humidity is 92 percent and
the rain counter is 12
On Wednesday, December, 03 at 17:51:31 the wind was blowing from the ENE at
2.5 mph, and it is 62.6 degrees F Outside. The humidity is 92 percent and
the rain counter is 12
On Wednesday, December, 03 at 17:51:51 the wind was blowing from the ENE at
0.0 mph, and it is 62.6 degrees F Outside. The humidity is 92 percent and
the rain counter is 12

I called the little python script 'readit.py', so the command looked like this:

 weatherstation 2>/dev/null | readit.py

Yes, I threw away the information and errors, trust me, when I run this for real, I'll be saving that stuff in /var/log/house and running logrotate on it.

There it is, the whole project to get the data, and do something with it.  Are you starting to understand why I cut it up into pieces?  You just had a primer on breaking into devices that they've been keeping secret, creating your very own USB driver for a new device in user space, using udev to create a custom file mount point specific to a vendor device, creating a piece of code that pipes its output so other pieces of code can work on it further, building a simple python script to format the data, did I forget anything?  Not bad for three settings.

Now, go forth and conquer.

Part 4 of this project is here <link>.

32 comments:

  1. Cool, thanks for the updates.

    I got it to compile, but to run I had to add /usr/local/lib to my LD_LIBRARY_PATH, otherwise I would get invalid symbol for libusb_strerror.

    But now that it is running, I'm not getting any data out of it. I've tried both modes 3 and 4 on the display, but no dice. The values in the JSON are always zero, and I'm getting quite a few Pipe Errors and Operation timed outs between each JSON line.

    I'll have to dig into it a bit more to see what's up.

    ReplyDelete
    Replies
    1. I had that happen when I got the /dev/weather link for the device messed up.

      Delete
    2. Oh, and it should be in USB mode 4. I'll have to edit that into the narrative above because I forgot it and one other thing as well. Thank for the reminder.

      Delete
    3. How did you know if the /dev/weather link got messed up? I ran the hexdump command and got data out of it.

      Delete
    4. That's how I knew, hexdump just hung. Turn on the debugging using the -u and see what libusb has to tell you.

      Delete
    5. Looks like libusb doesn't like how it is asking for data, there is an "unsupported control request" message. Here's what I get for the pipe error:

      [131.323160] [000008fc] libusb: debug [add_to_flying_list] arm timerfd for timeout in 10000ms (first in line)
      [131.323419] [000008fc] libusb: debug [libusb_handle_events_timeout_completed] doing our own event handling
      [131.323539] [000008fc] libusb: debug [handle_events] poll() 4 fds with timeout in 60000ms
      [131.323988] [000008fc] libusb: debug [handle_events] poll() returned 1
      [131.324096] [000008fc] libusb: debug [reap_for_handle] urb type=2 status=-32 transferred=0
      [131.324156] [000008fc] libusb: debug [handle_control_completion] handling completion status -32
      [131.324288] [000008fc] libusb: debug [handle_control_completion] unsupported control request
      [131.324350] [000008fc] libusb: debug [disarm_timerfd]
      [131.324411] [000008fc] libusb: debug [usbi_handle_transfer_completion] transfer 0x17cea40 has callback 0xb6ed0334
      [131.324463] [000008fc] libusb: debug [sync_transfer_cb] actual_length=0

      Delete
    6. Take a look over on the right at the 'about me' link, my email is in there. Drop me a note with the entire output and let me take a look at it.

      Delete
    7. After a few emails, Greg got it working. Congratulations Greg.

      Delete
  2. Color me impressed!

    When decoding the RF signal from my Oregon Scientific device, the temperature and humidity were fairly easy. Pick a nibble from the data packet to see if it tracked anything on the display. The barometric pressure, was not so easy.

    As in your case, I need the data to change to figure out what in a group of data meant what. When things change rapidly, it is much easier. I was thinking about putting the device in a container that I could force the pressure to change. Instead I googled one more time. Someone else had solved the problem. The barometer needed a number added to the values in the data stream.

    Maybe you could put the device in a plastic bag so you can force the pressure to change?



    ReplyDelete
    Replies
    1. I'm considering something like that. I have the air compressor and the vacuum pump, now I just need a airtight box that the console will fit into ....

      Delete
  3. I've decoded the rf on a fine offset wireless pws. On your R1, the C0 5 fits in with the ID of the transmitter I've seen. When I change the batteries, this code changes, so that you don't get your neighbours unit. Ie you sync to the code of your own tx.
    So what, you think, I don't use the ID. But I also get time information sent as B, so I need to know the difference. What I'm trying to say is, if you use the ID be aware it changes and just use the C, rather than C0.

    ReplyDelete
    Replies
    1. Thanks. I'd noticed that the C0 was something related to the id of the sensor package, but hadn't checked any further. Out here the chances of picking up another one are really slim.

      What I want to chase down from the sensor package is the battery level. It's in there somewhere, but I haven't found it yet.

      Delete
  4. Good stuff Dave! Do you know what they values of the rain counter are in?

    For example, my output is showing a value of 420, and my Accurite is showing .45 inches. Any idea? I need to convert the values back to inches for weather underground updates.

    ReplyDelete
    Replies
    1. Each count is 0.01 inch of rainfall. So, you've had 4.2 inches of rain since you put batteries in the weather head. The console is a different matter, the number there is since the last time it was reset. There's other factors also, yearly, daily, etc.

      If you want to record it yourself, you'll have to record a number and then display the difference between that and the current number.

      Delete
    2. Great! Thanks Dave!

      Any idea if the Rainfall Today is in the R2? If I could grab it from there, that would be way easier than having to store the last known and date value and doing a diff with additional date compare for that day's rainfall.

      Delete
    3. Not that I know of. The last two bytes are barometer and the second to last two bytes have been reported as inside temperature, but beyond that, I know nothing.

      Delete
  5. Awesome series, thanks for putting this together! The weather station is the only reason why I haven't completely dumped my windows laptop yet. I was able to follow your guide to access the real-time date being logged on my Ubuntu laptop so it's getting me there.

    Have you played around with downloading the stored data on the display, rather than the data that is streamed every 10 seconds or so? I'm not in a place where I can leave a computer connected to the display, so have to download this periodically, which I do every week or two.

    Any suggestions that you can provide would be appreciated. Thanks again!

    ReplyDelete
    Replies
    1. I did some looking to see if I could get the stored data off the station display, but didn't have any luck. I finally just gave up and took the data being sent by the weather head directly from the RF signal and logged it myself. After a couple of months, I really didn't need the console any more since a database query could get me anything I wanted from the data I had stored up.

      Delete
    2. Unfortunately I can't leave a computer connected to the console so this isn't an option. But ... my linux laptop does recognize the USB device properly now so I can access the console data through a windows VM by attaching the USB device to it. I can run the native acu-rite software there to periodically download the weather data from the console so am able to get it.

      Thanks again!

      Delete
    3. At some point, grab a cheap little linux machine and start holding the data yourself. The little machines are getting really cheap and you could just dedicate one to the weather station for a few bucks.

      That would free up your laptop from having to be used at all for this kind of thing. I used a Raspberry Pi for mine, because I had it handy, but if I were to do it again, I'd look into one of the recently announced devices instead.

      However, the beauty is that the longer you wait, the better the solution you can come up with. There's new possibilities being announced almost daily.

      Delete
  6. Loved your comment about not caring what the Apple and Windows people think! Most excellent!

    ReplyDelete
  7. Hi Dave,
    so I still do not know exactly what I am doing. I have been trying to follow these steps here. I am now on part 3. When I run a.out, I am not getting the same thing that you got. I might be missing something. I put the device on usb mode 4. Then I ran the file weatherstation.c from the root. And I ran a.out from the root. and this is what I got. Is there something I need to type in differently?

    root@elac-desktop:~/Desktop# cc -o weatherstation weatherstation.c -L/usr/local/lib -lusb-1.0
    weatherstation.c: In function ‘showit’:
    weatherstation.c:100:21: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘time_t {aka long int}’ [-Wformat=]
    fprintf(stdout, "{\"windSpeed\":{\"WS\":\"%.1f\",\"t\":\"%d\"},"
    ^
    weatherstation.c:100:21: warning: format ‘%d’ expects argument of type ‘int’, but argument 6 has type ‘time_t {aka long int}’ [-Wformat=]
    weatherstation.c:100:21: warning: format ‘%d’ expects argument of type ‘int’, but argument 8 has type ‘time_t {aka long int}’ [-Wformat=]
    weatherstation.c:100:21: warning: format ‘%d’ expects argument of type ‘int’, but argument 10 has type ‘time_t {aka long int}’ [-Wformat=]
    weatherstation.c:100:21: warning: format ‘%d’ expects argument of type ‘int’, but argument 12 has type ‘time_t {aka long int}’ [-Wformat=]
    weatherstation.c: In function ‘main’:
    weatherstation.c:422:32: warning: '0' flag ignored with precision and ‘%X’ gnu_printf format [-Wformat=]
    fprintf(stderr,"Endpoint Address: 0x%0.2X\n",(int)epdesc->bEndp
    ^
    root@elac-desktop:~/Desktop# ./a.out
    24c0:0003 (bus 1, device 6)
    Found the one I want
    got the device descriptor back
    I was able to open it
    Released the device list
    Currently active configuration is 1
    Just did the set configuration
    Claimed Interface
    Number of configurations: 1
    Number of Interfaces: 1
    Number of alternate settings: 1
    Interface Number: 0
    Number of endpoints: 1
    Descriptor Type: 5
    Endpoint Address: 0x81
    Direction is In (device to host)
    Done with device, release and close it
    root@elac-desktop:~/Desktop#

    ReplyDelete
    Replies
    1. Actually, it looks OK to me. Did you let it run long enough to try and read the device?

      Delete
    2. The program is not outputing this data like yours:

      {"windSpeed":{"WS":"0.0","t":"1417653478"},"windDirection":{"WD":"NNW","t":"0"},"temperature":{"T":"62.6","t":"1417653478"},"humidity":{"H":"92","t":"1417653478"},"rainCounter":{"RC":"0","t":"0"}}
      {"windSpeed":{"WS":"3.7","t":"1417653488"},"windDirection":{"WD":"NNE","t":"1417653488"},"temperature":{"T":"62.6","t":"1417653478"},"humidity":{"H":"92","t":"1417653478"},"rainCounter":{"RC":"12","t":"1417653488"}}

      After reading the endpoint access, it skips immediately to this line of output:

      Done with device, release and close it

      Delete
  8. There's two reasons for getting that message, look at the code and you'll see them. One is that you got a signal and the other is that the clear-halt failed. If you hit control c, there's an additional message, so that's not it. Look in the signal handler and put a message to tell you what's going on. I kind of messed up the indentation in the signal handler, so do let that confuse you.

    ReplyDelete
  9. I followed Dave's instructions up to this point, as I didn't want to mess with the RF stuff. His instructions didn't always work with my newer RPi, so I had to do some research myself to make them work. I also added a way to handle the data and store it in .csv files.

    I put all of my code and detailed instructions as to how I implemented it into a GitHub repo, with the hopes that it might help someone else save some time: https://github.com/jonathan1440/Get-Weather-Data

    ReplyDelete
  10. Hi Dave,

    So a few years ago my son and I followed your instructions and were successful in duplicating your steps and getting our weather station connected to the RPI. We learned a lot. Now (that I've forgotten most of what I learned) I'm reviewing the USB section of your project in the hopes that I can connect/talk to a "RS485 to USB" converter with the RPI (HXSP-2108GII). My question is: would not the same general procedure you used with the weather station's USB port work with other USB devices? Basically the RS485 port I need to interface with spits out short lines of text no more than 40 characters, with out any flow control, at the rate of about one line a minute. I need to monitor this port (via the USB connection and a RPI, and parse the output of each line once it appears. The first step, much like in your project, is going to be getting the data from the USB device. Any suggestions/ points in the right direction are appreciated.

    ReplyDelete
    Replies
    1. OK, here's the trick. Look on the web for something that does something similar to what you need. Then find their repository in GitHub and look at their techniques. Often you'll find exactly what you need that way.

      Delete
    2. Good idea. I'll be I could find code to talk to an rs232 to usb converter for the RPI, which would be really similar!

      Delete
  11. Here's an independent version that logs all the data to a comma delineated file suitable for opening, viewing, charting with excel / libre-office calc. It ports the python from Matthew Walls Accurite.py for weewx to C++ and libusb-1.0. As such, it isolates wind speed, wind direction, rainfall, outTemp, outHumidity, & pressure. All it does is read and log.
    http://comroestudios.com/AccuritePi/main.cpp

    ReplyDelete
    Replies
    1. Thanks, that might just be what some people need. Wish that would have been around when I looked into this. I could have save a lot of time.

      Delete
  12. Hey Dave,

    I have found this really useful and interesting. I look forward to reading more of your posts!

    I noticed some weird readings for wind speed and starting poking at it a little bit. On my machine, putting parenthesis in the assignment for the "rightSide" int produced different values. Here's the original code:

    float getWindSpeed(char *data){
    int leftSide = (data[3] & 0x1f) << 3;
    int rightSide = data[4] & 0x70 >> 4;
    // Yes, I use mph, never got used to kilometers.
    return((float)(leftSide | rightSide) * 0.62);
    }

    I changed the line to: "int rightSide = (data[4] & 0x70) >> 4;
    "
    Here's some sample output showing two floats, the first without the parenthesis and the second with. I did some testing at zero and a few fan speeds. The second value was closer to the weatherstation's console display.

    C9 F7 78 00 06 70 59 03 FF
    00 06
    00 00
    00 00
    3.72
    0.00
    {"windSpeed":{"v":"0.0","t":"1621059986"},"windDir":{"v":"S","t":"1621059976"},"temp":{"v":"48.0","t":"1621059986"},"humidity":{"v":"89","t":"1621059986"},"rainCounter":{"v":"64","t":"1621059976"}}

    ReplyDelete