Thursday, October 10, 2013

GroveStreams Another Data Service

In my continuing search for sites to put my house data on I encountered Grovestreams.com on the Arduino forum.  Well, since I have a cool setup to try these things out I decided to give it a try.  At first I got totally lost in the jargon inherent to cloud services.  These things talk about streams of data, component architecture, registering a service; you know, the kind of language that simply puts people that make things to sleep.  I persevered.

Once I started to understand what they meant by messing up example after example, I was ready to try some code.  I grabbed their python example and updated it to save one of my devices.  Since I'm a power usage freak, I chose my real time measurement of power usage.  It worked.  That was actually really cool, so I added outside temperature.  It worked too.  Then I went a little nuts and added the rest of my house data to it.

Then I was on a roll.  I modified the code again to use the techniques I used on the other cloud services.  I put it under the APSscheduler to do one minute updates, added it to the init table so it would restart if it had trouble and let it go.  It's running right now keeping track of the data I collect around the house for me.  So, what about a chart to look at?  Well, they have a dashboard like some of the other services and I created one to hold some charts; these charts are embeddable:

Pretty slick.  We've seen this before, but it's nice to see a new service like this implement it first instead of years later. They also have other 'widgets', gauges and things. What impressed me about this site is the versatility.  They have capabilities that I won't have time to try out. They can derive data from other data and present it.  It should be possible to build a chart based on my power usage and the billing charges multiplied together and actually show graphically how much money it's costing me to keep cool in the summer.  That's actually a little depressing when you think about it.

They even have alerts that can send SMS and email to you.  I created an alert to tell me when my house power usage goes over 10kWh.  It was annoying when it came in the first time and even more annoying when it came in the second time.  I actually have real time notification that I'm wasting money that comes in on my phone.  I could easily expand this to alert me when the garage door is open after sundown.  That would help a lot keeping the pack rats from stealing my tools and burying them in a mound outside.

Here's the code I'm actually running to update my house data on their site:

The python Sketch Script
#!/usr/bin/python
import time
from time import sleep
from datetime import datetime
import sys
from apscheduler.scheduler import Scheduler
import logging
from simplejson import encoder as jsonEncoder
import httplib
import StringIO
import gzip
import sqlite3
import pprint

# I keep all current values in a data base
DATABASE='/home/pi/database/desert-home'

# These are the secret numbers I use to get into
# the grovestreams service
org = "putyourownsecretthinghere";
api_key = "samewithyoursecretapikey";

#If you want lots of messages and debug information
# set this to true
DEBUG = False

# the site accepts compression, might as well use it
def compressBuf(buf):
zbuf = StringIO.StringIO()
zfile = gzip.GzipFile(mode = 'wb', fileobj = zbuf, compresslevel = 9)
zfile.write(buf)
zfile.close()
return zbuf.getvalue()

def updateGrovestreams():
# This is VERY different from their examples.  I named
# my streams with something I could understand and read
# Probably not the best way to do it, but it works really
# well for me.
component_id = "desert-home-id"
rpowerStream_id = "power_usage"
otempStream_id = "outside_temp"
apowerStream_id = "apparent_power"
voltageStream_id = "voltage"
currentStream_id = "current"
pfactorStream_id = "power_factor"
itempStream_id = "inside_temp"
ptempStream_id = "pool_temp"
pmotorStream_id = "pool_motor"

# This object really helps when displaying
# the rather complex data below
pp = pprint.PrettyPrinter(depth=10)

#get the millis since epoch
# in unix the epoch began back in 1970, look it up
now = datetime.now()
nowEpoch = int(time.mktime(now.timetuple())) * 1000

#assemble feed and convert it to a JSON string
feed = {};
feed['feed'] = {}
feed['feed']['component'] = []
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()

comp = {}
comp['stream'] = []
comp['componentId'] = component_id
feed['feed']['component'].append(comp)
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()

# Now I'm going to fill in the stream values, open database
# I took a brute force approach to building the dictionary that
# is converted into JSON.  I could have been much more elegant
# in building it, but the folks just starting out would have
# had a tough time understanding it
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
# So, you make a stream to stuff things into.  It's actually
# a python dictionary that we'll pass to a JSON encoder a ways
# down into the code.  I'll be adding entries to this as I pull
# items out of the database
stream1 = {}  
stream1['streamId'] = rpowerStream_id
stream1['time'] = [] 
stream1['data'] = []
comp['stream'].append(stream1)

current_value = c.execute("select rpower from power").fetchone()[0]  
stream1['time'].append(nowEpoch)
stream1['data'].append(float(current_value))
# this is a cool way to debug this kind of thing.
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()
# notice how I get and item out of the database
# and add it to the dictionary.  I'll do this
# several times
stream2 = {}
stream2['streamId'] = otempStream_id
stream2['time'] = []
stream2['data'] = []
comp['stream'].append(stream2)
current_value = c.execute(
"select currenttemp from xbeetemp").fetchone()[0]
stream2['time'].append(nowEpoch)
stream2['data'].append(float(current_value))

stream3 = {}
stream3['streamId'] = apowerStream_id
stream3['time'] = []
stream3['data'] = []
comp['stream'].append(stream3)
current_value = c.execute(
"select apower from power").fetchone()[0]
stream3['time'].append(nowEpoch)
stream3['data'].append(float(current_value))

stream4 = {}
stream4['streamId'] = voltageStream_id
stream4['time'] = []
stream4['data'] = []
comp['stream'].append(stream4)
current_value = c.execute(
"select voltage from power").fetchone()[0]
stream4['time'].append(nowEpoch)
stream4['data'].append(float(current_value))

stream5 = {}
stream5['streamId'] = currentStream_id
stream5['time'] = []
stream5['data'] = []
comp['stream'].append(stream5)
current_value = c.execute(
"select current from power").fetchone()[0]
stream5['time'].append(nowEpoch)
stream5['data'].append(float(current_value))

stream6 = {}
stream6['streamId'] = pfactorStream_id
stream6['time'] = []
stream6['data'] = []
comp['stream'].append(stream6)
current_value = c.execute(
"select pfactor from power").fetchone()[0]
stream6['time'].append(nowEpoch)
stream6['data'].append(float(current_value))

stream7 = {}
stream7['streamId'] = itempStream_id
stream7['time'] = []
stream7['data'] = []
comp['stream'].append(stream7)
current_value = c.execute(
"select avg(\"temp-reading\") from thermostats").fetchone()[0]
stream7['time'].append(nowEpoch)
stream7['data'].append(float(current_value))

stream8 = {}
stream8['streamId'] = ptempStream_id
stream8['time'] = []
stream8['data'] = []
comp['stream'].append(stream8)
current_value = c.execute(
"select ptemp from pool").fetchone()[0]
stream8['time'].append(nowEpoch)
stream8['data'].append(float(current_value))

stream9 = {}
stream9['streamId'] = pmotorStream_id
stream9['time'] = []
stream9['data'] = []
comp['stream'].append(stream9)
tmp = c.execute("select motor from pool").fetchone()[0];
if (tmp == 'High'): # a little special handling for the pool motor
motor = 2
elif (tmp == 'Low'):
motor = 1
else:
motor = 0
stream9['time'].append(nowEpoch)
stream9['data'].append(int(motor))

# all the values are filled in, close the database
dbconn.close() # close the data base
# This will print the entire database I just constructed
# so you can see what is going on
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()
# exit() # I put this in for debugging.  It exits before
# the JSON string is constructed and sent off to grovestreams
# Of course you want to keep it commented until needed
#
# And this is where the JSON string is built
encoder = jsonEncoder.JSONEncoder()
json = encoder.encode(feed);
# and this will print it so you can see what is happening
if DEBUG:
print json # for debugging
print
sys.stdout.flush()

#Upload the feed
try:
print "Updating GroveStream ", time.strftime("%A, %B %d at %H:%M:%S")
sys.stdout.flush()
conn = httplib.HTTPConnection('www.grovestreams.com')

url = '/api/feed?&org=%s&api_key=%s' % (org, api_key)

compress = True
if compress:
body = compressBuf(json)
headers = {"Content-type": "application/json", "Content-Encoding" : "gzip"}
else:
body = json
headers = {"Content-type": "application/json", "charset":"UTF-8"}

conn.request("PUT", url, body, headers)

response = conn.getresponse()
status = response.status

if status != 200 and status != 201:
try:
if (response.reason != None):
print('reason: ' + response.reason + ' body: ' + response.read())
sys.stdout.flush()
else:
print('body: ' + response.read())
sys.stdout.flush()
except Exception:
print('HTTP Fail Status: %d' % (status) )
sys.stdout.flush()

except Exception as e:
print('HTTP Send Error: ' + str(e))
sys.stdout.flush()
   
finally:
if conn != None:
conn.close()

# I just discovered the statement below.
# someday I'll have go figure out what it really does.
if __name__ == '__main__':
print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
sys.stdout.flush()
logging.basicConfig()

#------------------Stuff I schedule to happen -----
scheditem = Scheduler()
scheditem.start()
# every minute update the data store on Xively
scheditem.add_interval_job(updateGrovestreams, seconds=60)
#
# A couple of people asked me why I put this statement in
# since I have it scheduled to happen every 60 seconds already
# Well, when you're debugging something it sucks to have to
# wait 60 seconds to see if you fixed it, so I do it 
# first, then let the scheduler take care of the rest.
#
updateGrovestreams()
while True:
time.sleep(20) #This doesn't matter much since it is schedule driven
 

Yes, in my usual form, it's got way too many comments.  It's also very inelegant; I'm not a big fan of strange statements that run really well but take an hour to understand so I just brute forced the URL and body creation of the request sent.  Notice that there isn't a library with obscure calls and methods and stuff.  This is all python code that I simply stole from them and added my particular items to.  Makes it nice that I don't have to learn how to use some special library.  I also put in a variable to control debugging so you can see what is happening.  It gets a little complex at a couple of points.

I didn't stop there.  I took their arduino example and modified it to update some items as well.  It worked first try.  The example had a little trouble since Grovestreams used Strings as a way of making the code more easily read and it started losing memory.  The poor little arduino has so little memory that I knew it would die over time, so I sent them a note describing the problem.  Guess what?  They responded!

Yes, in this 21st century world of minimal text messages, they actually answered me and came up with a new example that works better.  Unfortunately, I didn't save that one as an illustration for you, you'll just have to go to their site and get it yourself.  However, I did save the first version I tried:

The Arduino Sketch
/*

 Arduino GroveStreams Stream Feed via Ethernet

 The GroveStreams client sketch is designed for the Arduino and Ethernet.
 A full "how to" guide for this sketh can be found at https://www.grovestreams.com/developers/getting_started_arduino_temp.html
 This sketch updates several stream feeds with an analog input reading,
 from a temperature probe, via the GroveStreams API: https://www.grovestreams.com/developers/apibatchfeed.html
 The Arduino uses DHCP and DNS for a simpler network setup.
 The sketch also includes a Watchdog / Reset function to make sure the
 Arduino stays connected and/or regains connectivity after a network outage.
 Use the Serial Monitor on the Arduino IDE to see verbose network feedback
 and GroveStreams connectivity status.

 GroveStreams Setup:

 * Sign Up for Free User Account - https://www.grovestreams.com
 * Create a GroveStreams organization and select the Arduino blueprint
 * Enter a unique MAC Address for this network in this sketch under "Local Network Settings" 
 *    (Newer shields have the mac address on a sticker on the shield. Use that.)
 *    (A MAC address can also be generated within a GroveStreams organization: tools - Generate MAC Address)
 * Enter the GroveStreams org uid under "GroveStreams Settings" in this sketch 
 *    (Can be retrieved from a GroveStreams organization: tools - View Organization UID)
 * Enter the GroveStreams api key under "GroveStreams Settings" in this sketch  
 *    (Can be retrieved from a GroveStreams organization: click the Api Keys toolbar button, 
 *     select your Api Key, and click View Secret Key)

 Arduino Requirements:

 * Arduino with Ethernet Shield or Arduino Ethernet
 * Arduino 1.0 IDE

 Network Requirements:

 * Ethernet port on Router    
 * DHCP enabled on Router
 * Unique MAC Address for Arduino

 Additional Credits:
 Example sketches from Arduino team, Ethernet by David A. Mellis

 */

#include <SPI.h>
#include <Ethernet.h>
#include <MemoryFree.h>


// Local Network Settings
byte mac[] = {0x90, 0xA2, 0xDA, 0x00, 0x33, 0x33};        // Change this!!! Must be unique on local network. 
                                               // Look for a sticker on the back of your Ethernet shield.

// GroveStreams Settings
String gsApiKey = "You\'re going to need your own number here";   //Change This!!!
String gsOrg = "another number you need to put in";      //Change This!!!
String gsComponentName = "Temperature";                     //Optionally change. Set this to give your component a name when it initially registers.

char gsDomain[] = "grovestreams.com";   //Don't change. The Grove Streams domain. 
String gsComponentTemplateId = "temp";  //Don't change. Tells GS what template to use when the feed initially arrives and a new component needs to be created.
                                        // The blueprint is expecting "temp".

//GroveStreams Stream IDs. Stream IDs tell GroveStreams which component streams the values will be assigned to.
//Don't change these unless you edit your GroveStreams component definition and change the stream ID to match this.
String gsStreamId1 = "s1";   //Temp C - Random Stream. 
String gsStreamId2 = "s2";   //Temp F - Random Stream. Don't change.
String gsStreamId3 = "s3";   //Temp C - Interval Stream (20 second intervals). Don't change.
String gsStreamId4 = "s4";   //Temp F - Interval Stream (20 second Intervals). Don't change.

const int updateFrequency = 20 * 1000; // GroveStreams update frequency in milliseconds (the GS blueprint is expecting 20s)
const int temperaturePin = 0;          // Then Temperature pin number.    

// Variable Setup
String myIPAddress;  //Set below from DHCP. Needed by GroveStreams to verify that a device is not uploading more than once every 10s.
String myMac;        //Set below from the above mac variable. The readable Mac is used by GS to determine which component the feeds are uploading into.

long lastConnectionTime = 0;    //Don't change. Used to determine if the Ethernet needs to be reconnected. 
boolean lastConnected = false;  //Don't change. 
int failedCounter = 0;          //Don't change. 

// Initialize Arduino Ethernet Client
EthernetClient client;


void setup()
{
  // Start Serial for debugging on the Serial Monitor
  Serial.begin(9600);

  // Start Ethernet on Arduino
  startEthernet();
  randomSeed(analogRead(1));
}

void loop()
{

  // Print Update Response to Serial Monitor
  if (client.available())
  {
    char c = client.read();
    Serial.print(c);
  }

  // Disconnect from GroveStreams
  if (!client.connected() && lastConnected)
  {
    Serial.println("...disconnected");
    Serial.println();
    showMem();

    client.stop();
  }

  // Update sensor data to GroveStreams
  if(!client.connected() && (millis() - lastConnectionTime > updateFrequency))
  {
    String tempC = getTemperatureC();
    Serial.print (tempC);
    String tempF = getTemperatureF();
    Serial.print (tempF);
    updateGroveStreams(tempC, tempF);
  }

  // Check if Arduino Ethernet needs to be restarted
  if (failedCounter > 3 ) {
    
    //Too many failures. Restart Ethernet.
    startEthernet();
  }

  lastConnected = client.connected();
}

void updateGroveStreams(String tempC, String tempF)
{
  Serial.println("\nin update routine");
  if (client.connect(gsDomain, 80))
  {         

    //Assemble the url used to pass the temperature readings to GroveStreams
    // The Arduino String class contains many memory bugs and char arrays should be used instead, but
    // to make this example simple to understand we have chosen to use the String class. 
    // No none memory issues have been seen with this example to date.
    //We are passing temperature readings into two types of GroveStreams streams, Random and Interval streams.
    String url = "PUT /api/feed?&compTmplId=" + gsComponentTemplateId + "&compId=" + myMac + "&compName=" + gsComponentName;
    url += "&org=" + gsOrg + "&api_key=" + gsApiKey;

    url += "&" + gsStreamId1 + "=" + tempC;  //Temp C - Random Stream
    url += "&" + gsStreamId2 + "=" + tempF;  //Temp F - Random Stream
    url += "&" + gsStreamId3 + "=" + tempC;  //Temp C - Interval Stream (20 second intervals)
    url += "&" + gsStreamId4 + "=" + tempF;  //Temp F - Interval Stream (20 second intervals)

    url += " HTTP/1.1";
    Serial.print(url);
    client.println(url);  //Send the url with temp readings in one println(..) to decrease the chance of dropped packets
    client.println("Host: " + String(gsDomain));
    client.println("Connection: close");
    client.println("X-Forwarded-For: "+ myIPAddress); //Include this line if you have more than one device uploading behind 
                                                      // your outward facing router (avoids the GS 10 second upload rule)
    client.println("Content-Type: application/json");
    client.println();

    if (client.available())
    {
      //Read the response and display in the the console
      char c = client.read();
      Serial.print(c);
    }

    lastConnectionTime = millis();

    if (client.connected())
    {     
      failedCounter = 0;
    }
    else
    {
      //Connection failed. Increase failed counter
      failedCounter++;

      Serial.println("Connection to GroveStreams failed ("+String(failedCounter, DEC)+")");   
      Serial.println();
    }

  }
  else
  {
     //Connection failed. Increase failed counter
    failedCounter++;

    Serial.println("Connection to GroveStreams Failed ("+String(failedCounter, DEC)+")");   
    Serial.println();

    lastConnectionTime = millis(); 
  }
}

void startEthernet()
{
  //Start or restart the Ethernet connection.
  client.stop();

  Serial.println("Connecting Arduino to network...");
  Serial.println();  

  //Wait for the connection to finish stopping
  delay(1000);

  //Connect to the network and obtain an IP address using DHCP
  if (Ethernet.begin(mac) == 0)
  {
    Serial.println("DHCP Failed, reset Arduino to try again");
    Serial.println();
  }
  else
  {

    Serial.println("Arduino connected to network using DHCP");
    Serial.println();

    //Wait to ensure the connection finished
    delay(1000);

    //Set the mac and ip variables so that they can be used during sensor uploads later
    myMac =getMacReadable();
    Serial.println("MAC: " + myMac);

    myIPAddress = getIpReadable(Ethernet.localIP());
    Serial.println("IP address: " + myIPAddress);
  }

}

String getMacReadable() 
{
  //Convert the mac address to a readable string
  char macstr[20];
  snprintf(macstr, 100, "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
  return String(macstr);
}

String getIpReadable(IPAddress p)
{
  //Convert the ip address to a readable string
  String ip;
  for (int i =0; i < 3; i++)
  {
    ip += String(p[i], DEC);
    ip += ".";
  }
  ip +=String(p[3], DEC);
  return ip;
}

String getTemperatureF() 
{
  //Get the temperature analog reading and convert it to a string
  float voltage, degreesC, degreesF; 

  //voltage = (analogRead(temperaturePin) * 0.004882814);
  //degreesF = degreesC * (9.0/5.0) + 32.0;
  degreesF = float(random(70, 120));

  char temp[20] = {0}; //Initialize buffer to nulls
  dtostrf(degreesF, 12, 3, temp); //Convert float to string

  String stemp = temp;
  stemp.trim();  //Trim off head and tail spaces
  return stemp; 
}

String getTemperatureC() 
{
  //Get the temperature analog reading and convert it to a string
  float voltage, degreesC, degreesF; 

  //voltage = (analogRead(temperaturePin) * 0.004882814);
  //degreesC = (voltage - 0.5) * 100.0;
  degreesC = float(random(20, 40));

  char temp[20] = {0}; //Initialize buffer to nulls
  dtostrf(degreesC, 12, 3, temp); //Convert float to string

  String stemp = temp;
  stemp.trim();  //Trim off head and tail spaces
  return stemp; 
}

void showMem(){
  char Dbuf [100];
  
  strcpy_P(Dbuf,PSTR("Mem = "));
  Serial.print(Dbuf);
  Serial.println(freeMemory());
}



This worked really well, but like I said above, it will die over time because the Streams library will run the arduino out of memory.  I left the code in it that I used to check memory usage and also the random number generation I used to test it.

Notice that the call to send the data is different?  Of course you did.  The call is part of their batch update API that is much simpler to use than the JSON call I used in the python example.  This makes getting the data up there much easier for the arduino folks.  They're even working on improvements to make the API more easily understood.  The python folk seem to like infinitely complex data structures, so I'm leaving that example as it is.

They even have a full blown tutorial on hooking the arduino up to their service.  Unlike a lot of the tutorials out there, this one can actually be read.  You don't even have to click on page after page like the darn instructables we've all learned to love.

I really like this site.  Like I said, I haven't gotten past the very surface of its capabilities, and probably never will, but it was nice to see a site where the folks running it actually care if people can use it.

23 comments:

  1. FYI, python does not have sketches. It has scripts and programs.

    Arduino doesn't really have sketches either (unless you're drawing a picture of one with a pencil), but it certainly has programs and firmware.

    ReplyDelete
  2. You're right about python, that's what copy and paste does for one. You're wrong about Arduinos, take a look at the IDE, it has a 'Sketchbook' for storing the 'sketches'. Remember they intended the devices for schools, artists, and normal people. I'll fix the python title line someday.

    Picky, picky, picky.

    ReplyDelete
  3. Dave:

    Great site!! I am a Python fan and have been using Misterhouse for years but now I want to upgrade to something that doesn't use Perl!!
    Are you using Python 3 or is your stuff coded in Python 2?
    Also, where can I download your XBee library. I want to run you House Monitor code but it calls for a couple of imports.
    Finally, correct me if I am wrong but do you not run your SQLite3 on your RaspberryPI?
    Keep up the great work. I will be following closely.

    Glenn.

    ReplyDelete
    Replies
    1. I'm using python 2.7. I did that intentionally since some of the libraries I want to use aren't ported to 3 yet (if ever). And, Yes, I run SQLite3; I chose that because it doesn't have a separate process involved that I have to monitor and take care of. It is close enough to others that, when I need to, I can port it to a bigger data base manager. For now, it works great.

      I installed the XBee library using pip. So, wherever it got it from is where it came from. I just did a 'sudo pip install xbee' and let it happen.

      Delete
  4. Dave:

    Couple of other quick comments. The site you linked for your current transformers apparently no longer carries them. Unfortunate, as I am looking to add this to my system.
    I have a bunch of X10 modules that I have been using for 30 plus years plus a couple wired in to control exterior lights. I will be wanting to use these while I accumulate some XBee modules to replace them for controls etc. Any thoughts on these?
    Thanks again.
    Glenn.

    ReplyDelete
    Replies
    1. Regarding the current transformer, go to http://imall.iteadstudio.com/ and then type 'current transformer' into the search box up at the top. You'll get back several of them and the ones I'm actually using are in the list. There are others if you don't need the big one I got.

      I don't use XBee to control the lights around the house because it's too darn much trouble to hook a relay to the little things, find a power supply, etc. Then even if I did, I'd have to figure out a way to get the entire mess to fit in a wall switch. I'm working on integrating Wemo (google it) switches into the house. I have three of them right now controlling the lights on the front of my house. They are working pretty well. I'll be posting more about that on my blog in the near future.

      The problem is that no one has really cool controls for the switches at anywhere near the price the X10 stuff comes in at. But around my house, X10 is horribly unreliable. Yes, I looked at insteon, Z-Wave, etc and they all cost so much I'm reluctant to invest money in something that may not be much better. I don't believe the hype out there about how reliable the various protocols are.

      So, I'm putting motion controlled switches in closets, pantry, attic, laundry room, etc. Places that just make sense having motion control. The other places I'm going to have to figure out a solution that makes sense. Maybe capture the RF signal from an X10 switch and translate it to control the Wemo, or something like that.

      Delete
  5. My bad. I had the wrong url. Now I have it. I just ordered my 200A current transformers.
    By the way, I've had a couple of SBC68EC's from Modtronix.com running for a few years. Great little devices.
    As to lights, I've been using the Leviton 6375's on X10. I also put in an X10 bridge across my 220v ac feed to the main breaker of the house. Everything works excellent since then. i control the exterior lights on my barn which is a couple hundred feet away, plus my house's exterior christmas light plugs in the eaves and out to the trees and fences. I have absolutely no complaints with this, it runs great. I'm interested in your Wemo switches however, as I would like feedback, short of adding current transformers after the Levitons.
    Can you provide any more detail on your vehicle battery monitoring? I have two batteries in my diesel truck I would like to monitor.
    One final question, it appears iMall has brought out an Iteaduino Lite for $8.00. You get three for $20.00. What are your thoughts on these? I'd be most interested in your comments.
    Thanks again Dave. You have been most helpful.

    ReplyDelete
    Replies
    1. Congratulations on finding the right devices. Now you just have to wait for the shipping to start having fun. For a long time I ran several X10 devices; then I got a super pump for the swimming pool and it produced so much noise that every X10 device went flaky. I already had the coupler between the AC phases, so I tried the high current noise filter, that didn't work, so I added another. That didn't work either. So, I put up with flaky until the wemo device became available.

      I'm not totally in love with the Wemo stuff, but I'm also patient, so time will tell the tale on that. At least I can program a controller myself for them; I'm not at the mercy of some automation provider that wants an arm and a leg for every little thing.

      More information on the battery monitoring?? I have six blog posts on that already that includes the schematics, display, etc. Go to the box in the upper left corner of this page, type in 'battery charger' and then sort them by date. The entire evolution of the charger with an XBee inside is there.

      However, I'm going to mess with them some more in the next couple of months. I found a really, really cool little float charger that does every darn thing I want to do to maintain batteries. I'm doing some tests on it right now that will tell me if it's a good solution. So, if it works out, I'll retask the float chargers I made to be true battery monitors and just report the current voltage of the battery over an XBee link. That way I can tell if one of them is having trouble and go check it. All the parts necessary to do something like that are already there, just a little rewiring and away it will go.

      The little iteaduino looks like a really, really good idea. I especially like the 3.5 - 5 V idea. That makes it directly compatible with an XBee. The extra pins don't hurt either.

      Based on my experience with the Raspberry Pi, the little arduinos are great devices for hooking sensors to and having limited control funtions. Then you wireless everything to a Pi for more capabilities and ethernet connections. So, a little 10 buck board that has all that stuff is a great deal.

      Delete
  6. Those big motors on swimming pool pumps are a pain. The create so many harmonics as well as noise. Luckily I don't have anything like that....just my welding machines.
    I've been checking out you battery monitor (after I found how to get there) and it looks like you have had some fun with it. I too like the LM317. Always have like the National products. They seem to work very well in designs.
    I'll be messing around with the float charger for my tractor as well. Going to go with solar as soon as I can. That should work great. Need to get some XBees now though as I want the remote sampling features they allow. Also looking forward to picking up some iteaduino's. Can't beat the price.....
    Also, got to take my Firex smoke alarms in the barn and interface them into my barn system for alarming. When hay ignites out there is can really get one hot under the collar.....
    Thanks again for posting all your work. It is most informative and helpful.
    Glenn.

    ReplyDelete
    Replies
    1. Glad to be of help. Remember though, as you get further into your project and things keep working, let me know. I love to steal ideas from other folk and try them out. Since you (apparently) live out in the sticks like me, you might be interested in the float I put in the septic tank to tell me when I needed to pay attention to it.

      Living away from city services takes a lot more attention to detail doesn't it?

      Delete
  7. You got that right. I already have a float in my sceptic tank that connects to an alarm which is on the outside of the barn....not much good when I'm in the house, so that will be rectified this spring with an arduino. Got door alarms to on the barn doors and video cameras so we can check the horses on stormy nights.
    Got to get my front gate system working soon though. The driveway alarm works great but need the cameras out there going as well.
    Then of course there is the grape vineyard to instrument....but that's another tale altogether. Power monitoring and solar panels are high on the list.

    Thanks again for all your ideas and help.
    Glenn.

    ReplyDelete
  8. Hi Dave;

    I'm trying to get you house monitor code working on my version of Python 2.7.6. Unfortunately, when I copy and paste the code from your window in your website, it strips all the indents. The result is that Python's IDE run command (F5) has a fit and wants all the indents put in before it will proceed.
    Am I missing something in the way I am doing this?

    ReplyDelete
    Replies
    1. Yes, you're missing something. An author that had some idea what he was doing ... sigh.

      I've been using the same code box for a long time and it sucks. I just spent a few hours figuring out how to create a code box in blogger that actually works and I think I have it finally. However, I need someone to test it, and I bet you'll be the first victim.

      Which page were you loading it from and I'll go edit it to see if the new stuff works for you.

      Delete
  9. I sent you an email reply with the details.
    Thanks. I look forward to getting the Python code running.

    ReplyDelete
    Replies
    1. Somehow, the email got lost. Could you send it again?

      Delete
  10. I sent it a second time but must not have arrived. I'm trying to get your housemonitor.py code going. It is the code that is in the Python script box on the RPi Controller tab.
    Sooooo....if you can repost it or else just email it to me then I can try it again.
    I'm also still waiting for my current transformers, 2 ethernet shields and 10 dallas one wire temperture probes. Sucks waiting for this stuff to arrive before I can get going.
    Once I get your smart thermostats running on the two heat pumps then I've got to get some single mode fiber pulled out to the barn and put my two media converters on it for 100 Meg. Then I can repurpose my wifi access node to the front gate.

    ReplyDelete
    Replies
    1. That is so weird. I checked the google spam filter and folder and nothing turned up. At any rate, the code box on that page had been modified and should work now. It's the one with the scroll bar on the bottom of the window that doesn't wrap lines around. I copied out of it and into an editor and it worked on my machine.

      10 probes ?? What the heck are you measuring? There's a guy north of me that has a bunch of them scattered around to keep track of temperatures in his orchard, so I can see cases where it's needed.

      Delete
  11. Hey Dave. Thanks.
    I found out why the others didn't go through. Pilot error.
    I've got two heat pumps that I'm going to put temp sensors on as well as the smart thermostats plus the barn and also out in the vineyard. No shortage of places for temp sensing. Also have to do a couple of fan speed sensors for the heat pumps.
    My ethernet shields arrived today. Can't wait to play with those. Now if only the CT's would get here. I'll be trying out the code later today if I can get a couple of moments.

    ReplyDelete
  12. I downloaded the script you refer to above. And again it didn't like the indenting. So, I downloaded the emoncms script. I had one indent that it didn't like. I removed that and it now works. I saw somewhere else that you mentioned you are using IDE 2.1. I believe this was the problem. IDE 2.7 is very format anal and therein lies the majority of the problem. I will go back and edit the balance of the formatting in the house monitor script and I believe it will work then.
    Sometimes you're the windshield and sometimes you're the bug..........hee hee.

    ReplyDelete
  13. Dave:

    What is that little waterproof box you used for your pool controller? Where did you get it?

    ReplyDelete
    Replies
    1. Man, I'm glad you asked about this. I got it from Amazon.com originally. They had a stock of them and I picked up a couple.

      http://www.amazon.com/gp/product/B004EI0E6O/ref=oh_details_o02_s00_i00?ie=UTF8&psc=1

      About 5 months ago a guy asked me where I got it and it had shot up in price so much I wouldn't buy another, but typical of me, I searched it out on the internet and found this:

      http://www.wlanparts.com/wlanparts-enctel-enclosure-9x6x3-inch-outdoor-1-port-gray/

      This is the actual supplier and they will ship to 'normal' people. It's the same price as the very first one I bought. Don't believe the reviews about it failing. I have one in the direct sun subject to the monsoon rains here in Arizona and it hasn't even shown a sign of failing. It never leaks; the only problem I have is chasing the bugs out before I stick my hand inside. They tend to attract scorpions and spiders if they're in the shade. If you install it outside, paint it. Just grab a can of some appropriate color spray paint and cover the outside plastic. It will last a lot longer than the project you put inside because you'll change the project over time and reuse the box.

      If this one doesn't fit your situation the trick is to look for 'cable box' or 'exterior cable enclosure' on google images and just page through until you find what will work.

      Delete
  14. Thanks for the info. Now you've got me working on the IRIS plugin. It's going to work great for my barn heaters.
    I'm looking at using the CT-30 thermostats for my two heat pumps as they are reasonable in price and there is a sample server software that Radio Thermostats of America has available. Probably won't use it though as I prefer having a Python base solution.
    Also, have you ever worked with Python Requests. I've been playing with it a bit and it looks to have some possibilities for web based stuff. Particularly the little boards I pointed to from Modtronix.

    ReplyDelete
  15. Yes, I've worked with Requests, but only a tiny bit. It's a nice library and has been around long enough to have most of the bugs worked out. I don't use web oriented services much inside the house and I try to limit my exposure to the web, so I don't need to understand it well. As these new products come out, that may have to change.

    ReplyDelete