The Minute You Turn Your Back

Last weekend I noticed an odd increase in power that shouldn't have happened.  Then, it happened again a couple of days ago.  Here's the graph:

Notice the two humps in the red graph, one around 12PM and the other at 2PM.  These shouldn't be there because all the heavy power using devices have been shut down during the afternoon peak usage period.

As I described on my Power Usage page <link>, this could easily drive my bill up by a hundred dollars or so.  Needless to say, I was concerned and started checking every appliance in the house to see if it could have come on during this period.  The length and duration looked suspiciously like the water heater assist coil could have been activated, but everything on that device checked out OK.

I scratched my head for a while and noticed that this was a recurring event during the heat of the day.  Here's a graph of the next day showing a few of the same kind of thing.

So, I went outside, looked at the meter for the demand level to see what I might have to pay come bill time and the peak level was at 1.5 kW, not the 3kW or so I expected.  This means that the humps in power usage aren't real, they're some kind of artifact of the power reading process.

Voltages checked out, things seemed ok until I flipped off the GFI that supplies the power monitor <link> and it wouldn't come back on when I tried.  A ton of troubleshooting later which included having to climb up on a ladder and disconnect the garage door opener and a partial dismantling of the mains box, I found the problem.

See, there were several problem that have been going on that I thought were unrelated.  The GFI I mentioned failed a week ago and seemed to be fine after I replaced it,  the freezer in my kitchen has had a problem with the interior lights flashing erratically when the door is open, electric tools in the garage have strangely shut off, and the garage light (on the door opener) comes on sometimes for no reason.  This phantom power usage indication was what got me off my butt and looking for the problem.

It was an arching circuit breaker in the mains panel.  These things can be easily removed and have a slide on connector to the power buss inside the panel.  Since my panel is full, I have a number of double breakers where two half size breakers can take the place of one full size.  One of these was arching where it connects to the power buss.  Here's the breaker that failed:

And here's a close up of the connection that was giving me the problem:

If you look closely there is charring on the spring connector in the slot.  There was charring on the post in the power panel as well. Replacing the breaker corrected the problem and all the other strange things inside the house settled right down.

OK, here's the disclaimer:  There's enough power inside the mains box to kill you, so if you're the least bit uncomfortable, don't mess with it; leave it to someone that thinks they're qualified to work on this.  Me, I took it apart without a second thought, but that's just me.

What was happening was that the connection was loosening when the heat inside the panel rose during the heat of the day.  That side of the house is in the direct afternoon desert sun and gets really hot; you can't put your hand on the panel in the heat of the day without gloves.  All the equipment is rated for this since I live in the desert, but the environment takes its toll over time, so this kind of thing can happen.

The loose connection arched, got some carbon deposits that caused it to arch some more, and created enough noise on the power lines that it drove the sensitive electronics of the power monitor nuts.  It also caused the lights to flicker in my freezer because the second switch on the breaker was hooked to the freezer in the kitchen.  My garage door and the GFI that failed are fed from the same circuit the power monitor is plugged into.  This little two circuit breaker was driving my house electronics crazy.

The really good thing about this is that the problem showed up as a bad reading on the power monitor.  This was before the breaker failed completely and shut down a critical part of my house...my freezer.  If I had been away and the thing failed, I would have lost the contents of my freezer and had a mess to clean up.  See, there's another good reason to automate and monitor my house.

So, if you have strange flickering lights, GFIs that don't work right, and garage doors that behave strangely, it may not be ghosts.

Pool Controller 2 for Goldline P4

No, I didn't take a month off; I've been off on other projects that aren't technical or related to power.  Stuff like fixing a chip in my motorcycle paint, equipping toy hauler to tow it, and spring pruning.  But, I started having some trouble with the transmissions from my pool controller.  I have a whole section on my swimming pool page <link> where I describe how I decoded the Goldline protocol and built a controller for my swimming pool.  Yes, there are remote controllers out there that allow you to change settings and such from the house, but mine hooks into the House Controller and allows me to change things from the internet.  Yes, I can turn my pool motor on from anywhere.

However, as I described, there are some problems.  First, it's based on an Arduino Uno which only has one serial port.  I need at least two, one for the XBee to transmit status and the other for the RS485 link to the Goldline pool controller.  It would be nice to have another one for debugging and testing commands as well.   I used SoftwareSerial code to handle the extra port needs, but there's the occasional over run and I'm really running the little Arduino on the edge with two high speed ports cranking away all the time.  Guess what, I have a Mega2560 that isn't doing anything.

So, I pulled the device off the fence and rebuilt it essentially from scratch.  I put the XBee on Serial3, the pool controller on Serial2, and use the regular Serial as my command and debugging port.  This is all possible because the watchdog timer and boot loader problems have been fixed as I detailed in several posts, and I discovered a new enclosure that I have already used on my Garage Controller <link> that gives me plenty of room and easy access.

Since it was going on a Mega2560, there would have to be software changes.  I decided to take the time to incorporate all that I've learned over the last couple of years and update the XBee and timer handling.  The device now uses the Arduino XBee library and is in API mode so the packets are handled well.  It knows what time it is by catching the House Clock <link> transmission, so it's possible to build in specific time of day alarms or do special things on the weekends.  There's room in memory and in the enclosure to incorporate the Acid Pump timer as well, freeing up a device.  I will also be able to hook in the float I installed in my septic tank so it can alarm when the tank filter gets clogged <link>.  Of course, it will be more than a Pool controller then, but what's in a name?  Besides, a septic tank is sort of a pool ... isn't it?

Here's a picture of the device after I mounted the boards into the enclosure:

And here it is on the fence working:

Here's the devices on that side of the house:

No, I didn't mount it crooked, the fence goes down hill at that point.

Naturally, I'll post the code on the Swimming Pool page, but I'll get to that in a day or two because I have to change a lot of stuff to illustrate it the way I want to.

Why aren't more people doing this kind of thing around their house?

Edit:  I had to turn off comments on this page...  Darn bots from all over were flooding this particular page with crap.  Don't ever have a page with a title line that has 'Gold" in it.

Water Heater and Solid State Relays Part 2

Part 1 of this is here <link>

I wandered out into the garage to admire my new SSR controlled water heater and noticed that the indicator led was on.  It was during the period when the water heater should have been turned off and that can't be allowed to happen.  I drug out the ladder, voltmeter, and screwdriver and dug into it.  There was no power getting to the SSR control circuit, so it shouldn't have been on, but the indicator led showed there was voltage on the output.  Sure enough, there was a little over 100V on the output.  Could this brand new SSR have failed already?  Fortunately, everything was fine.  I just didn't read enough about the devices.  They have transient suppression on them that is built right into the module.  This transient suppression will allow a little voltage to be seen on an open circuit.  So, the water heater had shut off by opening the switch that operates the heating element.  The SSR was off, but the suppression circuitry was allowing a little voltage through.  Nothing to worry about, but how to test it to be sure?

See, strange readings and unexpected results often happen when you're dealing with 240VAC circuits that use power at 15 - 25 amps.  Leakage of a few milliamps is expected and allowed for in this kind of circuitry for various suppression techniques.  The relay that used to do this job was a real mechanical open circuit, not a semiconductor.

Testing turned out to be simple; just drain enough hot water out of the tank to kick the mechanical heater switch, and then measure the voltage while the SSR was off.  Sure enough, when the element switch closed, the voltage on the output dropped to virtually nothing.  Everything was actually fine.  The little led indicator just doesn't draw enough current to suck it down to zero and will continue to glow.  I also noticed that the led was dim when the SSR was off and bright when the SSR allowed current through.  It's hard to tell the difference between the two levels, and I may look into the indicator at some point and see about correcting this.  It may just need a resistor in line or something.

No, I didn't waste the hot water.  I needed a bath anyway.

Water Heater and Solid State Relays

A few times I mentioned the solar water heater I have.  It works great and keep me in free hot water most of the time.  Yes, it's the desert and during the summer the coldest the water gets is lukewarm, but I get 140F water out of the heater for zero continuing expense except for the few days where it is overcast.

Those overcast days present a problem.  A couple of times I turned the power on to the water heater and used the helper coil to heat the water because the sun just didn't come out.  Once, it pushed my power bill up because an X10 controller didn't react to the turn off command.  This prompted me to hook the water heater power to the garage controller I have.  This worked fine.  I'm able to send an XBee signal and turn on the water heater.  I also have code in the garage controller that shuts the heater down during peak periods. It's working really great except...

The contactor I have started to buzz.  It works fine, but there is a constant 60Hz buzz coming from it any time it's on.  The first time I heard it, I had to hunt for a while to figure out what it was.  Then, as the days went by, it got more and more annoying.  For those of you that have no idea what a contactor is, here's a picture of the one I have.

They're really nothing much more than a really big relay.  They have specially built contacts to survive high current switching and lots of places to hook wires.  They are probably all over your house and you don't know it.  Appliances have them, some fireplaces, environmental heating, etc.  Most of them run on a 24V coil and last a really long time.  In my particular case, I used a 110V coil because I was originally switching it with an X10 appliance module.  Unfortunately, the 110V units tend to make noise as they age.  They're not wearing out, they just start to rattle at 60Hz and drive you nuts.  There is a solution, use a solid state relay instead.

So, I got online and started looking for an SSR that could handle 25A at 240VAC.  There's a lot of them out there, but I need two of them, one for each leg of the power.  That made converting expensive, but I kept looking and found a SSR for 240 VAC three phase.  This gives me three of them in a package and I only have to use two of them.  I don't know if I'll ever need the spare, but it's there if I need it.  Here's the device I bought:

Notice that is has inputs for three circuits and one control for all of them.  This is exactly what I wanted.  So, when it came in, I took the entire control assembly off the wall and rebuilt it essentially from scratch.  Here's what it looks like on the wall:

Simple isn't it?  Just one device with the control signal coming in the top and hooking up on the left side.  It only takes a little over 4VDC to turn it on and it doesn't make any noise at all.  Since there are no contacts, it should last forever just sitting there working it's little heart out.  

I probably should have used one of these to begin with, but I didn't have a clue XBees even existed when I started out on these projects.  I was convinced by the advertising and various articles on the web that X10 would do the job.  

It's sad that I was that naive.

The followup to this posting is here <link>.

Battery Charging - Part 6

My float charger that is based on ideas I stole from the Harbor City 42292 device has gone through another iteration.  I decided to actually have a circuit board made that would hold the components and allow me to mount the board properly.  Using Eagle, I drew up a board which included the charger parts and everything necessary to support the XBee.  It was fun and came back looking like this:

It was so cool getting my first printed circuit board design back in the mail, so I rushed right in and assembled it.  Naturally, there were a lot of problems.  I hooked the voltage divider for the analog measurement to the wrong pin on the XBee and had trouble telling it was transmitting because I didn't include  troubleshooting lights to look at.  I got Eagle back up and changed the design, and ordered another board.

On this one I also increased the size of some of the traces and moved the voltage regulators around so I could use the heatsink I kept from the original charger.  So, I populated the board and hooked it to the lid of the box.  The entire device looks like this:

I happened to catch the leds on in the picture, but they are disconnected in a running device because they draw unnecessary power when it is actually in use.  It works real nice, and keeps the battery voltage at a constant 13.4 volts.  I've had several hand wired ones in place for months now and they are doing the job.

I'm still not done though.  I discovered a way to sense current being put out by the device.  I can put a hall sensor over a trace on the board that carries current and it will change output relative to the current flowing through the trace.  I doubt that I can tell the exact amount of current passing through the trace, but I can certainly tell if it is actually charging the battery.  This idea will take some development though and I don't know what components will be necessary for the final result, so I'll just assemble the rest of them and put them to work. I hope to get back to the hall effect device idea in a couple of months and work out the details.  The previous post on these devices is here <link>

I learned a lot creating the board, and I think I can develop boards for other ideas I have.  Things like a timer that will catch the time broadcast on my XBee network to operate lights and other devices would be great. It gets to be a real pain tediously connecting wires on a protoboard, especially when you want several identical devices.  Having an XBee network and something like this means I can finally get rid of the X10 devices that are so unreliable.  It would be so cool to completely switch over to XBee throughout the house.

Still Wrapping up the Mega2560 Bootloader

More on the continuing drama of the 2560 bootloader.  Most recent post on this is here.

I just moved a lot of my code up to IDE 1.0.3 and decided to check and see if they had updated the bootloader in this.  Well, they haven't.  I just used it to program a 2560 and the bootloader supplied with the release failed on both the '!!!' and the watchdog timer problems.

But, I went searching to see if there was some schedule and ran across another bootloader that works.  This one is smaller because the author removed the code that samples for the '!!!' to jump into a rom monitor.  So, there is no problem with '!!!' because it doesn't look for it, and the watchdog timer works.  Plus, we get a little code space back (not that it matters on a 2560).  I tested it with several things and successfully loaded code that exceeds 60K (biggest I currently have) to the board.  The hex file is in text form here <link>.  If you want to get it and try it, just copy and paste it into notepad and save it as a .hex file.  I used an AVRISP mk 2 to replace the bootloader, I haven't tried any other device for programming the chip.

Here is the code I used to test the two problems notice the '!!!' in the Serial.print:

Code:

#include <avr/wdt.h>

void setup(){

  wdt_reset();   // First thing, turn it off
  wdt_disable();
  Serial.begin(57600);
  Serial.println("Hello, in setup!!!");
}

int firstTime = true;

void loop() {
      if (firstTime){
        firstTime = false;
        Serial.println("In loop waiting for Watchdog");
        wdt_enable(WDTO_8S);    // eight seconds from now the device will reboot
      }
      // now just do nothing
}

It's annoying that they haven't updated the IDE with a version that works.  But, there are a couple of solutions that work now.

Using the XBee Library Part 3

Part 1 <link>, Part 2 <link>.

As part of my porting of my devices to the latest version of the Arduino IDE, I decided to move the devices to using the XBee library.  Back in part 1 I described why I suddenly took an interest to it, and since I was going into the code, I might as well start using it for real at the same time.  The changes to make my XBees receive packets using the library went without a hitch.  It just worked fine and reduced the code I had to write by a lot.  However, when I looked at transmitting data, I was appalled.  All the examples show something like:

XBee Tx Example

void loop() {
  // break down 10-bit reading into two bytes and place in payload
  pin5 = analogRead(5);
  payload[0] = pin5 >> 8 & 0xff;
  payload[1] = pin5 & 0xff;

  xbee.send(zbTx);

  // flash TX indicator
  flashLed(statusLed, 1, 100);

I mean REALLY??  Setting the bytes into an array one at a time?  Well, I went searching the web to see what else could be done and found dozens of examples of this kind of thing.  Here's one from the New SoftSerial example:

Another Example

void loop() {

  payload[0] = 'H';
  payload[1] = 0x7E;
  payload[2] = 0x11;
  payload[3] = 0x13;
  payload[4] = 0x7D;

  xbee.send(zbTx);

I even saw examples that had many bytes filled in one at a time to compose the data that needed to be sent.  Clearly, the examples need to be updated.  That's not something that is easy to do, but I can put an example here to show how a simple string, "Hello World" can be transmitted to another XBee without all the hassle. Here is an extremely easy way to do it:

My Tx Example

/*
XBee TX test for a Arduino Mega2560 using Serial3 as the XBee serial
input for a Series 2 XBee.  This is NOT based on the examples that come with
the Arduino XBee library.

See, the examples there and most other places on the web SUCK.  Andrew's
library is much easier to use than the illustrations would lead you to believe.

This is a HEAVILY commented example of how send a text packet using series 2
XBees.  Series 1 XBees are left as an exercise for the student.
*/

#include <XBee.h>

XBee xbee = XBee();
// This is the XBee broadcast address.  You can use the address
// of any device you have also.
XBeeAddress64 Broadcast = XBeeAddress64(0x00000000, 0x0000ffff);

char Hello[] = "Hello World\r";
char Buffer[128];  // this needs to be longer than your longest packet.

void setup() {
  // start serial
  Serial.begin(9600);
  // and the software serial port
  Serial3.begin(9600);
  // now that they are started, hook the XBee into
  // Software Serial
  xbee.setSerial(Serial3);
  Serial.println("Initialization all done!");
}

void loop() {
  ZBTxRequest zbtx = ZBTxRequest(Broadcast, (uint8_t *)Hello, strlen(Hello));
  xbee.send(zbtx);
  delay(30000);
  strcpy(Buffer,"I saw what you did last night.\r");
  zbtx = ZBTxRequest(Broadcast, (uint8_t *)Buffer, strlen(Buffer));
  xbee.send(zbtx);
  delay(30000);
}

Yes, that's all there is to it; this example will compile and run using IDE 1.0.3 on a Mega2560.  Andrew made the library quite capable, but most people don't seem to want to look deeply enough into it to use it to advantage.  Notice that you fill in the Tx request with an address and then send the string pointer and its length to the XBee.  It's just that simple.  You can use a buffer and format it to be exactly what you want, then hand it off to the xbee.send() for transmission.  Really, really nice.

Remember though this example is for a mega2560 with four serial ports.  To do exactly the same thing on a UNO, just use SoftwareSerial as I described in part 1.  It will work equally well and you still have the Serial to output status and debugging information so you can see what is going on.

Suppose you want to send the packet to a specific XBee on your network, avoiding the problems that can happen with broadcast <link> it's actually really easy.  Just create another address like the one for Broadcast in the example above and use it instead.

Using this technique, I reduced a fifty or so line transmit routine down to:

How Simple It Can Be

XBeeAddress64 Broadcast = XBeeAddress64(0x00000000, 0x0000ffff);

void sendXbee(const char* command){
  ZBTxRequest zbtx = ZBTxRequest(Broadcast, (uint8_t *)command, strlen(command));
  xbee.send(zbtx);
}

Yes, I can compose the commands for my various devices and then just call sendXbee() with the string.  Simple, easy to read, and it works.

I like my little XBees more and more each day.