Wednesday, September 23, 2015

Fishtanks and the Pi - Part 1 - using Nagios for temperature monitoring

OK, so it's been quite a while since my last post, but I've not been idle, just not really concentrating on all things Pi for some time.  Not least, the Pi 2 has now come out, and in my opinion made the thing infinitely more useful!

My house now consists of two Pi devices in constant use:
  • Monitoring - I use a Pi running Nagios (more of that later!) to monitor my NAS and other services at home.
  • TV recording - I have a Pi running Tvheadend to record DVB-T via a cheap, simple Sandberg DVB-T USB tuner.
Remember back in my first post I said I'd be doing some fish tank automation?  Well, I've finally got around to putting the necessary things together, and after thinking through how to do temperature monitoring I decided that the best solution was to use the very powerful Nagios monitoring platform which I already have in the house, just move the Pi running the platform downstairs to be near the fish tank and write some scripts to make the temperature sensors work.

Now, it seems like a pretty trivial requirement, but I wanted a number of temperature sensors in the tank as I've been nervous for a while about how well the water mixes thermally, especially when I change the water or the house is colder than usual in the winter months.

In order to get this integration with Nagios working, I had to achieve a few other things first, namely:
  1. Find an appropriate, saltwater safe and heavy duty temperature sensor/probe which would work with the Pi.
  2. Work out how to connect a number of them up and avoid any risk of water damaging the electronics!
  3. Get them talking to the Pi.
  4. Get them integrated into Nagios.
  5. Provide time series logging.
In addition, I want to integrate this with a display at some point in the future - more on that later!

Choosing the temperature sensor

So, this was easier than I thought in fact.  The DS18B20 is a very robust 3-pin transistor-cased temperature sensor with a decent temperature range and 0.5°C accuracy.  You can buy versions of these sensors ready wired in a metal tube filled with potting compound, which should be pretty resistant to the salt water (and inert).  I bought these and found they worked really well for this purpose.  Time will tell in terms of how well they react to the salt (and the livestock react to them, though I think it'll be fine!).

Connecting the sensors together

The 1-wire protocol makes the sensors very easy to wire up - they basically have a single signal line, and require 0v and 5v supplies.  The can operate with 'parasitic power' - as shown in the datasheet, but as this requires additional components, and the Pi has a readily available 5v power supply, I thought it wasn't worth the trouble.

The real bonus with these devices is that you can connect as many of them as you like together in parallel.  This means that wiring becomes trivial.  I opted for a simple stripboard for this project, pretty quick and easy to wire together and designed for lots of parallel connections!

Sensors connected together using stripboard
The picture to the right shows the completed stripboard with the wires cable tied to it for strain relief.  You'll also note I've put a BPM180 PCB on the board as well - this is a different type of temperature sensor which I've included to use as a calibration check to ensure the DS18B20s are behaving.  More on making that work in a later post.  The labels are the identified serial numbers of the sensors.  I kind of jumped the gun here and did things backwards - I suggest you do as well (this isn't meant as a how-to article, just as advice!) - connect the sensors one at a time to the Pi and record the serial numbers generated in the /sys/bus/w1/devices directory so you know which one is which later on!

I've added a 3-pin header at the far end so that I can extend this board to even more devices; I intend to put one through a hole in the wall to measure the outside temperature, for example.

The grey multi-core cable at the bottom of the picture is a 6-core cable going to the Pi.  This has 3 connections (black, red and green) going to the 1-wire sensors, and the other 3 going to the BPM180 (with another 0v black line!) - annoyingly the BPM180 uses 3.3v, not 5v so there are two power lines for these two types of sensor.

You'll see I've used a liberal amount of heatshrink sleeving - this is highly recommended as not only does it keep the strands of wire / soldering from shorting between pins, but it also makes the thin wires less susceptible to breaking off as it reduces the likelihood of any right-angle bends in the wires.  You can buy it from any good electronics store, and a hairdryer or any gas powered soldering iron with heat gun attachment (my preference) will get the things to shrink.  At some point, I'll probably protect the whole thing with some larger heatshrink, but for now I've left it unclothed as I want to keep it accessible for debugging.

On the other end of the grey cable is the Pi itself, I've soldered additional headers onto the Adafruit 16x2 LCD display (ready for Part 2 of this project!) and soldered the wires from the cable onto those headers with additional heatshrink.

Interfacing with the sensors

The 1-wire protocol makes interfacing with the sensors pretty easy, but I did battle with a software update to the Pi breaking things for a while about 6 months ago.  Basically, Raspbian made the move to Device Tree (DT) at that point, which dynamically creates device files in /sys and /dev to ensure interoperability with all of the hardware ecosystem which has taken off massively over the past year or so.  You can read more about Device Tree here on the Raspberry Pi website.

What the DT update meant was that you now have to specify the 1-wire settings in /boot/config.txt as well as loading kernel modules:


Once you've made this change to config.txt, for me it just worked and the relevant kernel modules were automatically.  Some documentation does talk about enabling w1-gpio and w1-therm manually in /etc/modules, but I think this was written prior to DT being enabled.

Anyway, once this is all done, take a look in /sys/bus/w1/devices/.  If the sensors are connected and the module is loaded properly, you should see something like this:

As you can see, there are a bunch of hex-coded folder links, and a 'bus_master' folder.  It's the hex-coded names we're interested in.  These are the serial numbers of the connected sensors.  A peek in one of these folders shows a few files, and the one we care about - cryptically named w1_slave:

The w1_slave file contains - if the sensor is working correctly - the measured temperature, along with some checksum data and a 'YES/NO' field.
The fact that this file shows YES means that the checksum is accurate (there was no data loss in the wire, and the sensor didn't fail somehow) and so you can believe the temperature data.  If there was a NO in this field then we shouldn't trust the rest of the data in the file.

In case you're wondering where the temperature is, look at the 't=' number; familiar?  Yep, that's it - the sensor provides a number in millidegrees, so you need to divide by 1,000 to make sense of the number.  Remember me mentioning above that the accuracy is only 0.5°C - yeah, I'm not sure why the number needs to be that accurate either, but there we go.  You get 3 decimal places, but you can't believe them.

Integration with Nagios

Now it's all very well getting the data, but what are we going to do with it?  Well, we need to parse these files and then present the data in a form that Nagios can deal with.  Fortunately, thanks to Rule 21 of The Internet (I think that's the right rule, and no, I don't mean 34), someone has already written a nice Python library to parse this data - they are using the same sensors for brewing beer - good luck to them, I may try it myself at some point!.  Anyway, after pouring myself a pint I forked their snippet and made a few changes to better suit my needs in the Nagios integration, the code is available here on BitBucket.

Now for the Nagios plugin.  I won't go into lots of detail in the blog post, hopefully I'll get around to further documenting the code soon but it's still pretty self explanatory - go ahead and take a look here / clone the repo:'s one thing that probably warrants further information; I have combined maximum and minimum checks into the same call to - this is primarily for compactness within the Nagios config, but also for a pretty good reason, which is that the file reading actually can take some time (for the sensor to stabilise and write the data to the file), so it's best not to read it twice if you can avoid it.  Therefore there are both -w and -W, -c and -C parameters (for warning min/max and critical min/max respectively) to the check script.  This may well break Nagios guidelines, but hey, it works for me!

[UPDATE 1-OCT-2015] I'd missed a cunning feature in Nagios - you can specify max and min thresholds as ranges - this page from the plugin development guidelines outlines the solution.  I've checked in a change to the repo to do just this, thus removing the -C and -W parameters.

Installation is pretty easy, put the and in a folder somewhere, make it executable by the nagios user and set up your configuration files appropriately.

One thing you do need to think about is what 'host' to put the service checks in.  It doesn't really make sense to ping a host for temperature, so I created a quick check_forceok command which can be used in a host check to always return OK and keep Nagios happy.  That's also in the repo for your use.

My fishtank monitoring config files (the relevant bits - I've omitted the rest of my setup) are in the repo as an example file.  Run without any parameters to get the help text and get some advice on parameters, etc.

Time series data logging

This turned out to be the main thing which pursudaded me to use Nagios in the first place; although I could drop the data into rrdtool or similar, why not kill two birds with one stone and combine the aggregate time series data with the rest of my system monitoring?

Nagios (since version 3) has provided the ability to store performance data alongside the service and host checks, and via an add-on it is pretty simple to graph this stored data.  I chose to use nagiosgraph for this, but there are many others.  The specifics of the graphing solution are outside the scope of this article, but suffice it to say that it's pretty easy.  This is where the Pi 2 really comes into its own.  I tried installing graphing solutions using the original incarnation and producing a lot of graphs was frankly beyond the capabilities of the limited RAM of those older machines.