Friday, June 28, 2013

RPi-Monitor: Version 2.x advance usage and customization

Note: A new version of RPi-Monitor is available.
For all details about RPi-Monitor refer to RPi-Monitor Documentation.


Note: Information bellow are outdated




Edit: RPi-Monitor version 2.7 introduce a change in configuration name and location:
  • /etc/rpimonitor d.conf becomes /etc/rpimonitor/daemon.conf
  • /etc/rpimonitord.conf.d becomes /etc/rpimonitor/data.conf
  • /etc/rpimonitord.conf.d/*.conf becomes /etc/rpimonitor/*.conf

In this article we will see how to do the things described in RPi-Monitor: Advance usage and customization and maybe, a little bit more...

We will then answer the same questions : How should I do if I want to...
  • Customize RPi-Monitor v2.x to fit a particular needs?
  • Use it with my own Web Server? 
  • Add "friends" linking RPi-Monitor v2.x running on different platform together?
  • Use it into another Linux distribution than Raspbian? 
  • Add other graphs from additional sources (other mount point or external temperature probe)?

Photo from Flickr AttributionNoncommercialShare Alike cc-by-nc-sa by blogjunkie
Note: A prerequisite is to install the latest version of RPi-Monitor v2.x
Customize RPi-Monitor v2.x to fit a particular needs

If you want to customize RPi-Monitor in a way not described bellow you just have to know 2 things:
  • The man pages rpimonitord and rpimonitord.conf explains how to use and configure RPi-Monitor.
  • The configuration file is documented to let you customize to your need.
You will find bellow some example of customization showing you the capababilities of RPi-Monitor v2.x



Use it with my own Web Server

In this example I will use nginx server. You could use the same tactic with your preferred web server.

Let's first update the configuration file to disable the embedded web server. Edit the file /etc/rpimonitord.conf and set daemon.noserver=1.

Then restart RPi-Monitor with the command:
    sudo service rpimonitor restart

The embedded server is no more running so, we will need to configure another server to access the data.

For nginx we will create the file /etc/nginx/sites-enabled/rpimonitor with the following content:


and restart the server with the command:
    sudo service nginx restart

Now you can reach RPi-Monitor with your favorite browser on your favorite web server.

To go further, you can read the article RPi-Monitor: Build a multi-sites SSL certificate to improve user experience.


Add "friends" linking RPi-Monitor running on different platform together

Friends is a notion that comes in version 1.2. A friend is simply another computer running RPi-Monitor. Configuring friends will add a drop down list on the right of the top menu with a links to RPi-Monitor sitting on the other computer.

Friends menu is visible on the top right

In /etc/rpimonitord.conf.d/default.conf, each friend is identified by its and is described by the 2 following parameters:

Here is an example of configuration for 3 friends:



Use it into another Linux distribution than Raspbian

RPi-Monitor has been designed to run into a Raspberry Pi but as it is using only standard Linux resources, it is not hardware dependent. A simple configuration update can make it run on UbuntuCentOS or any other distribution.

Ubuntu is a Debian based distribution as Raspbian. The installation can then be done using the deb package available for each releases. Download and install the package as described in this previous post.

With CentOS and withe any non Debian based distribution it will required to perform a manual installation.
First install the perl dependencies: HTTP::Daemon (perl-libwww-perl), RRD (rrdtool-perl) and JSON(perl-JSON)

Connect to github  and select the latest stable branch on the top left dropdown list.
Then download the code as zip file from the link "Download zip" visible at the bottom of the right menu.

Unpack it.
    unzip Version-2.x.zip

Finally install rpimonitor manually:
    su -
    mv Version-2.x/rpimonitor /usr/local

You can now start RPi-Monitor with the following commands:
    cd /usr/local/rpimonitor
    ./rpimonitird -c rpimonitor.conf -c default.conf

Note: I will not describe here how to configure the auto startup since each distribution has its own way to do so. An upstart script is available into RPi-Monitor github tools directory, it may help you in such an action.


Once the installation is done you can start RPi-Monitor and connect to it with your favorite browser.
You may notice that some values are undefined or displayed as NaN (Not a Number). To fix these issues, you will have to update the configuration file (rpimonitord.conf or default.conf in /etc/ + /etc/rpimonitord.conf.d or /usr/local/rpimonitor/ depending on your installation).

CentOS 6.3 before configuration file customisation



Add other graphs from additional sources: other mount point

For this section we will take the example of an additional disk on sda as described previous article  RPi-Monitor: Advance usage and customization.

The disk have a disk with two partition /dev/sda1 and /dev/sda3.
The command and regular expression will be the following:
  • sda1 disk size command : df -t ext2,  regular expression: sda1\s+(\d+)
  • sda1 used space command : df -t ext2,  regular expression: sda1\s+\d+\s+(\d+)
  • sda3 disk size command : df -t ext4,  regular expression: sda3\s+(\d+)
  • sda3 used space command : df -t ext4,  regular expression: sda3\s+\d+\s+(\d+)
First we need to configure the extraction of partitions sizes which are extracted once at RPi-Monitor startup. We will create a file /etc/rpimonitord.conf.d/custo.conf with the data configured as static data like this:

The id of the KPIs start at 10 since in my configuration files the previous KPI was 9. This comment is the same for next ids.
The post processing is configured to transform kB into MB by dividing the extracted result by 1024.

For dynamic values extracted every 10 seconds, the configuration will be:


Dynamic stat will be stored into a RRD File as GAUGE. Ref to RRDTool help for detail about Data Source Types.
Now we will add a status line for this disk whit the following icon:

Disk icon has been found here
This icons has to be installed into the img directory of RPi-Monitor which is by default /usr/share/rpimonitor/web/img/.

The configuration to add a new status strip will then be the following:
    

The configuration may need some explanation:
We do configure 4 lines. Each line is describing a javascript line using some predefined functions: KMG, Precent and ProgressBar. This function are called by the browser while rendering the page. Some variable coming from the extracted data are also used. These variables are starting by the keyword 'data.'. For deeper detail about this configuration execute the command man rpimonitord.conf

To see our modification we need to restart RPi-Monitor and refresh the statistics page into our browser.
    sudo  service rpimonitor restart

The result of the configuration is at the bottom of the following screenshot:



The status page is working, let's now add a graphic of the disk usage. This is done with the following configuration:

The configuration may also need some explanation
We do configure 2 graphs each having 2 curves. The first curve represent the total and is using static data extracted previously. This curve will be represented as a light red line.
The second curve is representing the usage of disk and is represented as a light blue line filled. The parameters defining the curve are define by the keyword ds_graph_options. Details of this parameter can be found in javascriptrrd help page. Restart rpimonitor to activate the new graph.

After waiting a little time to let the system extract data you will see this kind of graph.





Add other graphs from additional sources: external temperature probe

Let's build a little electronic device to measure the room temperature and add this in the temperature graph.
Do do that, you will need:
  • 1 x DS18B20
  • 1 x 4.7k resistor
Plug them as described in the following schema extracted from Adafruit Lesson 11:


Lets first load the kernel module required to get the information from this probe:
    sudo modprobe gpio
    sudo modprobe w1-gpio
    sudo modprobe w1-therm

First we have to identify the id of our probe:
    ls /sys/bus/w1/devices/
    28-000004fe1847  w1_bus_master1

the id is: 28-000004fe1847. Let's now check if we can get the temperature:
    cat /sys/bus/w1/devices/28-000004fe1847/w1_slave
    5a 01 4b 46 7f ff 06 10 a3 : crc=a3 YES
    5a 01 4b 46 7f ff 06 10 a3 t=21625

It looks to work. So now we need to extract the the numbers ending the second line (21625). The regular expression will then be t=(\d+)$. I'll not do a course about what a regexp is there are many site on the internet explaining it.

The following configuration have to be added at the end of /etc/rpi-monitord.conf.d/custo.conf. It will extract the temperature measured by the external probe:

We will then add the livingroom temperature as Ambient temperature into the existing Temperature status strip by adding the following line at the end of the file custo.conf:

After restarting RPi-Monitor, you will see the Ambient temperature into the status page as in the screenshot bellow:



We will then add the Ambiant temperature into the existing Temperature graph by adding the following line to custo.conf:

After restarting RPi-Monitor, you will see the Ambient temperature as an additional curve into the Temperature graph of the statistics page as in the screenshot bellow:




25 comments :

  1. You can also easily add WiFi network by adding the following to /etc/rpimonitord.conf.d/default.conf as indicated or just copy and paste into /etc/rpimonitord.conf.d/custo.conf

    # Additions to display WiFi stats

    #This goes on the end of the KPI's section around line 187
    dynamic.15.name=wifi_received
    dynamic.15.source=/sys/class/net/wlan0/statistics/rx_bytes
    dynamic.15.regexp=(.*)
    dynamic.15.postprocess=$1*-1
    dynamic.15.rrd=DERIVE

    dynamic.16.name=wifi_send
    dynamic.16.source=/sys/class/net/wlan0/statistics/tx_bytes
    dynamic.16.regexp=(.*)
    dynamic.16.postprocess=
    dynamic.16.rrd=DERIVE

    # This goes on the end of the section that begins 'web.status.1.content.8.name=Network' around line 352
    web.status.1.content.8.line.2="WiFi Sent: <b>"+KMG(data.wifi_send)+"<i class='icon-arrow-up'></i></b> Received: <b>"+KMG(Math.abs(data.wifi_received)) + "<i class='icon-arrow-down'></i></b>"

    #This goes in the web.statistics section at the end of the the section that begins 'web.statistics.1.content.2.name=Network' around line 375
    web.statistics.1.content.2.graph.3=wifi_send
    web.statistics.1.content.2.graph.4=wifi_received
    web.statistics.1.content.2.ds_graph_options.wifi_send.label=WiFi Upload bandwidth (bits)
    web.statistics.1.content.2.ds_graph_options.wifi_send.lines={ fill: true }
    web.statistics.1.content.2.ds_graph_options.wifi_send.color="#FF7700"
    web.statistics.1.content.2.ds_graph_options.wifi_received.label=WiFi Download bandwidth (bits)
    web.statistics.1.content.2.ds_graph_options.wifi_received.lines={ fill: true }
    web.statistics.1.content.2.ds_graph_options.wifi_received.color="#77FF00"

    ReplyDelete
  2. Hi Xavier!

    Nice work. I found it very usable! Thanks a lot!

    I was wondering if it is possible to use mysql as datasource in rpi-monitor somehow?
    For example I would make a python script to extract one particular data from mysql table. Could I use this script as a source?
    Or actually could I use environmental variables as source. (Maybe I could set env var from bash script)
    As I have experienced the SD card do not like that I write and read a txt file every 10 seconds just for data source purposes.

    ReplyDelete
    Replies
    1. OK! It's a blame!
      I have done it whith a simple bash script! Sorry for disturbing!
      For whom may want to use this clever software whith mysql source here is my solution for the problem:

      I have made a bash script with a MySQL SELECT, which gets $foo from database and an echo $foo at the end of script. (named e.g.: sample.sh)
      I've made a symlink in /usr/bin which is showing to my bash script (Note: the symlink has not got extension. So it is like: ~sample rather then ~sample.sh)
      In rpimonitord.conf.d/default.conf I've used the symlink as source, like this:

      dynamic.16.source=sample

      That's it. And it works properly.

      Delete
  3. How can I add the "lpstat -p"? This is a simple status message.
    It's not working:

    dynamic.10.name=cups
    dynamic.10.source=lpstat -p
    dynamic.10.regexp=(.*)
    dynamic.10.postprocess=
    dynamic.10.rrd=DERIVE

    web.status.1.content.9.name=CUPS
    web.status.1.content.9.icon=printer.png
    web.status.1.content.9.line.1="Status: "+data.cups+""

    Thank you!

    ReplyDelete
    Replies
    1. To customize RPi-Monitor, I would advise to read the comment I made in DHT11/22 article (http://rpi-experiences.blogspot.fr/2013/07/rpi-monitor-use-dht11-or-dht22-humidity.html).
      Before configuring RPi-Monitor, you should first try the regexp with perl and once it works, configure it.
      Once you will have find the perfect setup, I'll let you post you configuration and share it to everybody.
      (Note: sorry for this short response but I started a new job and will have less time for RPi-Monitor...)

      Delete
    2. Check the status of CUPS printer server:

      static.10.name=printer_status
      static.10.source=lpstat -p
      static.10.regexp=(.*)
      static.10.postprocess=

      web.status.1.content.9.name=CUPS
      web.status.1.content.9.icon=printer.png
      web.status.1.content.9.line.1='Status: ' + data.printer_status + '.'

      The 64x64px icon: http://goo.gl/UcQo4E

      ---

      I have character coding problem, I can't repair it:
      http://dl.dropbox.com/u/3577295/Kijel%C3%B6l%C3%A9s_014.png

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Hi ! Nice work.

    I'm reading the temperature sensor from another Pi on the network using the following command :
    ssh pi@192.168.1.10 cat /sys/bus/w1/devices/28-000004f14128/w1_slave
    and it works fine.

    Now when I use (in custo.conf) :
    dynamic.18.source=ssh pi@192.168.1.10 /sys/devices/w1_bus_master1/28-000004f14128/w1_slave

    I get an "undefined°C" temp in RPI-Monitor...

    I'm guessing I'm using the wrong command for the source... Could anyone help ?

    Thanks.

    ReplyDelete
    Replies
    1. I have similar problem with other system variables.

      Delete
    2. Hi Bjorn. Did you find a solution to your remote Pi data acquisition?

      Xavier suggested (in another post) creating a custom script that would poll for information (in your case, the temperature data) then save that information to a local file. You could then query this local file with RPi-Monitor.

      Anyhow - I'd like to accomplish the same task and was wondering if you have a solution.

      Delete
  6. Hello,

    the string "/usr/share/rpimonitor/img/" above needs to be "/usr/share/rpimonitor/web/img/" on my RPi-Monitor installation (2.5.1).

    ReplyDelete
  7. Is it possible to show CPU temp as F instead of c?

    ReplyDelete
    Replies
    1. Yes, it is even very easy.
      You just have to enter the formula converting from C to F as postprocess.
      $1 is the temp in C so formula should be: $1*9/5+32

      Delete
  8. I found this also, which one is correct?
    To have the temperature in °F, you should change the post-processing formula from $1/1000 to ($1/1000)*1.8+32 into the same file.

    ReplyDelete
  9. Is it possible to make the setup for "other mount point" basically a USB HDD setup but its NTFS i have NTFS setup on my PI and RPi-mon kinda see's it it shows the size but this is what is says "/storage1 Used: NaNkB (NaN%) Free: NaNkB Total: 293.29GB"

    ReplyDelete
    Replies
    1. Check the formula extracting the space used on disk. It looks that the total is ok. The "used" doesn't looks OK and as free is calculated with the formula "Total"-"Used" you also have the issue.
      Check also that the name of the variable are correct into the static javascript definition (web.status.X.content.Y.line.Z=)

      Delete
  10. Im sorry, im not that familiar with coding...i have tried many diffrent things, changing this "dynamic.14.source=df -t ext2" to "dynamic.14.source=df -t ntfs" ect... also "dynamic.14.regexp=sda1\s+\d+\s+(\d+)" to "dynamic.14.regexp=*/dev/sda1*\s+\d+\s+(\d+)" */dev/sda1* is where the NTFS drive is mounted.

    ReplyDelete
  11. This is my setup in custo.conf if it helps..

    static.10.name=storage1_total
    static.10.source=df -t ntfs
    static.10.regexp=/dev/sda1\s+(\d+)
    static.10.postprocess=$1/1024

    dynamic.19.name=storage1_used
    dynamic.19.source=df -t ntfs
    dynamic.19.regexp=/dev/sda1\s+\d+\s+(\d+)
    dynamic.19.postprocess=$1/1024
    dynamic.19.rrd=GAUGE

    ReplyDelete
    Replies
    1. To customize RPi-Monitor, I would advise to read the comment I made in DHT11/22 article (http://rpi-experiences.blogspot.fr/2013/07/rpi-monitor-use-dht11-or-dht22-humidity.html - September 10, 2013 at 11:34 AM ).
      Before configuring RPi-Monitor, you should first try the regexp with perl and once it works, configure it.

      Note: if the perl execution give you the good information, check if you correctly refer to "storage1_used" into the status lines.

      Delete
  12. I'm really new to this, im basically taking the example that you show above,(which worked great with ext4 format) and when i changed the drive to ntfs i lost the info, and i have no idea what regexp with perl is, im sorry for the newness, i really really like your system and want to get it to work with my ntfs share drive but i'm just lost beyond the basic config in the given example above about storage drive. Any help would be greatly appreciated.

    ReplyDelete
    Replies
    1. Not sure if you got it working already but I had the same problem. I figured out how to solve this.

      Check your file system type by using the command "df -T" without the quotes.

      Look up the Type of the drive you want to use. Even though mine is formatted ntfs, it's Type is written as "fuseblk". Therefor I was now able to use the command "df -t fuseblk" and it received the information of the drive.

      Now just change this in your custo.conf or whatever configuration file you are using and it should work fine.

      Hope this helps for you.

      Delete
  13. I can't get the webserver to work with apache on my Pi. When I try to create the file "rpimonitor" in /etc/apache2/sites-enabled/ with the tekst you provided, I get the message "Access Denied"

    How can I get RPi-Monitor to work with apache?

    ReplyDelete
    Replies
    1. "Access Denied" is a page given by apache and not related to RPi-Monitor.

      To fix you issue, I would advised to create a page named index.html with, as content, the sentence "It works". Then configure your apache to serve this page. Once it works, you will just have to do a similar configuration pointing to the directory /usr/share/rpimonitor/web/.
      I did my test with nginx so I can't help more...

      Delete
  14. Hello,
    I installed the tool and like it very much. Everything works so far and I'm also monitoring my USB-disk, that is attached to the pi. I'm only worried about the additional access to the disk that will be generated by the monitoring. The disk is usually in sleep mode, when it is not actively used by the pi. Does the statistic for the monitor generate real disc access that will wake my disk? It this is the case, I have to disable it again to protect my disk and save energy.

    BR,
    Matthias

    ReplyDelete
    Replies
    1. Hello Matthias,

      RPi-Monitor is gathering information on the system every 10 seconds (by default) and write these information into RRD files (generating disk access where RRD file are stored).
      Each time it probes the system RPi-Monitor execute commands (define into the configuration file). These commands may have impact on your system (and maybe wakeup your disk :-( ). This have to be tested...
      If your test show that RPi-Monitor have an impact on your disk, the workaround I would propose to avoid awaking your disk every 10sec is to create an external script gathering information in a frequency matching your needs and writing the result into a file which will be read by RPi-Monitor.

      GR,
      Xavier

      Delete