Full monitoring system: Graphite, collectd and StatsD – Part 3

Introduction

In our three part tutorial for installing and configuring a full monitoring system we have already installed both Graphite and collectd on the Ubuntu 16.04 server.

• Graphite is a graphic library and it’s the part of the system that display required data
• collectd is the daemon that collect and flush to Graphite system data

In this tutorial we’ll install and configure

, which is a very simple daemon, written in Node.js, which collects and aggregates arbitrary data (while

is used for running system related metrics). For instance, with

it’s possible to collect information at an application level, which is useful during development.
Let’s look at the process for installing and configuring StatsD on Ubuntu 16.04 and flushing data to

.

Getting started – Install Node.js

As said in the introduction,

is a daemon that runs on Node.js. It’s not available as an Ubuntu package, so we will just have to create a new one!
First of all, install the following packages:

# apt install nodejs debhelper devscripts

Install StatsD

Create a directory in which to build

, for example:

$ mkdir /opt/statsd_build

There, clone the source code:

$ cd /opt/statsd_build
$ git clone https://github.com/etsy/statsd.git

Next, build the package:

$ cd statsd
$ dpkg-buildpackage

At the end, a new

package will be available in

. So, stop Carbon and install it:

# systemctl stop carbon-cache
# dpkg -i statsd*.deb

will start automatically. Stop it and restart Carbon:

# systemctl stop statsd
# systemctl start carbon-cache

Now, it’s time to configure the daemon.

Configure StatsD

To configure

, edit its configuration file. In Ubuntu, after the installation, there are two of them in

directory. Edit

:

# $EDITOR /etc/statsd/localConfig.js

In that file, there is code that looks something like this:

{
  graphitePort: 2003
, graphiteHost: "localhost"
, port: 8125
}

It’s not required to change these values because these are the same used during the 

configuration.

The only thing required here is to set off legacy namespacing. This was used in the past by

to organize its data, but it’s no longer required because a more intuitive way has been introduced. So:

{
  graphitePort: 2003
, graphiteHost: "localhost"
, port: 8125
, graphite: {
    legacyNamespace: false
  }
}

Save and exit.

Configure a storage schema

As for

, the next step is to configure a storage schema for

. Of course, the syntax is the same used for the other daemon. So, edit the storage schemas configuration file:

# $EDITOR /etc/carbon/storage-schemas.conf

Above the default block, add this new block:

[statsd]
pattern = ^stats.*
retentions = 20s:1d, 5m:7d, 10m:1y

Note that the smaller interval time in the retentions line must be the same as the one used for Graphite. If not, some data could be lost!

Configure data aggregation

The last step is to configure a way to aggregate data. Edit the storage aggregation configuration file:

# $EDITOR /etc/carbon/storage-aggregation.conf

There should be content like this:

[min]
pattern = \.min$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = \.max$
xFilesFactor = 0.1
aggregationMethod = max

[sum]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum

[default_average]
pattern = .*
xFilesFactor = 0.5
aggregationMethod = average

It’s important to notice that metrics for minimum and maximum values should not be averaged. If they were, it wouldn’t be possible to preserve the lowest and highest points.

To configure downsampling, the

file should be edited as follows:

[min]

<strong>pattern = \.lower$

</strong>xFilesFactor = 0.1

aggregationMethod = min



[max]

<strong>pattern = \.upper(_\d+)?$</strong>

xFilesFactor = 0.1

aggregationMethod = max



[sum]

pattern = \.sum$

xFilesFactor = 0

aggregationMethod = sum



[count]

pattern = \.count$

xFilesFactor = 0

aggregationMethod = sum



[default_average]

pattern = .*

xFilesFactor = 0.5

aggregationMethod = average

This means that for metrics ending with 

or 

only the minimum and the maximum value must be kept.

Save and close the file.

Conclusions

When everything is well configured, you’ll want to restart all services:

# systemctl stop carbon-cache 
# systemctl start carbon-cache
# systemctl start statsd

And that’s it! Just go to http://localhost using a web browser and visualize graphed data!