Graylog: a powerful logs management system
Graylog is an open source logs management system which parses and enriches log messages, wire and event data from any source, thus providing a centralized configuration management system for third-party collectors, like fluentd, beats and nxlog. For example, with Graylog it is possible to enrich log messages with geo-coordinates translated from IP addresses, or to map a user ID to a user name.
A few of Graylog’s most notable features include:
- Central logs management system, which gives your team access to runtime configuration and log data without touching the Graylog servers.
- Grouping users into roles to simplify permissions management. Graylog has a very powerful system for restricting data access to users, which can really come in handy.
- LDAP integration.
- REST API usage for programatically log data accessing.
This tutorial will cover Graylog installation and basic configuration on a machine running Ubuntu 16.04.
- One server running Ubuntu 16.04 with at least 2 GB of RAM.
- Elasticsearch 2.x.
- Oracle JDK 8.
If your system matches the above listed prerequisites, you can start the Graylog 2 installation process.
Keep server update:
$ sudo apt-get update && sudo apt-get upgrade $ sudo apt-get install apt-transport-https uuid-runtime pwgen
As noted above, Graylog 2.0.0 (and higher) requires Elasticsearch 2.x. You’ll want to modify the Elastisearch configuration file:
/etc/elasticsearch/elasticsearch.yml, setting the cluster name so that the it matches the one set in the Graylog configuration file. In this tutorial, the cluster name chosen is graylog.
With a text editor, open the Elasticsearch configuration file:
$ sudo $EDITOR /etc/elasticsearch/elasticsearch.yml
Search for the
cluster.name line and uncomment it. Next, modify as follows:
# ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: graylog #
Save and close the file, then restart the Elastisearch service:
$ sudo systemctl restart elasticsearch
Once the server configuration is complete, we can move onto Graylog installation. Configure the Graylog repository with the following commands:
$ wget https://packages.graylog2.org/repo/packages/graylog-2.2-repository_latest.deb $ sudo dpkg -i graylog-2.2-repository_latest.deb
Next, install the package:
$ sudo apt-get update && sudo apt-get install graylog-server
At the end of installation process, enable Graylog to start at boot time:
$ systemclt enable graylog-server
Before starting Graylog, we’ll need to configure it.
The Graylog configuration file is
/etc/graylog/server/server.conf. Well have to edit some parameters in this file before we can start the logs management program.
First of all, we need to set the
password_secret value. This must be at least 64 characters long. We will generate it using pwgen.
You can install this tool with apt:
$ sudo apt-get install pwgen
Next, using sed, we write the generated characters directly into the Graylog configuration file:
$ sudo -E sed -i -e "s/password_secret =.*/password_secret = $(pwgen -N 1 -s 128)/" /etc/graylog/server/server.conf
In order to check that everything was done correctly, use the following code:
$ cat /etc/graylog/server/server.conf | grep password_secret
The command should display the
password_secret line. In our case:
password_secret = hjg5nBbZQcgLVW3do5uw1irfbq9UiRwhISZgPie8r96dejt4hgWdHUJcIaK1onQfFFatbrPZ3WV4yEhoqX9ITtaEUmn9SKn2aRT62uCO9KRZGK81q2xrO1aMQnOELPqP
The next step is to set the
root_password_sha2, which is the SHA-256 hash of our desired password. First, execute the following command:
$ sudo sed -i -e "s/root_password_sha2 =.*/root_password_sha2 = $(echo -n 'your_password' | shasum -a 256 | cut -d' ' -f1)/" /etc/graylog/server/server.conf
To be able to connect to Graylog, we must also configure the
web_listen_uri values to hostname or the public IP address of the machine we can connect to. Web interface URI and REST API must be accessible by everyone using the web interface and this means that Graylog must listen on a public network interface.
Open the Graylog configuration file:
$ sudo $EDITOR /etc/graylog/server/server.conf
In this file, search for the
rest_listen_uri line, which, by default, is:
# REST API listen URI. Must be reachable by other Graylog server nodes if you run a cluster. # When using Graylog Collectors, this URI will be used to receive heartbeat messages and must be accessible for all collectors. rest_listen_uri = http://127.0.0.1:9000/api/
127.0.0.1 with the server public IP.
Next, search for the
# Web interface listen URI. # Configuring a path for the URI here effectively prefixes all URIs in the web interface. This is a replacement # for the application.context configuration parameter in pre-2.0 versions of the Graylog web interface. #web_listen_uri = http://127.0.0.1:9000/
Uncomment it, and change the IP, just as you did in the
Save and close the file, then start Graylog:
$ sudo systemctl start graylog-server
Check the Graylog status using the following code:
$ sudo systemctl status graylog-server
graylog-server.service - Graylog server Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vend Active: active (running) ...
With a web browser on a client, go to
http://192.168.1.138:9000/. The browser will show a login page
Use admin as user name, and the password entered in the configuration step (‘your_password‘).
After logging in, you’ll see a ‘Getting Started’ page:
By going on System > Inputs, we have access to inputs configuration.
This is where all inputs will be configured, which is the first step of data collecting in Graylog for logs management.
At this point we have a Graylog server correctly up and running on an Ubuntu 16.04 machine. In a future guide we will see how to configure inputs and send data from a server to this powerful logs management system.
We will also see how to configure a multi-node Graylog system, for more advanced logs management.