Introduction
One of the most imporant task for an system administrator is the troubleshooting and diagnostic of problems. Today you can say goodbye to top, lsof, strace, iostat, ps, etc., and you can use sysdig: The most powerful Linux system troubleshooting.
Sysdig is open source, system-level exploration and a troubleshooting tool that captures system state and activity from a running Linux instance using a linux kernel facility called tracepoint , then save, filter and analyze. Sysdig combines the benefits of many utilities such as strace, tcpdump, and lsof into one single application.
Sysdig is also packed with a set of scripts called Chisels that make it easier to extract useful information and do troubleshooting.
In this article we’ll explore the installation and basic usage of sysdig to perform system monitoring and troubleshooting on Linux.
For more information, you can see the website of Sysdig.
Features Of Sysdig
- Fast and stable
- Easy to Use
- Debugging pretty much anything
- Crazy Powerful
- and more
Installing Sysdig On Ubuntu
To install
via apt we will need to setup the apt repository maintained by Draios the company behind
. We can do this by running the following
commands.
The following commands should be run by root user.
curl -s https://s3.amazonaws.com/download.draios.com/DRAIOS-GPG-KEY.public | apt-key add -
curl -s -o /etc/apt/sources.list.d/draios.list http://download.draios.com/stable/deb/draios.list
After this command you need to update your server. Use this command to do that.
apt-get update
The
tools requires the kernel headers package before installing it. Flow the command to install the kernel headers package.
Use this command to check the kernel version.
uname -r
apt-get install linux-headers-<kernel version>
Now you can install sysdig on ubuntu using this command.
apt-get install sysdig
Installing Sysdig On CentOS
Before stating with the installation itself, you will need to setup the yum repository that will use this key to verify the authenticity of the package you’re about to download.
Use the
tool with the
flag To manually add the Draios key to your RPM keyring.
sudo rpm --import https://s3.amazonaws.com/download.draios.com/DRAIOS-GPG-KEY.public
Then, download the Draios repository and configure yum to use it:
sudo curl -s -o /etc/yum.repos.d/draios.repo http://download.draios.com/stable/rpm/draios.repo
After that try to make update to your server using this command
yum update
Then you need to enable extra packages for enterprise Linux (EPEL) The EPL repository is needed in order to download the Dynamic Kernel Module Support (DKMS) package, which is used by sysdig tool. Flow the command to enable EPEL repository:
sudo yum -y install epel-release
Then you need to install kernel headers in order to setup
sudo yum -y install kernel-devel-$(uname -r)
Now you can install sysdig.
sudo yum -y install sysdig
Using Sysdig
The simplest and easiest method to use sysdig is by invoking it without any argument.By default, sysdig prints the information for each captured event on a single line.
sysdig
you will see output like this example.
25722 22:48:41.788762892 3 sshd (1733) > rt_sigprocmask 25723 22:48:41.788763312 3 sshd (1733) < rt_sigprocmask 25724 22:48:41.788763603 3 sshd (1733) > rt_sigprocmask 25725 22:48:41.788763801 3 sshd (1733) < rt_sigprocmask 25726 22:48:41.788764486 3 sshd (1733) > read fd=11(<f>/dev/ptmx) size=16384 25727 22:48:41.788765210 2 sysdig (3745) > switch next=318 pgft_maj=0 pgft_min=910 vm_size=92452 vm_rss=5532 vm_swap=0 25728 22:48:41.788766062 3 sshd (1733) < read res=118 data=8622 22:48:41.759980138 2 sysdig (3745) > switch next=318 pgft_maj=0 pgft_min... 25729 22:48:41.788778756 2 <NA> (318) > switch next=3745(sysdig) pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_swap=0 25731 22:48:41.788785389 3 sshd (1733) > select
To write output of the sysdif in a file you can use the
flag and specify the file name.
sysdig -w <output file>
Example:
sysdig -w trace.dump
You can read the output file using this command
sysdig -r <output file>
Example:
sysdig -r trace.dump
command has filters that allow you to filter the output to specific information. You can find a list of available filters by running this command.
sysdig -l
sysdig -l
----------------------
Field Class: fd
fd.num the unique number identifying the file descriptor.
fd.type type of FD. Can be 'file', 'ipv4', 'ipv6', 'unix', 'pipe', 'e
vent', 'signalfd', 'eventpoll', 'inotify' or 'signalfd'.
fd.typechar type of FD as a single character. Can be 'f' for file, 4 for
IPv4 socket, 6 for IPv6 socket, 'u' for unix socket, p for pi
pe, 'e' for eventfd, 's' for signalfd, 'l' for eventpoll, 'i'
for inotify, 'o' for uknown.
fd.name FD full name. If the fd is a file, this field contains the full path.
If the FD is a socket, this field contain the connection tuple.
<truncated output>
You can use the “proc.name” filter to capture all of the
events for a specific process.
sysdig -r tracefile.dump proc.name=name-of-process
Example:
sysdig -r tracefile.dump proc.name=sshd
530 23:45:02.804469114 0 sshd (917) < select res=1
531 23:45:02.804476093 0 sshd (917) > rt_sigprocmask
532 23:45:02.804478942 0 sshd (917) < rt_sigprocmask
533 23:45:02.804479542 0 sshd (917) > rt_sigprocmask
534 23:45:02.804479767 0 sshd (917) < rt_sigprocmask
535 23:45:02.804487255 0 sshd (917) > read fd=3(<4t>10.0.0.12:55993->162.0.0.80:22) size=16384
You can see here many examples how to use sysdig.
That all!
Hope this article help you.
