Setup Logrotate to Manage Log Files on Ubuntu

If you are a System Administrator handling large number of servers and systems that generate high volume of log files, then logrotate utility will help you by saving the disk space to avoid potential damage. It is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

Install Logrotate On Ubuntu

sk@server1:~$ sudo apt-get install logrotate

Configure Logrotate

The main  config file /etc/logrotate.conf file has the general and default options. The service and application specific configuration files are kept in /etc/logrotate.d/ directory.

The main config file /etc/logrotate.conf will look like below.

sk@server1:~$ cat /etc/logrotate.conf 
# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here

/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

Let me explain you a short introduction about the options.

weekly: This is the log rotation interval.

rotate 4: Logrotate will keep 4 weeks of log files backup.

create: New empty will be created after rotating the old ones.

compress: make it uncomment if you want to compress the log files.

The lines within /var/log/wtmp and /var/log/btmp represents wtmp and btmp services log rotate configuration. If you want to add specific service log rotation configuration, then you can define them in the /etc/logrotate.d/ directory.

Sample Service Log Configuration

Let us create an example service log configuration file called unixmen under /etc/logrotate.d/ directory.

sk@server1:~$ sudo nano /etc/logrotate.d/unixmen

Add the following lines into it.

/var/log/unixmen.log {
missingok
notifempty
compress
size 100M
daily
create 0755 sk sk 
}

Save and exit the file. As i mentioned above, here:

missingok: avoids the output error if any log file missing.

notifempty: do not rotate if the file empty.

compress: the log files will be compressed with gzip method.

size 100M: logs will be rotated if they reach 100MB size.

daily: log rotation interval.

create 0755 sk sk: creates new file with permission 755 where owner is sk and group is sk.

Automate Logrotation with Cron

Automate the logrotation process using the cron file /etc/cron.daily/logrotate. The logroatation will be performed daily.

Test Log Files

sk@server1:~$ cat /var/lib/logrotate/status 
"/var/log/apport.log" 2013-7-18
"/var/log/unixmen.log" 2013-7-18
"/var/log/apache2/error.log" 2013-7-18
"/var/log/upstart/procps-virtual-filesystems.log" 2013-7-18
"/var/log/auth.log" 2013-7-18
"/var/log/upstart/udev-fallback-graphics.log" 2013-7-18
"/var/log/wtmp" 2013-7-18
"/var/log/ConsoleKit/history" 2013-7-18
"/var/log/apache2/other_vhosts_access.log" 2013-7-18
"/var/log/upstart/procps-static-network-up.log" 2013-7-18
"/var/log/mysql/mysql-slow.log" 2013-7-18
"/var/log/debug" 2013-7-18
"/var/log/tt-rss.log" 2013-7-18

Thats it. For more information about Logrotate look into the man pages using the following command:

sk@server1:~$ man logrotate

For best performance, i suggest you to create a separate partition for your /var/log/ directory.