How log rotation works with logrotate

Log rotation on Linux systems is more complicated than you might expect. Which log files are rotated, when and how often, whether or not the rotated log files are compressed, and how many instances of the log files are retained all depend on settings in configuration files.

Rotating log files is important for several reasons. First, you probably don't want older log files eating up too much of your disk space. Second, when you need to analyze log data, you probably don't want those log files to be extremely large and cumbersome. And last, organizing log files by date probably makes spotting and analyzing changes quite a bit easier (e.g., comparing last week's log data to this week's).

The logrotate utility makes log rotation fairly easy and automatic. It puts a lot of intelligent practices to use, but to manage and modify how this process works, you would need to be able to peer into the files that control how log files are rotated.

What is log rotation?

Log rotation is the process that renames a current log file (e.g., auth.log becomes auth.log.1) and sets up a new log file (e.g., auth.log) for new log entries. Depending on the number of files to be retained, we might see something like logfile.6 becoming logfile.7 (with the old logfile.7 disappearing) and logfile.5 becoming logfile.6, etc. before the new logfile is created. The older log files might also be compressed, particuarly if they tend to be very large files. So, you might see logfile.1.gz instead of logile.1.


How logrotate works

The logrotate tool is commonly used to manage the process of log rotation, though logrotate itself is run through cron.

The important files to pay attention to are:

  • /usr/sbin/logrotate -- the logrotate command itself (the executable)
  • /etc/cron.daily/logrotate -- the shell script that runs logrotate on a daily basis (note that it might be /etc/cron.daily/logrotate.cron on some systems)
  • /etc/logrotate.conf -- the log rotation configuration file

Another important file is /etc/logrotate.d, included in the process through this line in the /etc/logrotate.conf file:

include /etc/logrotate.d

As you can see from the entries below, seven generations of syslog files are retained and most are compressed.

$ ls -l /var/log/syslog*
-rw-r----- 1 syslog adm 14255 Aug 22 19:55 /var/log/syslog
-rw-r----- 1 syslog adm 74662 Aug 22 07:35 /var/log/syslog.1
-rw-r----- 1 syslog adm  6924 Aug 21 07:35 /var/log/syslog.2.gz
-rw-r----- 1 syslog adm  3713 Aug 20 07:35 /var/log/syslog.3.gz
-rw-r----- 1 syslog adm  7272 Aug 19 07:35 /var/log/syslog.4.gz
-rw-r----- 1 syslog adm  3815 Aug 18 07:35 /var/log/syslog.5.gz
-rw-r----- 1 syslog adm  6905 Aug 17 07:35 /var/log/syslog.6.gz
-rw-r----- 1 syslog adm  3627 Aug 16 07:35 /var/log/syslog.7.gz

For many log files, only four generations of old files are retained. To understand why seven syslog files are retained by default, take a look at this section of the /etc/logrotate.d/rsyslog file. Note the "rotate 7" specification.

root@stinkbug:/etc/logrotate.d# more rsyslog
        rotate 7        <==
                invoke-rc.d rsyslog rotate > /dev/null

The syslog file rules also specify "delaycompress" meaning the most recent file will not be compressed until the next rotation cycle.

For a number of other log files, the rotation specifications are quite different. Only three generations of these log files are retained. They're rotated weekly instead of daily.

        rotate 4
                invoke-rc.d rsyslog rotate > /dev/null

For wtmp and btmp files, rotation details are included in the /etc/logrotate.conf file. These log files are rotated monthly, and only one older file is retained. Note that the configuration lines below also determine the rotated files' permissions and ownership.

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    create 0664 root utmp
    rotate 1

/var/log/btmp {
    create 0660 root utmp
    rotate 1

Here's what these other settings mean:

  • weekly: Rotate logs once per week. Available options are daily, weekly, monthly, and yearly
  • missingok: It's OK if no *.log files are found
  • rotate #: Keep specified number of files before deleting older log files
  • compress: Compress (gzip) log files
  • delaycompress: Delays compression until second time around
  • compresscmd: Set which command to used to compress. Defaults to gzip
  • uncompresscmd: Set the command to use to uncompress. Defaults to gunzip
  • notifempty: Don't rotate empty files
  • create 640 root adm: Create new log files with set permissions/owner/group
  • postrotate: Scripts to run after rotating is done
  • prerotate: Scripts to run before log rotating begins
  • size: Rotate when the file size reaches a particular limit
root@stinkbug:/var/log# ls -l auth.log*
-rw-r----- 1 syslog adm  13629 Aug 22 17:55 auth.log
-rw-r----- 1 syslog adm 283548 Aug 21 07:35 auth.log.1  <== not compressed
-rw-r----- 1 syslog adm  11316 Aug 13 07:35 auth.log.2.gz
-rw-r----- 1 syslog adm  11591 Aug  7 06:43 auth.log.3.gz
-rw-r----- 1 syslog adm  14536 Jul 24 07:35 auth.log.4.gz

The delaycompress setting is often used for files that are more likely to be used fairly soon, so leaving them ready for use for a day makes sense.

The logrotate.conf file specifies the rotation schedule (default is weekly) for most log files, the group to assign, whether to create new files, whether to compress by default, etc.

There are quite a few files in the /var/log directory on most Linux systems, but many of these files are not rotated by default and many are in subdirectories. A simple command with wc can count them for you:

root@stinkbug:~# find /var/log -type f | wc -l

The /var/lib/logrotate/status file, created when /etc/cron.daily/logrotate runs, shows the date and time when each of the log files was last rotated.

$ more status
logrotate state -- version 2
"/var/log/apt/term.log" 2017-8-7-6:44:11
"/var/log/cups/error_log" 2017-8-19-7:35:1
"/var/log/unattended-upgrades/unattended-upgrades.log" 2017-8-7-6:44:11
"/var/log/ufw.log" 2017-8-23-7:0:0
"/var/log/dpkg.log" 2017-8-7-6:44:11
"/var/log/lightdm/seat0-greeter.log" 2017-8-23-7:35:2
"/var/log/unattended-upgrades/unattended-upgrades-shutdown.log" 2017-7-15-7:0:0
"/var/log/auth.log" 2017-8-21-7:35:1
"/var/log/apt/history.log" 2017-8-7-6:44:11
"/var/log/atop/dummy_after" 2017-8-23-7:35:2
"/var/log/speech-dispatcher/debug-flite" 2017-8-23-7:0:0
"/var/log/wtmp" 2017-8-7-6:44:11
"/var/log/lightdm/lightdm.log" 2017-8-23-7:35:2
"/var/log/speech-dispatcher/speech-dispatcher.log" 2017-8-23-7:0:0
"/var/log/mysql/error.log" 2017-8-23-7:35:2
"/var/log/repowerd.log" 2017-8-23-7:35:2
"/var/log/syslog" 2017-8-23-7:35:2
"/var/log/kern.log" 2017-8-21-7:35:1
"/var/log/cups/access_log" 2017-8-23-7:35:2
"/var/log/daemon.log" 2017-8-23-7:0:0
"/var/log/mail.warn" 2017-8-23-7:0:0
"/var/log/" 2017-8-23-7:0:0
"/var/log/speech-dispatcher/debug-festival" 2017-8-23-7:0:0
"/var/log/ppp-connect-errors" 2017-8-23-7:0:0
"/var/log/alternatives.log" 2017-8-7-6:44:11
"/var/log/speech-dispatcher/speech-dispatcher-protocol.log" 2017-8-23-7:0:0
"/var/log/lightdm/x-0.log" 2017-8-23-7:35:2
"/var/log/messages" 2017-8-23-7:0:0
"/var/log/atop/dummy_before" 2017-8-23-7:35:2
"/var/log/upstart/*.log" 2017-8-23-7:0:0
"/var/log/lightdm/x-1.log" 2017-6-1-10:51:54
"/var/log/apport.log" 2017-8-23-7:35:2
"/var/log/btmp" 2017-8-7-6:44:11
"/var/log/mail.err" 2017-8-23-7:0:0
"/var/log/cron.log" 2017-8-23-7:0:0
"/var/log/user.log" 2017-8-23-7:0:0
"/var/log/mysql.log" 2017-8-23-7:0:0
"/var/log/debug" 2017-8-23-7:0:0
"/var/log/vsftpd.log" 2017-7-9-7:35:1
"/var/log/unattended-upgrades/unattended-upgrades-dpkg.log" 2017-8-7-6:44:11
"/var/log/mail.log" 2017-8-23-7:0:0
"/var/log/speech-dispatcher/debug-epos-generic" 2017-8-23-7:0:0
"/var/log/lpr.log" 2017-8-23-7:0:0
0 (0)
Article Rating (No Votes)
Rate this article
There are no attachments for this article.
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
RHEL: Rebuilding the initial ramdisk image
Viewed 6738 times since Sat, Jun 2, 2018
RHEL: Reserved space on a ext2/ext3/ext4 filesystem
Viewed 4101 times since Sun, May 27, 2018
Install Security Patches or Updates Automatically on CentOS and RHEL
Viewed 1553 times since Fri, Oct 26, 2018
RHCS6: Install a two-node basic cluster
Viewed 3191 times since Sun, Jun 3, 2018
RHEL: Displaying system info (firmware, serial numbers... )
Viewed 11565 times since Sun, May 27, 2018
haproxy linux
Viewed 2233 times since Sun, Dec 6, 2020
Sample logrotate configuration and troubleshooting part 2
Viewed 9230 times since Fri, Nov 30, 2018
10 Linux rsync Examples to Exclude Files/Directories
Viewed 10630 times since Wed, Oct 31, 2018
WatchDog script for checking server running
Viewed 5107 times since Tue, Jul 31, 2018
RHEL: GPT/MBR partition tables (using disks larger than 2 TiB)
Viewed 11950 times since Sun, May 27, 2018