How log rotation works with logrotate
Log rotation on Linux systems is more complicated than you might expect. Which log files are rotated, when and how often, whether or not the rotated log files are compressed, and how many instances of the log files are retained all depend on settings in configuration files.
Rotating log files is important for several reasons. First, you probably don't want older log files eating up too much of your disk space. Second, when you need to analyze log data, you probably don't want those log files to be extremely large and cumbersome. And last, organizing log files by date probably makes spotting and analyzing changes quite a bit easier (e.g., comparing last week's log data to this week's).
The logrotate utility makes log rotation fairly easy and automatic. It puts a lot of intelligent practices to use, but to manage and modify how this process works, you would need to be able to peer into the files that control how log files are rotated.
What is log rotation?
Log rotation is the process that renames a current log file (e.g., auth.log becomes auth.log.1) and sets up a new log file (e.g., auth.log) for new log entries. Depending on the number of files to be retained, we might see something like logfile.6 becoming logfile.7 (with the old logfile.7 disappearing) and logfile.5 becoming logfile.6, etc. before the new logfile is created. The older log files might also be compressed, particuarly if they tend to be very large files. So, you might see logfile.1.gz instead of logile.1.
How logrotate works
The logrotate tool is commonly used to manage the process of log rotation, though logrotate itself is run through cron.
The important files to pay attention to are:
- /usr/sbin/logrotate -- the logrotate command itself (the executable)
- /etc/cron.daily/logrotate -- the shell script that runs logrotate on a daily basis (note that it might be /etc/cron.daily/logrotate.cron on some systems)
- /etc/logrotate.conf -- the log rotation configuration file
Another important file is /etc/logrotate.d, included in the process through this line in the /etc/logrotate.conf file:
As you can see from the entries below, seven generations of syslog files are retained and most are compressed.
For many log files, only four generations of old files are retained. To understand why seven syslog files are retained by default, take a look at this section of the /etc/logrotate.d/rsyslog file. Note the "rotate 7" specification.
The syslog file rules also specify "delaycompress" meaning the most recent file will not be compressed until the next rotation cycle.
For a number of other log files, the rotation specifications are quite different. Only three generations of these log files are retained. They're rotated weekly instead of daily.
For wtmp and btmp files, rotation details are included in the /etc/logrotate.conf file. These log files are rotated monthly, and only one older file is retained. Note that the configuration lines below also determine the rotated files' permissions and ownership.
Here's what these other settings mean:
- weekly: Rotate logs once per week. Available options are daily, weekly, monthly, and yearly
- missingok: It's OK if no *.log files are found
- rotate #: Keep specified number of files before deleting older log files
- compress: Compress (gzip) log files
- delaycompress: Delays compression until second time around
- compresscmd: Set which command to used to compress. Defaults to gzip
- uncompresscmd: Set the command to use to uncompress. Defaults to gunzip
- notifempty: Don't rotate empty files
- create 640 root adm: Create new log files with set permissions/owner/group
- postrotate: Scripts to run after rotating is done
- prerotate: Scripts to run before log rotating begins
- size: Rotate when the file size reaches a particular limit
The delaycompress setting is often used for files that are more likely to be used fairly soon, so leaving them ready for use for a day makes sense.
The logrotate.conf file specifies the rotation schedule (default is weekly) for most log files, the group to assign, whether to create new files, whether to compress by default, etc.
There are quite a few files in the /var/log directory on most Linux systems, but many of these files are not rotated by default and many are in subdirectories. A simple command with wc can count them for you:
The /var/lib/logrotate/status file, created when /etc/cron.daily/logrotate runs, shows the date and time when each of the log files was last rotated.