Some older versions of rsyslog may have trouble resuming on a log file after the log rotate has run. We can force rsyslog to pick up the new log file by adding a postrotate command in logrotate. This will restart rsyslog and delete the state files so it will continue reading from the beginning of the newly created file.
Log Rotate Setup
1. Open the logrotate configuration file
Configuration files are located in this directory on most linux distributions
Find the appropriate configuration file and open it with a text editor. In this example, we’ll use the nginx log files.
sudo vim nginx
You will see the default configuration given below
create 640 nginx adm
[ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
All the commands written between the postrotate and endscript gets executed after each log rotation. In this example, we can see that nginx is restarted. We will add additional commands here soon.
2. Find your rsyslog state files
Find the files rsyslog writes to track the state of the files are monitoring. We will configure the postrotate command to delete these configuration files so rsyslog starts fresh at the beginning of the new log file. If you used the configure-file-monitoring script, it will include the alias you passed as a parameter.
Here we can see two state files for nginx
3. Add postrotate commands
Add the following commands in the postrotate section to restart rsyslog and delete the state files. Replace the path given to the rm command with the path to the rsyslog state files found above. In this example, we are deleting the state files for nginx.
service rsyslog stop
service rsyslog start
Advanced Log Rotate Options
Troubleshooting Your Log Rotate Configuration
If you don’t see any data show up in the verification step, then check for these common problems.
- Wait a few minutes in case indexing needs to catch up
- If you see duplicate events send to Loggly, check to see if you accidentally deleted the wrong state files. Also check to make sure a new file of zero length is created after the log rotation runs.
- Make sure the state files are deleted and recreated after rsyslog restarts
- Troubleshooting Rsyslog if the files are being written but not being sent to Loggly
- Search or post your own question in the community forum.