Log Rotation
The log file generated by the application can quickly increase in size, and if you want to make sure it doesn't take up too much disk space, you can introduce log rotation. Log rotation will periodically clear the old logs, thus preventing the log file from taking up all the disk space.
On most Linux systems, the logrotate command is already located at /usr/sbin/logrotate. The way to set up log rotation can differ based on the kind of Linux distribution you use, whether you set up the application as root or as a simple user, whether you placed the application in /opt or in /home, and the exact location where the logs are placed. You will find 2 typical use cases below:
A) When the log files are placed inside /opt/timeline or some similar place, and docker is executed as a root user:
- Create a log rotation config file, for example at: /etc/logrotate.d/timeline
The file should contain the following:
/opt/timeline/logs/* {
size 1G
copytruncate
rotate 1
}
The path should point to the log file generated by the docker-compose up command. This particular configuration would clear the log file when it exceeds the size of 1 GB and copy its original content to another file called /home/<USER>/timeline/logs/docker-compose.log.1. The next time the log rotation runs and finds that the log size exceeds 1 GB again, it overrides the docker-compose.log.1 with its new contents and clears the original log file. - This assumes that the Linux system already has a log rotation installed and registered as a cron job. You can verify this by checking that an /etc/logrotate.conf file exists, and it contains the line include /etc/logrotate.d, and also that there is a file called /etc/cron.daily/logrotate that runs the command /usr/sbin/logrotate /etc/logrotate.conf. Different Linux distributions might have these files arranged in a different way.
B) When the log files are placed inside /home/<USER>, and are written by a non-root user:
- Create a log rotation config file, for example at: /home/<USER>/logrotate.conf
The file should contain the following:
/home/<USER>/timeline/logs/* {
size 1G
copytruncate
rotate 1
}
The path should point to the log file generated by the docker-compose up command. This configuration works the same way as described in the previous case. - Register a cron job to run the log rotation procedure once a day.
Run the following command to create a user-specific cron job:
crontab -e
This will open a text editor where you can register cron jobs by adding lines like the following:
0 * * * * /usr/sbin/logrotate /home/$USER/logrotate.conf --state /home/$USER/logrotate-state.txt
This line will result in the logrotate command executed once per day using the previously defined log rotation configuration, and storing its state in logratate-state.txt (this can be any file, and doesn't have to exist at the beginning)
Important. For log rotation to work correctly, you have to write the log files in append mode (in bash '>>' instead of just '>'), otherwise the log file cannot be cut, since the process would keep writing at its current offset at the location that used to be the end of the file, even after the file was cleared.
9/5/2024 4:23:54 PM