Part II · Your system
26.3 Log rotation and logs that fill the disk
alsologrotatelog rotationjournal sizesystemmaxusevacuumjournalctl --vacuum-sizejournalctl --disk-usagelog too biglogs filling diskcopytruncatedocker logs size
TL;DR
- journald limits itself: by default up to 10 % of its filesystem, at
most 4 GiB.
SystemMaxUse=changes it. - logrotate handles text logs: on a schedule it renames the file, starts a new one, compresses old ones and deletes the oldest.
- A log that grows without limit usually has no rotation rule, or a program writing too much.
journalctl --disk-usage
sudo journalctl --vacuum-size=500M # shrink the journal now
sudo logrotate -d /etc/logrotate.conf # dry run: what would rotate
On your machine
$ atfq 26.3 Journal on disk (/var/log/journal): kept across reboots · 300 B in 2 files Settings SystemMaxUse=500M Syslog rsyslog also writes text files in /var/log Rotation 2 rules in /etc/logrotate.d Biggest /var/log/syslog 4.9 KiB · /var/log/nginx/access.log 2.9 KiB · /var/log/auth.log 1.2 KiB
An example: Alice's laptop, one of the test machines. In your terminal, this block shows your own machine, read live.
Pitfalls
- Deleting a big log a program still writes to frees nothing: the file
stays open. Empty it instead (
: > file), or restart the program;sudo lsof +L1lists deleted files still held open. - After rotation, a program writing to the renamed file must be told to
reopen (
postrotatewith a reload), or rotation must usecopytruncate. - Docker's default log driver keeps container output forever: set
max-sizeandmax-filein/etc/docker/daemon.json.
Deep dive 2 questions
What does a logrotate rule look like?
/var/log/myapp/*.log {
weekly
rotate 4
compress
missingok
postrotate
systemctl reload myapp
endscript
}
Four weeks are kept, compressed, then deleted. A systemd timer (or cron) runs logrotate daily; each rule decides how often it acts.
Why do journald limits sometimes not apply?
They count only the journal's own files, archived ones first: the file
being written stays until it's rotated. Limits are checked as it writes,
so after lowering SystemMaxUse=, run ⚠ --vacuum-size or restart
journald to shrink at once.