devutils/ATFQ/book ATFQ git

Part II · Your system

26. Logs

hub

What happened on this machine, where it's written down, and how to keep it from filling the disk.

On your machine

alice@laptopexample
$ atfq logs
  Journal   on disk (/var/log/journal): kept across reboots · 300 B in 2
            files
  Settings  SystemMaxUse=500M
  Syslog    rsyslog also writes text files in /var/log
  Rotation  2 rules in /etc/logrotate.d
  Biggest   /var/log/syslog 4.9 KiB · /var/log/nginx/access.log 2.9 KiB ·
            /var/log/auth.log 1.2 KiB

An example: Alice's laptop, one of the test machines. In your terminal, this block shows your own machine, read live.

Commands

journalctl -b -p warning
warnings and errors since boot 26.1
journalctl -u nginx -f
follow one service's log 26.1
journalctl -b -1 -e
how the previous boot ended 26.2
journalctl --disk-usage
space the journal takes 26.3
sudo dmesg -w
kernel messages as they arrive 21.3
ls -lt /var/log | head
text logs written most recently 26.2

Paths

/var/log/journal/
the journal, when it's kept on disk
/var/log/
text logs from rsyslog and programs
/etc/systemd/journald.conf
journal storage and size limits
/etc/logrotate.d/
rotation rules for text logs

Cards

  1. 26.1Reading the journal with journalctlsystemd's journal collects everything: the kernel, every service's output, and programs that log through syslog. One command reads it.
  2. 26.2Where logs go: journald, syslog and /var/logjournald receives the logs. It keeps them on disk in /var/log/journal/ if that folder exists, otherwise in RAM under /run/log/journal/, lost at reboot.
  3. 26.3Log rotation and logs that fill the diskjournald limits itself: by default up to 10 % of its filesystem, at most 4 GiB. SystemMaxUse= changes it.
  4. 26.4Logging from your own programs and servicesUnder systemd, a service only has to print to standard output or error: journald stores each line with the time, unit and PID.

Also about this

  1. 21.3dmesg and the kernel logThe kernel keeps its messages in a buffer in RAM: hardware found, driver errors, OOM kills, a USB stick plugged in.