journalctl disk usage — vacuum bails, SystemMaxUse caps

journalctl disk usage — vacuum bails, SystemMaxUse caps

journalctl disk usage explained with receipts: what --disk-usage counts, why --vacuum-size refills itself, and the one drop-in that closes the tap for good.

HUMANby @mrkeepalive2d9 min read
the journal (the tub)/var/log/journalgrows on every log line48.2M → 56.2M measured the tap (open)no SystemMaxUse --vacuum-size (the bail)56.2M → 16M → refilled 20.3M valve wheelSystemMaxUse=5M capped journalsame flood → 1.2M, held
Fig 1 — vacuum scoops water out once; the tap keeps pouring. SystemMaxUse is the valve on the pipe.

What does journalctl --disk-usage actually measure?

The claim sentence: journalctl --disk-usage reports the size of your on-disk journal files — archived and active — which live in /var/log/journal on any persistent install. It is a read: it deletes nothing, restarts nothing. On the bench VM, before any logs were added:

$ sudo journalctl --disk-usage
Archived and active journals take up 48.2M in the file system.
$ ls /var/log/journal/
fd766b9ea5a540e68e6c43ea74f98a15   ← one directory per machine-id

verify: stock Arch ships /var/log/journal present — Storage=auto (the commented #Storage=persistent default) means persistent from first boot. Run it with sudo: without membership in adm/systemd-journal/wheel, a plain user's read covers only their own files — the bench measured 16M as the user and 48.2M as root on the same second.

Two things trip people up here. First, the number includes the active journal — the file journald is writing to right now — so even a fresh boot is never zero. Second, if /var/log/journal doesn't exist, journald is volatile: everything lives in /run/log/journal (tmpfs), reads near-zero, and dies at reboot. That's not "a small journal" — that's no journal history at all. sudo mkdir -p /var/log/journal && sudo systemctl restart systemd-journald flips you to persistent.

Why does the journal keep growing?

Because the tap is open by design. The stock /etc/systemd/journald.conf ships every size knob commented out:

$ grep -E '^#(Storage|Compress|SystemMaxUse)' /etc/systemd/journald.conf
#Storage=persistent
#Compress=yes
#SystemMaxUse=

verify: pasted from the bench box — SystemMaxUse exists in the file only as a comment; journald reads it as unset and enforces no ceiling.

With no ceiling, nothing ages out and nothing caps: a chatty service or one noisy cron adds megabytes per day, forever. The bench proved the slope the honest way — flood it, force a rotation, and read the meter again:

$ for i in $(seq 1 400); do logger -t benchflood "entry $i: …~1KB line…"; done
$ sudo journalctl -t benchflood --no-pager | wc -l     # all 400 landed
400
$ sudo journalctl --rotate                             # close the active file
$ sudo journalctl --disk-usage
Archived and active journals take up 56.2M in the file system.

verify: 48.2M → 56.2M from one small flood. Two receipts hide in that: growth only becomes visible at rotation — journald preallocates the active file to 8M chunks, so mid-file floods don't move the meter until the file closes — and du -sh /var/log/journal (57M) agrees with journald's own number within rounding.

The flood was 400 one-kilobyte lines — a rounding error on a real box's daily noise. The size is not the point; the direction is: uncapped means the only steady state is "more", in 8M steps you never chose.

Does journalctl --vacuum-size delete it for good?

No — and the confession is printed in the command's own output. Vacuum is the bail bucket: one scoop, immediately satisfying, structurally nothing:

$ sudo journalctl --vacuum-size=1M
Deleted archived journal …/system@…-adf16c8ac48706f4.journal~ (8M).
Deleted archived journal …/system@d1d286….journal (8.2M).
Deleted archived journal …/user-1000@…journal~ (8M).
Deleted archived journal …/system@…c3f9281226bbfdc4.journal~ (8M).
Deleted archived journal …/system@…5244-….journal (4.4M).
Deleted archived journal …/user-1000@…57b6-….journal (3.4M).
Deleted archived journal …/system@…59c8-….journal (976K).
Deleted archived journal …/user-1000@…59cb-….journal (296K).
Vacuuming done, freed 41.4M of archived journals from /var/log/journal/fd766b9e….
$ sudo journalctl --disk-usage
Archived and active journals take up 16M in the file system.

verify: the deletion list and "Vacuuming done, freed 41.4M" are --vacuum-size's own stdout, pasted verbatim — 56.2M fell to 16M.

Look hard at the floor, because it's the part the cron jobs never notice: 16M stayed. That's the two active journal files — system.journal and user-1000.journal, preallocated 8M each — and --vacuum-size only ever deletes archived files. Vacuum cannot cut below what journald is currently writing into, no matter how small the number you pass.

Then the refill test — the part nobody runs. The same flood again, rotate, read:

$ sudo journalctl --disk-usage
Archived and active journals take up 20.3M in the file system.
$ sudo du -sh /var/log/journal
21M	/var/log/journal

verify: 16M → 20.3M from an identical 400-line flood — vacuum deleted history; it changed nothing about tomorrow.

This is why the vacuum cron is a folk remedy that never quite works: it fights the tap with a bucket, on a schedule, forever. If the cron runs --vacuum-size=100M weekly and the journal hits 1G mid-week, readers pay the 1G all week. And every pass is a quiet data-loss event — the freed lines are the oldest ones, exactly the history you reach for during an incident.

How do you cap journal size permanently?

One drop-in, one restart — then journald enforces the ceiling itself, on every rotation, forever:

$ sudo mkdir -p /etc/systemd/journald.conf.d
$ sudo tee /etc/systemd/journald.conf.d/size.conf >/dev/null <<'EOF'
[Journal]
SystemMaxUse=5M
EOF
$ sudo systemctl restart systemd-journald
$ sudo systemd-analyze cat-config systemd/journald.conf | grep SystemMaxUse
SystemMaxUse=5M

verify: systemd-analyze cat-config prints the MERGED config — the drop-in's SystemMaxUse=5M is live, not just written to a file nobody reads.

Now the identical flood that lifted the vacuumed journal to 20.3M:

$ for i in $(seq 1 400); do logger -t benchflood "entry $i: …"; done
$ sudo journalctl --rotate
$ sudo journalctl --disk-usage
Archived and active journals take up 1.2M in the file system.
$ sudo ls -lh /var/log/journal/*/ | awk '{print $5, $9}'
640K system.journal
640K user-1000.journal
$ sudo journalctl -t benchflood --no-pager | wc -l
0

verify: the same flood that grew an uncapped journal to 20.3M left the capped one at 1.2M — journald rotated, applied the ceiling, and deleted the flood outright: 400 entries in, 0 retained, no cron, no supervision.

Note what the cap did that vacuum couldn't: it went below the 16M floor. Vacuum never touches active files, but the cap governs rotation itself — journald rebuilds smaller active files once the ceiling is live. That's the difference between bailing and plumbing.

Pick your number by headroom, not by taste: check today's size with journalctl --disk-usage, then set SystemMaxUse to roughly the most you'd ever want to keep — 100M or 200M is plenty for a workstation; a busy server might want 1G. SystemKeepFree=1G is the belt-and-suspenders sibling: cap the journal and promise the filesystem a floor. The deleted material is always the oldest archived files — journald keeps the active file and whatever fits under your ceiling.

The cure, in order

Bail vs valve, at a glance

toolwhat it doeswhen it helpsmeasured on the bench
--vacuum-size=1Mdeletes oldest archived journals down to a size, onceemergency: disk full now56.2M → 16M (freed 41.4M), refilled to 20.3M by the next identical flood
--vacuum-time=2weekssame, by age instead of sizeone-time archive cleanupsame one-shot shape — nothing stops regrowth
SystemMaxUse=5M drop-injournald self-enforces a ceiling at every rotationpermanent: the tap closesidentical flood held at 1.2M — flood entries deleted outright (400 in, 0 kept)
SystemKeepFree=1Gpromise the filesystem free space, regardless of capsmall disks, alongside the capstock-default spirit, now explicit
"Vacuum bails the tub; SystemMaxUse closes the tap. Everyone bails — the tub wins by default."
what the bench did not cover

Measured: one fresh Arch VM (systemd 262, 262.1-1-arch) — baseline usage, a 400-line flood with forced rotation, --vacuum-size=1M with its full stdout, an identical refill flood, the SystemMaxUse=5M drop-in + restart with a merged-config readback, an identical flood under the cap, and file-level du/ls witnesses at every step. Not measured: SystemMaxFileSize/file-count interplay under the ceiling, per-unit LogNamespace= journals, remote logging (systemd-journal-remote), forward-secure sealing (Seal=), and time-based retention (MaxRetentionSec). Flood lines rode the syslog socket, which truncates long lines (~1 KB landed) — irrelevant to the accounting, since every number above is journald's own --disk-usage cross-checked against du. The baseline carried a previous boot's archives (a lived-in 48.2M, like any real box). One machine, one build: the tap-vs-bail reading is the portable part; your own --disk-usage is the verdict.

HUMAN-VERIFIED BENCH · systemd 262 (262.1-1-arch) · fresh Arch VM · last tested 2026-11-12

One number tells you the truth — journalctl --disk-usage. One line keeps it true — SystemMaxUse. Close the tap; retire the bucket. ■