diff --git a/docs/BUILDING.md b/docs/BUILDING.md index a43c208..177de49 100644 --- a/docs/BUILDING.md +++ b/docs/BUILDING.md @@ -110,16 +110,43 @@ The machine-level protections below need no per-project changes. ```bash # Reserve memory for the SSH daemon so it can't be swapped out entirely. sudo systemctl edit ssh # [Service] / MemoryMin=128M - -# Kill on memory *pressure* rather than waiting for true OOM. This reacts -# during thrashing — exactly the window where the box currently becomes -# unreachable. Ubuntu ships it but leaves it inactive. -sudo systemctl enable --now systemd-oomd ``` -`earlyoom` is the lighter-weight alternative (`sudo apt install earlyoom`); -it kills only the single highest-scoring process, which during a build is -`rustc` itself. +**Do not bother with `systemd-oomd` on this box** — it is a separate +package whose only reverse-dependencies are `ubuntu-desktop*`, so Ubuntu +Server never installs it (`systemctl is-enabled systemd-oomd` → `not-found` +here). Use **earlyoom** instead, which is in `universe` and kills only the +single highest-scoring process rather than a whole cgroup: + +```bash +sudo apt install earlyoom +# /etc/default/earlyoom +EARLYOOM_ARGS="-m 8 -s 5 -r 60 \ + --avoid '(^|/)(systemd|sshd|mosh-server|tmux.*|bash|fish)$' \ + --prefer '(^|/)(rustc|cargo|ld|lld|collect2)$'" +``` + +`--avoid` protects your shell; `--prefer` points it at the compiler. Fedora +enabled earlyoom by default for exactly this "system becomes completely +unresponsive, user has no choice but to force power off" scenario. + +### Cap every build at once, forever (needs root once) + +This is the global knob — no wrapper script, no per-project config: + +```bash +sudo systemctl set-property user-1000.slice MemoryHigh=5G MemoryMax=6500M +``` + +Applies to every shell and every build you start, persists across reboots. +Because `sshd` itself lives in `system.slice`, capping the user slice can +never lock you out of a new login. (Ubuntu already ships `TasksMax=33%` +this way via `/usr/lib/systemd/system/user-.slice.d/`, so it's a +distro-blessed pattern.) + +Note `MemoryHigh` at the *slice* level is reasonable — it throttles a +sprawling session gradually. Do not put it on a single short-lived build +scope, where it causes the reclaim-stall described below. ### Cap one build ad-hoc (no root) @@ -136,10 +163,11 @@ OOM — it was *thrashing*. With swap available the kernel pages `sshd` out to feed the build and the box goes catatonic while `oom_kill` stays at 0. Denying the build swap turns a slow total failure into a fast contained one. -**Do not add `MemoryHigh`.** It sounds safer but traps the process in -continuous reclaim so it grinds forever instead of dying. Measured against -an identical 256 MB ceiling: `MemoryMax` alone → clean kill in seconds; -`MemoryMax` + `MemoryHigh` → still spinning after 45 seconds. +**Do not add `MemoryHigh` to a single build scope.** It sounds safer but +traps the process in continuous reclaim so it grinds forever instead of +dying. Measured against an identical 256 MB ceiling: `MemoryMax` alone → +clean kill in seconds; `MemoryMax` + `MemoryHigh` → still spinning after 45 +seconds. ### Diagnosing which crate is expensive