keydr/docs/BUILDING.md

# Building keydr

## TL;DR

```bash
cargo build
cargo test
cargo build --release
```

Plain cargo. No wrapper, no memory ceiling, no reduced optimisation.
A clean release build peaks at **1.92 GB** and takes **2m17s** on a
4-core / 7.6 GB VM.

If that's not what you're seeing, read on.

---

## The build used to be unusable — here's what was actually wrong

Release builds consumed 5+ GB and never finished; the VM thrashed so
badly that SSH stopped responding and needed a console reboot.

The obvious explanations were all wrong:

- **Not too many parallel jobs.** The peak came from a *single* rustc
  process. A single process's memory is unaffected by `--jobs`.
- **Not insufficient RAM.** 8 GB is fine for a project this size.
- **Not LTO** (though LTO made it worse).

**The cause was a codegen bug in `rust-i18n` v3.** Its macro emitted one
`HashMap::from([...])` construction *and* one `add_translations()` call
**per translation string** — all inside a single function body. With 21
locales and ~9,000 keys that's 8,652 separate HashMap constructions in
one function. LLVM's optimiser scales superlinearly on function size, so
it consumed ~4.5 GB trying to optimise that one initialiser.

Verified by expanding the macro:

| | rust-i18n v3.1.5 | rust-i18n v4.2.1 |
|---|---|---|
| `HashMap::from` constructions | 8,652 | **0** |
| `add_translations` calls | 8,652 | **21** (one per locale) |
| Expanded lines (lib target) | 93,026 | 73,444 |

Upstream fixed this in **v4.0.0**. Reproduce the check yourself:

```bash
cargo install cargo-expand
cargo expand --lib | grep -c 'HashMap::from'
```

### The fix

```bash
cargo add rust-i18n@4
```

That's it. Measured on this box, same machine, clean builds:

| Configuration | Peak | Time |
|---|---|---|
| v3, `opt-level=3`, thin LTO | 5.2 GB | stalled >18 min, never finished |
| v3, `opt-level=3`, no LTO | 4.5 GB | stalled >12 min, never finished |
| v3, `opt-level=1`, no LTO (workaround) | 1.8 GB | 15m30s |
| **v4, `opt-level=3`, thin LTO** | **2.08 GB** | **3m25s** |
| **v4, stock cargo defaults** | **1.92 GB** | **2m17s** |

The earlier workaround — dropping `opt-level` to 1 and disabling LTO —
has been **removed**. It was treating a symptom.

---

## What's still in `.cargo/config.toml`, and why

Nothing there reduces the optimisation of shipped code.

- **`jobs = 3`** — uses 3 of 4 cores so the box stays interactive while
  compiling. Purely about responsiveness; delete it if you don't care.
- **`lld` linker** — ships with the Rust toolchain, no install needed.
  GNU ld is single-threaded and holds the whole link graph in memory.
- **`debug = 1` for this crate, `debug = 0` for dependencies** (dev/test
  profiles only). Debug info is the largest contributor to `target/` size
  and link time. Level 1 keeps line numbers, so backtraces still work.

`[profile.release]` is deliberately absent — cargo's defaults are right.

---

## Known remaining inefficiency

`src/main.rs` re-declares the whole module tree (`mod app; mod config;
...`) instead of depending on the `keydr` lib target, so **every module
compiles twice** — once for the lib, once for the bin. The `i18n!` macro
must therefore be invoked in both crate roots (`t!()` expands to
`crate::_rust_i18n_t`, so it must exist in each crate).

With v4 this duplication is cheap (21 calls, not 8,652), so it's no
longer worth fixing for build performance alone. If you ever restructure
`main.rs` to `use keydr::...`, build time would roughly halve again.

---

## If a build ever does go wild again

The machine-level protections below need no per-project changes.

### Keep SSH alive (recommended, needs root once)

```bash
# Reserve memory for the SSH daemon so it can't be swapped out entirely.
sudo systemctl edit ssh        # [Service] / MemoryMin=128M
```

**Do not bother with `systemd-oomd` on this box** — it is a separate
package whose only reverse-dependencies are `ubuntu-desktop*`, so Ubuntu
Server never installs it (`systemctl is-enabled systemd-oomd` → `not-found`
here). Use **earlyoom** instead, which is in `universe` and kills only the
single highest-scoring process rather than a whole cgroup:

```bash
sudo apt install earlyoom
# /etc/default/earlyoom
EARLYOOM_ARGS="-m 8 -s 5 -r 60 \
  --avoid '(^|/)(systemd|sshd|mosh-server|tmux.*|bash|fish)$' \
  --prefer '(^|/)(rustc|cargo|ld|lld|collect2)$'"
```

`--avoid` protects your shell; `--prefer` points it at the compiler. Fedora
enabled earlyoom by default for exactly this "system becomes completely
unresponsive, user has no choice but to force power off" scenario.

### Cap every build at once, forever (needs root once)

This is the global knob — no wrapper script, no per-project config:

```bash
sudo systemctl set-property user-1000.slice MemoryHigh=5G MemoryMax=6500M
```

Applies to every shell and every build you start, persists across reboots.
Because `sshd` itself lives in `system.slice`, capping the user slice can
never lock you out of a new login. (Ubuntu already ships `TasksMax=33%`
this way via `/usr/lib/systemd/system/user-.slice.d/`, so it's a
distro-blessed pattern.)

Note `MemoryHigh` at the *slice* level is reasonable — it throttles a
sprawling session gradually. Do not put it on a single short-lived build
scope, where it causes the reclaim-stall described below.

### Cap one build ad-hoc (no root)

`memory` is delegated to the user slice on this box, so you can confine a
single command without any wrapper script:

```bash
systemd-run --user --scope -p MemoryMax=4G -p MemorySwapMax=0 \
    cargo build --release
```

`MemorySwapMax=0` is the important half. The lockup was never really an
OOM — it was *thrashing*. With swap available the kernel pages `sshd` out
to feed the build and the box goes catatonic while `oom_kill` stays at 0.
Denying the build swap turns a slow total failure into a fast contained one.

**Do not add `MemoryHigh` to a single build scope.** It sounds safer but
traps the process in continuous reclaim so it grinds forever instead of
dying. Measured against an identical 256 MB ceiling: `MemoryMax` alone →
clean kill in seconds; `MemoryMax` + `MemoryHigh` → still spinning after 45
seconds.

### Diagnosing which crate is expensive

```bash
cargo build --release --timings      # HTML report in target/cargo-timings/
cargo install cargo-llvm-lines
cargo llvm-lines --release | head -30 # which functions generate the most IR
cargo expand --lib | wc -l            # how much code a macro really emits
```

`cargo llvm-lines` and `cargo expand` are what actually found this bug.
Reach for them before touching `opt-level`.

## Upstream context

Cargo has no memory-aware job scheduling; it schedules on core count only.
That's a known gap ([rust-lang/cargo#12912](https://github.com/rust-lang/cargo/issues/12912)),
and maintainers have said they'd rather delegate resource limits to the OS
(cgroups) than build it into cargo. So the systemd approach above isn't a
hack — it's the sanctioned answer. But in this case none of it was needed:
the real fix was a dependency upgrade.