thallada
ce4a1a127d
The previous text claimed restructuring main.rs to use the lib crate would 'roughly halve' build time. Measured with --timings on a clean release build, that was wrong: 33.7s keydr (bin) 26.0s keydr (lib) 1.4s generate_test_profiles (bin) 61.0s of 292.2s total unit-seconds (21%) The duplicated units are indeed the two slowest in the graph, but the saving is ~26s of unit-work (less in wall time, since they overlap with dependency compilation across 3 jobs), not half the build. The other ~79% is one-time dependency compilation that incremental builds skip. Replace the estimate with the measurement.
210 lines
7.5 KiB
Markdown
210 lines
7.5 KiB
Markdown
# Building keydr
|
|
|
|
## TL;DR
|
|
|
|
```bash
|
|
cargo build
|
|
cargo test
|
|
cargo build --release
|
|
```
|
|
|
|
Plain cargo. No wrapper, no memory ceiling, no reduced optimisation.
|
|
A clean release build peaks at **1.92 GB** and takes **2m17s** on a
|
|
4-core / 7.6 GB VM.
|
|
|
|
If that's not what you're seeing, read on.
|
|
|
|
---
|
|
|
|
## The build used to be unusable — here's what was actually wrong
|
|
|
|
Release builds consumed 5+ GB and never finished; the VM thrashed so
|
|
badly that SSH stopped responding and needed a console reboot.
|
|
|
|
The obvious explanations were all wrong:
|
|
|
|
- **Not too many parallel jobs.** The peak came from a *single* rustc
|
|
process. A single process's memory is unaffected by `--jobs`.
|
|
- **Not insufficient RAM.** 8 GB is fine for a project this size.
|
|
- **Not LTO** (though LTO made it worse).
|
|
|
|
**The cause was a codegen bug in `rust-i18n` v3.** Its macro emitted one
|
|
`HashMap::from([...])` construction *and* one `add_translations()` call
|
|
**per translation string** — all inside a single function body. With 21
|
|
locales and ~9,000 keys that's 8,652 separate HashMap constructions in
|
|
one function. LLVM's optimiser scales superlinearly on function size, so
|
|
it consumed ~4.5 GB trying to optimise that one initialiser.
|
|
|
|
Verified by expanding the macro:
|
|
|
|
| | rust-i18n v3.1.5 | rust-i18n v4.2.1 |
|
|
|---|---|---|
|
|
| `HashMap::from` constructions | 8,652 | **0** |
|
|
| `add_translations` calls | 8,652 | **21** (one per locale) |
|
|
| Expanded lines (lib target) | 93,026 | 73,444 |
|
|
|
|
Upstream fixed this in **v4.0.0**. Reproduce the check yourself:
|
|
|
|
```bash
|
|
cargo install cargo-expand
|
|
cargo expand --lib | grep -c 'HashMap::from'
|
|
```
|
|
|
|
### The fix
|
|
|
|
```bash
|
|
cargo add rust-i18n@4
|
|
```
|
|
|
|
That's it. Measured on this box, same machine, clean builds:
|
|
|
|
| Configuration | Peak | Time |
|
|
|---|---|---|
|
|
| v3, `opt-level=3`, thin LTO | 5.2 GB | stalled >18 min, never finished |
|
|
| v3, `opt-level=3`, no LTO | 4.5 GB | stalled >12 min, never finished |
|
|
| v3, `opt-level=1`, no LTO (workaround) | 1.8 GB | 15m30s |
|
|
| **v4, `opt-level=3`, thin LTO** | **2.08 GB** | **3m25s** |
|
|
| **v4, stock cargo defaults** | **1.92 GB** | **2m17s** |
|
|
|
|
The earlier workaround — dropping `opt-level` to 1 and disabling LTO —
|
|
has been **removed**. It was treating a symptom.
|
|
|
|
---
|
|
|
|
## What's still in `.cargo/config.toml`, and why
|
|
|
|
Nothing there reduces the optimisation of shipped code.
|
|
|
|
- **`jobs = 3`** — uses 3 of 4 cores so the box stays interactive while
|
|
compiling. Purely about responsiveness; delete it if you don't care.
|
|
- **`lld` linker** — ships with the Rust toolchain, no install needed.
|
|
GNU ld is single-threaded and holds the whole link graph in memory.
|
|
- **`debug = 1` for this crate, `debug = 0` for dependencies** (dev/test
|
|
profiles only). Debug info is the largest contributor to `target/` size
|
|
and link time. Level 1 keeps line numbers, so backtraces still work.
|
|
|
|
`[profile.release]` is deliberately absent — cargo's defaults are right.
|
|
|
|
---
|
|
|
|
## Known remaining inefficiency
|
|
|
|
`src/main.rs` re-declares the whole module tree (`mod app; mod config;
|
|
...`) instead of depending on the `keydr` lib target, so **every module
|
|
compiles twice** — once for the lib, once for the bin. The `i18n!` macro
|
|
must therefore be invoked in both crate roots (`t!()` expands to
|
|
`crate::_rust_i18n_t`, so it must exist in each crate).
|
|
|
|
Measured with `cargo build --release --timings` on a clean tree, the two
|
|
duplicated units are the slowest in the whole graph:
|
|
|
|
```
|
|
33.7s keydr (bin)
|
|
26.0s keydr (lib)
|
|
1.4s generate_test_profiles (bin)
|
|
------
|
|
61.0s of 292.2s total unit-seconds (21%)
|
|
```
|
|
|
|
So restructuring `main.rs` to `use keydr::...` would save roughly 26s of
|
|
compile work — real, but nowhere near half the build. Wall-clock saving is
|
|
smaller still, since those units partly overlap with dependency
|
|
compilation across 3 parallel jobs (292s of unit-work compressed into
|
|
~2m15s wall). Worth doing for code hygiene; not a build-performance
|
|
emergency.
|
|
|
|
Note the remaining ~79% is dependency compilation (`reqwest` 12.7s,
|
|
`clap_builder` 11.4s, `toml_edit` 9.3s, `tokio` 8.0s...), which is
|
|
one-time work that incremental builds skip entirely.
|
|
|
|
---
|
|
|
|
## If a build ever does go wild again
|
|
|
|
The machine-level protections below need no per-project changes.
|
|
|
|
### Keep SSH alive (recommended, needs root once)
|
|
|
|
```bash
|
|
# Reserve memory for the SSH daemon so it can't be swapped out entirely.
|
|
sudo systemctl edit ssh # [Service] / MemoryMin=128M
|
|
```
|
|
|
|
**Do not bother with `systemd-oomd` on this box** — it is a separate
|
|
package whose only reverse-dependencies are `ubuntu-desktop*`, so Ubuntu
|
|
Server never installs it (`systemctl is-enabled systemd-oomd` → `not-found`
|
|
here). Use **earlyoom** instead, which is in `universe` and kills only the
|
|
single highest-scoring process rather than a whole cgroup:
|
|
|
|
```bash
|
|
sudo apt install earlyoom
|
|
# /etc/default/earlyoom
|
|
EARLYOOM_ARGS="-m 8 -s 5 -r 60 \
|
|
--avoid '(^|/)(systemd|sshd|mosh-server|tmux.*|bash|fish)$' \
|
|
--prefer '(^|/)(rustc|cargo|ld|lld|collect2)$'"
|
|
```
|
|
|
|
`--avoid` protects your shell; `--prefer` points it at the compiler. Fedora
|
|
enabled earlyoom by default for exactly this "system becomes completely
|
|
unresponsive, user has no choice but to force power off" scenario.
|
|
|
|
### Cap every build at once, forever (needs root once)
|
|
|
|
This is the global knob — no wrapper script, no per-project config:
|
|
|
|
```bash
|
|
sudo systemctl set-property user-1000.slice MemoryHigh=5G MemoryMax=6500M
|
|
```
|
|
|
|
Applies to every shell and every build you start, persists across reboots.
|
|
Because `sshd` itself lives in `system.slice`, capping the user slice can
|
|
never lock you out of a new login. (Ubuntu already ships `TasksMax=33%`
|
|
this way via `/usr/lib/systemd/system/user-.slice.d/`, so it's a
|
|
distro-blessed pattern.)
|
|
|
|
Note `MemoryHigh` at the *slice* level is reasonable — it throttles a
|
|
sprawling session gradually. Do not put it on a single short-lived build
|
|
scope, where it causes the reclaim-stall described below.
|
|
|
|
### Cap one build ad-hoc (no root)
|
|
|
|
`memory` is delegated to the user slice on this box, so you can confine a
|
|
single command without any wrapper script:
|
|
|
|
```bash
|
|
systemd-run --user --scope -p MemoryMax=4G -p MemorySwapMax=0 \
|
|
cargo build --release
|
|
```
|
|
|
|
`MemorySwapMax=0` is the important half. The lockup was never really an
|
|
OOM — it was *thrashing*. With swap available the kernel pages `sshd` out
|
|
to feed the build and the box goes catatonic while `oom_kill` stays at 0.
|
|
Denying the build swap turns a slow total failure into a fast contained one.
|
|
|
|
**Do not add `MemoryHigh` to a single build scope.** It sounds safer but
|
|
traps the process in continuous reclaim so it grinds forever instead of
|
|
dying. Measured against an identical 256 MB ceiling: `MemoryMax` alone →
|
|
clean kill in seconds; `MemoryMax` + `MemoryHigh` → still spinning after 45
|
|
seconds.
|
|
|
|
### Diagnosing which crate is expensive
|
|
|
|
```bash
|
|
cargo build --release --timings # HTML report in target/cargo-timings/
|
|
cargo install cargo-llvm-lines
|
|
cargo llvm-lines --release | head -30 # which functions generate the most IR
|
|
cargo expand --lib | wc -l # how much code a macro really emits
|
|
```
|
|
|
|
`cargo llvm-lines` and `cargo expand` are what actually found this bug.
|
|
Reach for them before touching `opt-level`.
|
|
|
|
## Upstream context
|
|
|
|
Cargo has no memory-aware job scheduling; it schedules on core count only.
|
|
That's a known gap ([rust-lang/cargo#12912](https://github.com/rust-lang/cargo/issues/12912)),
|
|
and maintainers have said they'd rather delegate resource limits to the OS
|
|
(cgroups) than build it into cargo. So the systemd approach above isn't a
|
|
hack — it's the sanctioned answer. But in this case none of it was needed:
|
|
the real fix was a dependency upgrade.
|