Web dashboard: briefs for steps 2-7

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM

docs/plans/briefs/web-dashboard/02-full-issues.md

@@ -0,0 +1,47 @@
+# Step 2 — Full issue views, rating widget, POST /rate
+
+Read `00-shared.md`, then the plan `docs/plans/2026-09-03-web-dashboard.md`
+(§3, §4.3–§4.4, §5.4, §6.3–§6.4, §8 including §8.1, §16, §17), then
+`handoff-step1.md` for what the foundation actually shipped. This is plan §19
+item 2. Build on the step 1 code as it exists; do not restructure it.
+
+## Deliverables
+
+1. `issue_full.html` (§8): masthead + dateline, The Brief (`front_page_html`
+   `|safe`), download buttons (from `IssueView.downloads`), the per-section
+   index like the EPUB's In-this-issue page (title → article page,
+   `source · N min read`, summary, Why it's here), the rating widget for admins,
+   links to World Briefing / Behind the paper (only when present), colophon
+   facts footer. `/` and `/issues/{date}` render this for any signed-in viewer.
+2. `article.html` at `/issues/{date}/articles/{article_id}` (§8): header
+   (title → source, byline, meta line, Why, social line via
+   `chapters::social_line`, summary), body from `ammonia::clean` with the same
+   configuration the EPUB uses (find it in `src/html.rs`/`src/epub`; do not run
+   `to_xhtml`), `<img>` keep remote `src` and get `loading="lazy"
+   referrerpolicy="no-referrer"`, the discussion via `comments::render_xhtml`
+   when present, prev/next in issue order, "Read online ↗", rating widget for
+   admins. 404 (site error page) when the article is not in that issue.
+3. `world.html` (`/issues/{date}/world`, `world::render_xhtml`, 404 when
+   absent) and `behind.html` (`/issues/{date}/behind`, the same lines as the
+   EPUB chapter via `chapters::behind_*_line`; near misses link to
+   `/dashboard/articles/{id}` for admins).
+4. These three routes sit under the `login_required!` layer (§3).
+5. The HMAC confirmation page (`server::confirmation_page`) gains the
+   "Open this issue on the site" link (§8).
+6. Rating widget partial `_rating_widget.html` + `src/web/rate.rs` (§8.1):
+   `POST /rate` under the admin layer and the origin check; form and JSON
+   variants; `RatingEvent { source: "dashboard", user_id: Some(viewer.id) }`;
+   `cleared` writes value 0.0 with label `cleared`; `issue_date` optional with
+   the dashboard fallback (`db::latest_issue_date_for_article`); JSON →
+   `200 {"article_id","label","event_id"}`, form → flash + 303 to a validated
+   same-site `next`. Check how the CLI (`main.rs` `cmd_ratings`) writes the
+   `cleared` event and match it exactly.
+7. `app.js`: intercept `.rating` forms, `fetch` with
+   `credentials: "same-origin"` and `Accept: application/json`, update the
+   active button; keep the no-JS form path working.
+8. Tests (§17 "Full rendering", "Rating"): signed-in `/issues/{date}` contains
+   Brief, summaries and why; article page contains body and discussion; the
+   fallback loader (no `issue_json`) renders without world/discussion links;
+   downloads listed only for files that exist; `POST /rate` as admin appends a
+   `dashboard` event with `user_id`, as user → 403, anonymous → redirect/401,
+   JSON and form variants, `next` validated, `cleared` value 0.

docs/plans/briefs/web-dashboard/03-dashboard-reads.md

@@ -0,0 +1,53 @@
+# Step 3 — Dashboard reads: overview, runs, articles
+
+Read `00-shared.md`, the plan (§3, §4, §9 in full, §16, §17 "Dashboard
+queries"), the curation v2 plan sections it cites
+(`docs/plans/2026-09-02-personalized-curation-v2.md` §7 tables, §9–§13
+signals/admission/utility/editor, §15 explain/stats) and the handoffs for
+steps 1–2. This is plan §19 item 3. Study `src/curate/telemetry.rs`
+(`ExplainRow`, `resolve_run`, `explain_row`, `near_misses`, `render_explain`,
+`SignalsJson`) — the dashboard renders the same data as HTML and must reuse
+those loaders/types where they fit rather than re-deriving them.
+
+## Deliverables
+
+1. `src/web/dashboard/mod.rs` — the overview (§9.1): last-run card from
+   `runs.report_json` (counters fallback), budget-today card per provider
+   (`config.referenced_providers()` + voyage, `db::provider_spend_for_utc_day`),
+   ratings this week by label, "Unrated picks" (last three issues' picks with
+   no explicit event, with the inline rating widget from step 2), jobs summary
+   (the `jobs` table; the pages come in step 6), config-on-disk `! ` lines from
+   `Config::check_report`. Sparklines are step 6 — leave a clearly marked
+   placeholder section or omit.
+2. `src/web/dashboard/runs.rs` (§9.2): the list with `?status=` filter and
+   pagination; the detail with header + prev/next run, the funnel (from
+   `candidate_runs` grouped by stage/reason, in pipeline order, bars via the
+   `.funnel` CSS), admission mix, preference state, timings, provider usage,
+   warnings, per-feed top 20, the **config diff** against the previous non-dry
+   run (flatten both `config_json` documents to dotted keys), near misses via
+   `telemetry::near_misses`, and the candidates table with the allow-listed
+   filters (`stage`, `reason`, `admitted_by` via
+   `json_extract(admitted_by,'$[0]')`, `q`, `flag`) and sorts (utility default,
+   rank, triage, quality, fit, title), 100 per page, a `<details>` per row with
+   `_signals_table.html` built from `signals_json`.
+3. `src/web/dashboard/articles.rs` (§9.3): the list with the latest
+   `candidate_runs` row join (the index `idx_candidate_runs_article_run` exists
+   from migration 0004 — confirm), all listed filters and sorts allow-listed,
+   50 per page; the detail with the six blocks in the plan's order (article
+   facts, assessments incl. `provider_rejected`, run history with expandable
+   signals, neighbours/interests, embedding metadata never the vector, rating
+   events). The "Explain (text)" `<details>` wraps `telemetry::render_explain`
+   output in `<pre>`. The rating widget here carries the note field.
+4. Templates: `dashboard/overview.html`, `dashboard/runs.html`,
+   `dashboard/run.html`, `dashboard/articles.html`, `dashboard/article.html`,
+   partials `_signals_table.html`, `_candidate_row.html`. Every table inside
+   `.scroll-x`; badges per stage/reason/label; sticky thead; the `data-filter`
+   client-side filter from §4.4 in `app.js`.
+5. Sort/filter parameters are validated against allow-lists and never
+   interpolated into SQL; an unknown sort falls back to the default, never
+   errors.
+6. Tests (§17 "Dashboard queries"): funnel counts against a seeded
+   `candidate_runs` set; candidate filters/sorts allow-listed; config diff finds
+   changed dotted keys and ignores unchanged; articles list filters; article
+   detail shows assessments, run history and rating events. Router-level tests
+   for the admin guard on each new route.

docs/plans/briefs/web-dashboard/04-ratings-profile.md

@@ -0,0 +1,42 @@
+# Step 4 — Ratings page and profile page
+
+Read `00-shared.md`, the plan (§3, §10, §11, §14.1 for the job names the
+profile page references, §16, §17 "Ratings contributions" and "Profile"), the
+curation v2 plan §6, §9.2–§9.3, §11 (profile/learned adjustments), and the
+handoffs for steps 1–3. This is plan §19 item 4. Reuse `signals::decay`,
+`signals::gate`, `signals::direct_feeds`, `PreferenceState::load/summary`,
+`profile::parse_profile_str`, `profile::parse_interests`,
+`themes::group_into_themes`, `profile::is_stale`; make
+`profile::MAX_RATINGS_IN_REBUILD` `pub`.
+
+## Deliverables
+
+1. `src/web/dashboard/ratings.rs` + `dashboard/ratings.html` (§10): header from
+   `PreferenceState::summary()` with the "no embedding" notice; the "How
+   ratings enter the algorithm" `<details>` with current config values inlined
+   and links to the settings groups (the settings page is step 5; link to
+   `/dashboard/settings#curation.feedback` etc. — anchors the settings page will
+   provide); the **Current** tab with every column in the plan's table (verdict
+   + inline widget with note, article, when·by with username, note, age→decay,
+   neighbour weight or "no embedding", feed credit, in prompt, in rebuild, used
+   last run from `signals_json.neighbours` of the latest non-dry run), filters
+   by label/source/feed/q; the **Events** tab (100 per page, `superseded`
+   marking, filters by label/source/user/date range).
+2. `src/web/dashboard/profile.rs` + `dashboard/profile.html` (§11): the
+   `profile.md` editor (textarea, Save = origin-checked POST; reject empty or
+   > 64 KB; insert the previous content into `profile_versions`; write
+   atomically via `<path>.tmp` + rename preserving mode; flash), the live parsed
+   preview (server-rendered on GET is fine: the passthrough sections and the
+   extracted `## Interests` lines), history with 200-char previews and Restore
+   (`POST /dashboard/profile/restore`), standing OPML interests grouped by
+   theme (read-only, with path and count), the system prompt
+   (`kv.taste_profile` in a collapsed `<pre>`), the learned adjustments block
+   with its age and whether a rebuild is due, and a "Rebuild profile now"
+   button that POSTs to `/dashboard/jobs/profile-rebuild` (step 6 implements
+   the route; render the form now, disabled with a note if `jobs_enabled` is
+   false).
+3. Tests (§17): decay/weight/feed-credit/in-prompt/in-rebuild against
+   hand-checked values; "used last run" counts neighbours in `signals_json`;
+   profile save writes the file and a `profile_versions` row; restore swaps;
+   oversized rejected; the parsed preview matches `parse_profile_str`; admin
+   guard on every new route.

docs/plans/briefs/web-dashboard/05-settings.md

@@ -0,0 +1,49 @@
+# Step 5 — Settings
+
+Read `00-shared.md`, the plan (§3, §4.2 "Config reload", §13 in full, §15,
+§16, §17 "Settings schema" and "Settings writer"), `src/config.rs` in full
+(every struct, `validate`, `load`, `resolve_path`, `check_report`, `ENV_PREFIX`,
+`ENV_SPLIT`, the `DAILY_EPUB_SECRET` alias, the stale-key checks), the README
+configuration table and `config.example.toml`, and the handoffs for steps 1–4.
+This is plan §19 item 5.
+
+## Deliverables
+
+1. `src/web/dashboard/settings.rs`: `schema(config, file) -> Vec<SettingGroup>`
+   derived by walking `toml::Value::try_from(Config::default())` and the
+   current config in parallel (§13.1) — one `SettingField` per leaf with
+   `path`, `group`, `kind`, `current`, `default`, `source`
+   (Default/File/Env(name)), `help`, `restart_required`. Kinds and the static
+   enum table exactly as §13.1 item 3; `SETTINGS_HELP` seeded from the README
+   table and `config.example.toml` comments, one entry per shipped key (test).
+   Groups render in the §13.1 item 5 order; each group gets an `id` anchor equal
+   to its dotted path (step 4 links to these).
+2. `GET /dashboard/settings` (§4.2 reload-on-mtime first; report a file that
+   fails to load with the `!` banner and keep the previous config live),
+   `dashboard/settings.html`: `Secret` fields as set/not set, `Env` fields
+   disabled with the lock note, everything else as an input with the default
+   beside it and a reset affordance; weights note. `POST /dashboard/settings`
+   implements §13.2 exactly: `toml_edit::DocumentMut`, only changed fields
+   written ("explicit beats implicit"), typed by kind, `TextList` as a
+   multi-line array with trailing comma, field errors collected before any
+   write, render → `<path>.tmp.<pid>` → validate with `Config::load(Some(tmp))`
+   → copy permissions → rename → swap `state.config`; one `config_changes` row
+   per changed key; flash with the restart note for `restart_required` keys;
+   `config_path == None` → 400.
+3. Providers (§13.3): remove (refused when an `[llm]` role names it) and add
+   (`[a-z0-9_]+`, must not exist, `ProviderConfig::default()` with the chosen
+   kind and placeholders) through the same validate-and-rename path, with
+   `config_changes` rows.
+4. `GET /dashboard/settings/history` (§13.4), `dashboard/settings_history.html`.
+5. `systemd/daily-epub.service`: add `/etc/daily-epub` to `ReadWritePaths`
+   (the `SupplementaryGroups` line is step 6).
+6. Tests (§17): every leaf of `Config::default()` appears exactly once; every
+   shipped key has help; secrets are `Secret` with no value; env detection uses
+   the derived name (set the env var inside the test with a unique key and
+   restore it); enum options round-trip through `validate()`; starting from
+   `config.example.toml` changing three keys preserves every comment and the
+   order of untouched lines; a new key in an absent table creates the table;
+   `TextList` multi-line; an invalid value (e.g. `deep_keep < shortlist_keep`)
+   is rejected and the file untouched; permissions preserved; `config_changes`
+   rows; provider add/remove; removing a referenced provider refused; reload
+   on mtime swaps the config.

docs/plans/briefs/web-dashboard/06-jobs-stats.md

@@ -0,0 +1,51 @@
+# Step 6 — Jobs and stats
+
+Read `00-shared.md`, the plan (§2 host facts, §3, §4.2, §9.1 sparklines, §12,
+§14 in full, §15 `jobs_enabled`/`journal_lines`, §16, §17 "Stats refactor" and
+"Jobs"), `src/main.rs` (`lock_holder`, `cmd_*` helpers, `GenerateOptions`),
+`src/curate/telemetry.rs` (`stats`), `systemd/`, and the handoffs for steps
+1–5. This is plan §19 item 6.
+
+## Deliverables
+
+1. `src/jobs.rs` (§14.1): the `Job` enum, `parse` (regex `^[a-z0-9-]+$` and
+   the listed forms only), `name`, `unit`, `description`, `takes_lock`,
+   `dangerous`. Register in `lib.rs`.
+2. `daily-epub job run <name>` (§14.2) in `main.rs`: parse (unknown → exit 2
+   with the catalogue), take the lock via `lock_holder` when `takes_lock()`,
+   find-or-insert the `requested` jobs row → `running`, run the mapped command
+   in-process with the same functions `main` uses, set `ok`/`failed`,
+   `finished_at`, `run_id` (generate, from `GenerateOutcome.run_id`), a one-line
+   `message`; non-zero exit on failure. The job lifecycle helpers live in
+   `src/jobs.rs` as db functions or on `Db`.
+3. `systemd/daily-epub-job@.service` (copy of the generate unit with the `%i`
+   description and `job run %i`), `systemd/50-daily-epub.rules` exactly as
+   §14.3, `SupplementaryGroups=systemd-journal` on `daily-epub.service`.
+4. `JobRunner` (§14.4): finalize the trait from step 1's stub; `SystemdRunner`
+   with `tokio::process::Command`, 10 s timeout, stderr in the error;
+   `MockRunner` with recorded calls and scripted statuses; `DisabledRunner`
+   when `jobs_enabled = false`. `serve` picks the runner from config.
+5. `src/web/dashboard/jobs.rs` + `dashboard/jobs.html` + `dashboard/job.html`:
+   the catalogue cards (Start with `data-confirm` for `dangerous()`, a date
+   input on the `generate-<date>` card), the jobs table, `POST
+   /dashboard/jobs/{name}` (409 + flash for a duplicate `requested`/`running`
+   unit; insert; `runner.start`; failed start marks the row), `GET
+   /dashboard/jobs/{id}` (row, live `UnitStatus`, journal tail of
+   `server.journal_lines` in `<pre>`, the 30-second "unit exited before the job
+   started" rule). `reload_if_changed` on every job start (§4.2). `app.js`
+   refreshes the job page every 5 s while `requested`/`running`.
+6. Stats (§12): refactor `telemetry::stats` into `stats_data -> StatsData` +
+   `render_stats_text(&StatsData) -> String` with byte-identical CLI output (a
+   test pins it against a seeded DB before and after); `GET /dashboard/stats`
+   with `?days=14|30|90`, the tables, three server-rendered SVG sparklines via
+   `_sparkline.html`, the retriever yield table.
+7. Overview sparklines (§9.1): cost per run, selected per run, generation
+   seconds over the last 30 non-dry runs, using the same partial.
+8. Tests (§17 "Stats refactor", "Jobs"): `Job::parse` accepts the catalogue and
+   the dated form, rejects `../x`, uppercase and unknown names; `POST
+   /dashboard/jobs/{name}` inserts a row and calls `MockRunner::start` with the
+   right unit; a running duplicate is refused; a failed start marks the row;
+   `job run` in-process (a `--skip-llm`-style config with mocked providers, e.g.
+   `features-prune` or `profile-rebuild` against a temp DB) flips
+   `requested → running → ok`; the polkit rule file is present and its regex
+   matches the unit name format (string test); stats text unchanged.

docs/plans/briefs/web-dashboard/07-docs-polish.md

@@ -0,0 +1,34 @@
+# Step 7 — Docs, users page, polish
+
+Read `00-shared.md`, the plan (§3, §6.1, §7.2, §18, §19 item 7, §20
+acceptance criteria), the README, `config.example.toml`,
+`docs/plans/2026-08-15-implementation-notes.md`, `docs/runbooks/`, and every
+`handoff-step*.md`. This is plan §19 item 7.
+
+## Deliverables
+
+1. `src/web/dashboard/users.rs` + `dashboard/users.html`: read-only list of
+   users (username, role, disabled, created, last login, open session count)
+   and a note that edits happen through the `users` CLI.
+2. README: the site (public vs signed-in vs admin), roles, the `users` CLI,
+   jobs (unit + polkit), settings (in-place writes, env locks), the updated
+   route table, the reverse-proxy note about `X-Forwarded-For`.
+3. `config.example.toml`: confirm the new `[server]` keys are documented with
+   comments matching the README table.
+4. `docs/plans/2026-08-15-implementation-notes.md`: a "Web dashboard
+   (2026-09-03)" section recording the verified facts with dates (polkit 124
+   and the rule, unit changes, toml_edit in-place writes, axum-login git rev +
+   tower-sessions 0.15 + our store, password-auth, tower_governor, the
+   sandbox/test caveats) and the implementation-time decisions the handoffs
+   recorded.
+5. `docs/runbooks/web-dashboard-rollout.md` with the nine steps of plan §18,
+   written for the production host.
+6. Layout polish: Atom `<link rel="alternate">` and favicon in `<head>`, the
+   Users link in the admin nav, a pass over dark mode and narrow screens in
+   `app.css` (every wide table scrolls in `.scroll-x`; the masthead wraps; nav
+   collapses to wrapping links), and the 404/500 pages in the site layout.
+7. Walk plan §20's acceptance criteria one by one and fix any gap you can
+   close inside this step; list the rest in your handoff.
+8. Tests: the users page under the admin guard; a smoke test that every
+   template referenced by a route renders with the fixture data (extend the
+   existing router tests rather than duplicating setup).