diff --git a/docs/plans/2026-09-03-web-dashboard.md b/docs/plans/2026-09-03-web-dashboard.md new file mode 100644 index 0000000..9e33a59 --- /dev/null +++ b/docs/plans/2026-09-03-web-dashboard.md @@ -0,0 +1,811 @@ +# Web site and operator dashboard + +**Date:** 2026-09-03 +**Repository:** `thallada/the-daily-epub` +**Status:** implementation plan, ready to execute +**Builds on:** `docs/plans/2026-09-02-personalized-curation-v2.md` (the ranker this dashboard inspects; its §23 deferred a "web dashboard for ratings and knobs" — this is that plan) + +This plan is written so that a fresh implementation agent can execute it end to end. It records the decisions made in the 2026-09-03 brainstorming session with the operator, the verified facts about the current code and the production host, the target design in enough detail to code from, and the order in which to land it. + +--- + +## 0. Decisions recorded from the brainstorm (2026-09-03) + +These are settled. Do not reopen them during implementation. + +| Topic | Decision | +|---|---| +| Purpose | Publicly, `https://daily.hallada.net` is an aggregator: every issue as a stripped HTML page of titles linking out, authors, sources and comment links. Privately, after login, it is the operator's dashboard: every article ever considered and why it was or was not in the paper, the full rating history with edits, every issue in full HTML with downloads, all config knobs, and pipeline actions. | +| Frontend | Server-rendered **askama** templates plus one hand-written CSS file and a few hundred lines of plain JavaScript. No node build, no SPA, no framework; assets are embedded in the binary with `include_str!`. | +| Auth | Username + password, sessions in a SQLite table, one `HttpOnly; Secure; SameSite=Lax` cookie. Users are created from the CLI (`daily-epub users add`). No third-party identity, no passkeys. **Use the ecosystem, not hand-rolled code**: `axum-login` over `tower-sessions` for login/logout/session lifecycle and route guards, `password-auth` for argon2 hashing, `tower_governor` for the login throttle. axum-login is taken from **git at the pinned rev `151c72d7a1b4646830f86b4332e6bd6e34d719a7`** (`main`, 2026-05-07: tower-sessions 0.15 and the finalized `Require` API), not from the 0.18.0 crates.io release; §2 records why and what was verified. The only auth code we write is a ~60-line `SessionStore` over our own sqlx pool (the official store pins sqlx 0.8; we are on 0.9) and the `AuthnBackend` glue (§6). | +| Roles | `user` and `admin`. **Signed-in users of either role** see every issue in full HTML (Brief, summaries, article bodies, discussions, World Briefing, Behind the paper) and can download the EPUB/XTC files. **Admins** additionally rate, browse the dashboard, edit settings and the profile, and start jobs. | +| Copyright boundary | Anonymous visitors never see generated or scraped text: no Brief, no summaries, no `why` lines, no article bodies, no comments, no World Briefing. Sections, reading time and word count are fine. The public renderer takes a dedicated `PublicIssue` type that cannot carry the private fields. | +| Personalization | One shared algorithm for now. `rating_events` gains a nullable `user_id` so a per-user algorithm is possible later; nothing else is per user. HMAC links from the EPUB stay unattributed (`user_id NULL`, `source = 'epub'`). | +| Settings | Every key of `config.toml`, grouped by table, editable from the UI and written back **in place with comments preserved** (`toml_edit`). A key overridden by a `DAILY_EPUB_*` environment variable is shown locked. API keys and the HMAC secret are shown as present/absent only. The page re-reads the file on every view, so hand edits show up. `data/profile.md` gets an editor with version history. | +| Actions | A Jobs page starts pipeline work through **systemd**: `systemctl start daily-epub-job@.service`, permitted by a polkit rule for the `daily-epub` user. The server never runs the pipeline in-process (its unit has `MemoryDenyWriteExecute=yes`, which the XTC converter's Node JIT cannot live with). | +| Feed | A public Atom feed at `/feed.xml` with one entry per issue, carrying the same stripped list as the public page. | +| Existing routes | `/r/…`, `/opds…`, `/files/…`, `/healthz`, `/issues.json` keep working unchanged. `/files/*` additionally accepts a session cookie. | +| Retention | HTML issue pages are built from the database, so they outlive the EPUB files (`retention_days`); download links appear only while the file exists. The operator should raise `curation.ranking.telemetry_retention_days` if they want candidate history beyond 180 days; this plan does not change the default. | +| Design | Newspaper masthead in a serif, dashboard tables in the system sans stack, no external fonts or CDNs, light and dark via `prefers-color-scheme`, every wide table scrolls inside its own container. | + +--- + +## 1. Read these first + +- `docs/plans/2026-08-15-implementation-notes.md` — conventions (runtime sqlx queries, `jiff`, error style, askama, no network in tests). All of it applies here. +- `docs/plans/2026-09-02-personalized-curation-v2.md` §6 (feedback), §7 (tables), §9–§13 (signals, admission, utility, editor — the numbers the dashboard explains), §15 (explain/stats), §19 (config). +- `src/server.rs` — the existing axum router, `AppState`, `handle_rating`, Basic auth, `safe_join`, the tiny e-ink pages. This plan extends this file's router and moves nothing. +- `src/auth.rs` — HMAC rating links (unchanged). +- `src/config.rs` — `Config`, `Config::load`, `Config::resolve_path`, `validate`, `check_report`, `providers_redacted`, `stale_toml_error`, `ENV_PREFIX`/`ENV_SPLIT`. +- `src/db.rs` — `Db`, `current_ratings`, `append_rating_event`, `upsert_issue`, `replace_issue_articles`, `start_run`/`finish_run`, `provider_spend_for_utc_day`, `get_article`. +- `src/curate/telemetry.rs` — `ExplainRow`, `resolve_run`, `explain_row`, `near_misses`, `render_explain`, `stats`, `SignalsJson`. The dashboard renders the same data as HTML. +- `src/curate/signals.rs` — `decay`, `gate`, `direct_feeds`, `PreferenceState::load/summary`, used by the ratings page. +- `src/curate/profile/mod.rs` — `parse_profile_str`, `load_profile`, `load_or_build`, `rebuild`, `KV_LEARNED_ADJUSTMENTS`. +- `src/epub/chapters.rs`, `src/epub/templates/` — what an issue contains and how it is rendered; `prepare_body`, `social_line`, `behind_*_line`, `render_in_this_issue`. +- `src/epub/fixtures.rs` — `fixtures::issue()` builds a full `Issue` for tests; the web tests use it. +- `src/main.rs` — CLI layout (`clap` derive), `lock_holder`, `cmd_*` helpers. +- `systemd/`, `README.md` §"Reverse proxy", `docs/runbooks/curation-v2-migration.md` — how the host is set up. + +--- + +## 2. Verified facts (2026-09-03) + +Host and deployment (checked on the production box, which is also the dev checkout): + +- nginx site `daily` proxies `/` → `http://127.0.0.1:3499` with `Host`, `X-Forwarded-For` and `X-Forwarded-Proto` set, TLS from Let's Encrypt, HTTP→HTTPS redirect. `snippets/security-headers.conf` adds `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy: strict-origin-when-cross-origin`, a `Permissions-Policy`, and HSTS. **No Content-Security-Policy**, so inline `