Web dashboard step 6: jobs and stats

Job catalogue and daily-epub job run, the systemd job unit and polkit rule,
SystemdRunner/MockRunner, the Jobs pages with status and journal tail, the
stats_data refactor with a byte-identical CLI, the stats page and the
overview sparklines.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM
This commit is contained in:
2026-09-03 18:01:21 +00:00
co-authored by Claude Fable 5.1
parent d36f203e4c
commit aa3de51d9c
21 changed files with 2958 additions and 112 deletions
@@ -0,0 +1,73 @@
# Step 6 handoff — jobs and stats
## Landed
- Added `src/jobs.rs`: the fixed, regex-constrained job catalogue; jobs-table
query/lifecycle helpers; single-line terminal messages; `SystemdRunner` with
ten-second command timeouts, captured stderr and systemd status parsing.
- Added `daily-epub job run <name>`. It uses the existing command lock policy,
claims or creates the requested row, calls the same in-process helpers as the
ordinary CLI commands, records `ok`/`failed`, stores the generated run id,
and propagates failures to a non-zero process exit. Existing profile,
features and social CLI output retains its prior line content.
- Finalized `JobRunner`: production `serve` installs `SystemdRunner` when
`server.jobs_enabled` is true; disabled servers use `DisabledRunner`;
`AppState::new` keeps its existing test behavior; `MockRunner` records calls
and scripts starts, statuses and logs.
- Added the admin Jobs list/start/detail routes and templates: catalogue cards,
dated generation, job history, duplicate conflict handling, start-failure
persistence, live unit fields, configured journal tail, 30-second
pre-claim failure detection, disabled-state messaging and five-second active
refresh.
- Added the systemd job template, the §14.3 polkit rule verbatim, and
`SupplementaryGroups=systemd-journal` on the server unit.
- Split stats into `stats_data` plus `render_stats_text` while pinning the
complete previous CLI output. Added the 14/30/90-day stats page, aggregate
tables, retriever yield, per-run table, daily provider costs and the three
requested SVG charts.
- Replaced the overview placeholder with cost-per-run, selected-per-run and
generation-time sparklines over the last 30 finished non-dry runs. SVGs use
presentation attributes and CSS classes only; no inline styles or template
`safe` filters were added.
- Added/expanded tests for the catalogue, job lifecycle and run link, CLI lock
mapping, in-process job success/failure, polkit/unit files, runner status
parsing, every jobs route behavior, exact stats text/data, stats routes and
both sparkline forms.
## Deviations and integration notes
- Per the parallel-step constraint, job start contains exactly the requested
`// TODO(step 5 merge): reload_if_changed` marker and does not duplicate step
5's helper. The step-5 merge must replace that marker with its helper call.
- The offline in-process job test uses `features-prune`; generate's `run_id`
persistence is tested separately through the same lifecycle helper because a
full generate would require Miniflux/network fixtures. Production generate
and dry-run mappings both store `GenerateOutcome.run_id`.
- The server unit intentionally does not add `/etc/daily-epub` to
`ReadWritePaths` here; step 5 owns that additive unit change.
- `StatsData.durations` retains finished dry runs because that is what the old
CLI calculation included and the text must remain byte-identical. Dashboard
per-run series and overview sparklines exclude dry runs as specified.
## Left for later
- Step-5 merge: call its `reload_if_changed` implementation at the marked job
start site and combine its `/etc/daily-epub` unit path change.
- Step 7 owns user-facing documentation and deployment/runbook polish.
## Verification
- `cargo fmt`: pass.
- `cargo clippy --all-targets -- -D warnings`: pass.
- Raw `cargo test`: **406 passed, 13 failed** in the library before Cargo
stopped; every failure was a sandbox-denied loopback listener (four named
Anthropic tests, three pre-existing OpenAI fake-server tests documented in
the step-1 handoff, the named extractor test and five named server tests).
- `cargo test` with those listener tests and both `tests/m7_server.rs` listener
tests skipped: **440 passed, 0 failed, 15 filtered out** across all targets.
- Focused `cargo test jobs::`, `cargo test job_run`, and `cargo test stats`:
pass.
- `node --check` on the polkit JavaScript (via stdin): pass.
- `systemd-analyze verify systemd/daily-epub-job@.service`: no diagnostics for
the job unit.
@@ -0,0 +1,64 @@
# Web dashboard step 6 implementation review
The finished step-6 implementation matches the jobs and stats brief. The fixed
job catalogue, CLI lifecycle, systemd runner and files, dashboard routes,
stats-data refactor, SVG sparklines, production runner wiring, and required
tests are all present. The review found and corrected two faulty test
assertions, restored the polkit file to the brief's verbatim rule, made job
messages reliably one-line, and strengthened the duplicate-start test to use a
genuinely running row. No outstanding correctness finding remains.
## Critical
None.
## High
None.
## Medium
None.
## Low
None.
## Nits
None.
## Plan Coverage
| Requirement | Status | Evidence |
|---|---|---|
| Fixed job catalogue and validated names | Implemented as planned | `src/jobs.rs`: `Job`, `parse`, `name`, `unit`, `description`, `takes_lock`, `dangerous` and traversal/uppercase/unknown-name tests. |
| `daily-epub job run <name>` lifecycle | Implemented as planned | `src/main.rs`: CLI dispatch, existing lock path, in-process command mapping, requested-row claim, terminal update, failure propagation and tests. `src/jobs.rs` owns row lifecycle helpers. |
| Unit template, polkit rule and journal group | Implemented as planned | `systemd/daily-epub-job@.service`, verbatim `systemd/50-daily-epub.rules`, and `SupplementaryGroups=systemd-journal` in `systemd/daily-epub.service`; repository-content tests cover their security-sensitive strings. |
| Production/test/disabled runners | Implemented as planned | `src/jobs.rs::SystemdRunner` uses bounded `tokio::process::Command`; `src/web/mod.rs` contains scripted `MockRunner` and `DisabledRunner`; `src/server.rs::serve` selects the production runner only when jobs are enabled while `AppState::new` remains test-compatible. |
| Jobs list/start/detail pages | Implemented as planned | `src/web/dashboard/jobs.rs` and the two jobs templates implement catalogue cards, date selection, history, duplicate refusal, failed-start persistence, live status, journal tail, the 30-second failure rule, authorization tests and five-second refresh. |
| Config reload integration boundary | Partially implemented by design | The required `// TODO(step 5 merge): reload_if_changed` is at the start site. Step 5 owns the helper and merge-time call per the brief's parallel-work constraint. |
| Stats data/text split | Implemented as planned | `src/curate/telemetry.rs` exposes `StatsData`, `stats_data`, and `render_stats_text`; seeded fixtures pin the complete legacy output byte-for-byte. |
| Stats page | Implemented as planned | `src/web/dashboard/stats.rs` and `dashboard/stats.html` provide the three allowed windows, tables, retriever yield, per-run data and the three requested charts. |
| Overview sparklines | Implemented as planned | `src/web/dashboard/mod.rs` loads the last 30 finished non-dry runs and renders cost, selected count and duration through `_sparkline.html`. |
| CSP-safe rendering | Implemented as planned | SVG geometry uses presentation attributes and stylesheet classes; changed templates contain no inline `style` attributes and no `safe` filter. |
## Testing Assessment
Meaningful tests cover every catalogue form and rejection case, job-row claim
and completion (including run-id persistence), real router authorization and
POST behavior with `MockRunner`, requested/running duplicate refusal, failed
starts, the unit-exited grace rule, disabled jobs, systemd output parsing,
polkit/unit contents, in-process job success and failure, exact
legacy stats text, stats aggregation, SVG geometry, page output, and overview
series.
The tests deliberately do not execute systemd or the network. The mapped
generate branch is therefore covered by its option construction and the
separate job run-id lifecycle test rather than a live pipeline invocation;
this follows the brief's offline-test rule. No additional step-6 test is needed.
## Open Questions
- During the step-5 merge, replace the required job-start TODO with the single
`reload_if_changed` call supplied by step 5; do not duplicate that helper.
+388 -78
View File
@@ -624,16 +624,179 @@ fn first_retriever(admitted_by: Option<&str>) -> String {
.unwrap_or_else(|| "unknown".to_string()) .unwrap_or_else(|| "unknown".to_string())
} }
#[derive(Debug, Default, Clone, Copy)] /// Rated picks by admitting retriever: how many were rated, up, down.
struct UpDown { #[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
rated: i64, pub struct UpDown {
up: i64, pub rated: i64,
down: i64, pub up: i64,
pub down: i64,
}
impl UpDown {
/// `"NN% up"`, or `n/a` with nothing rated.
pub fn ratio(&self) -> String {
if self.rated > 0 {
format!("{:.0}% up", 100.0 * self.up as f64 / self.rated as f64)
} else {
"n/a".to_string()
}
}
}
/// One finished, non-dry run as a point of the per-run series (dashboard
/// plan §9.1, §12).
#[derive(Debug, Clone, PartialEq)]
pub struct RunPoint {
pub run_id: i64,
pub date: String,
pub started_at: String,
pub status: String,
pub cost_usd: f64,
pub selected: i64,
pub duration_secs: Option<i64>,
}
/// Everything `daily-epub stats` prints, as data (dashboard plan §12): the
/// CLI renders it with [`render_stats_text`], the stats page as tables and
/// sparklines.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct StatsData {
pub days: i64,
/// First UTC date of the window.
pub since_date: String,
/// UTC date of `now`.
pub today: String,
pub issues: i64,
pub published: i64,
/// Explicit ratings per label, label order; `cleared` included.
pub ratings_by_label: Vec<(String, i64)>,
/// Explicit ratings excluding `cleared`.
pub total_ratings: i64,
pub per_retriever: BTreeMap<String, UpDown>,
pub exploration_admitted: i64,
pub exploration_selected: i64,
pub exploration_positive: i64,
/// Provider → spend over the whole window.
pub provider_totals: BTreeMap<String, f64>,
/// UTC date → provider → spend.
pub cost_by_day: BTreeMap<String, BTreeMap<String, f64>>,
/// Seconds of every finished run in the window (dry runs included), for
/// the mean generation time.
pub durations: Vec<i64>,
/// Finished non-dry runs in the window, oldest first.
pub runs: Vec<RunPoint>,
/// Issue date → picks, oldest first.
pub selected_per_issue: Vec<(String, i64)>,
/// Week (Monday, UTC) → label → explicit ratings.
pub ratings_per_week: BTreeMap<String, BTreeMap<String, i64>>,
}
impl StatsData {
/// `n` per issue with one decimal, or `n/a` without issues.
pub fn per_issue(&self, n: i64) -> String {
if self.issues > 0 {
format!("{:.1}", n as f64 / self.issues as f64)
} else {
"n/a".to_string()
}
}
pub fn mean_issue_size(&self) -> String {
self.per_issue(self.published)
}
pub fn ratings_per_issue(&self) -> String {
self.per_issue(self.total_ratings)
}
/// Spend per day over the window for one provider total.
pub fn per_day(&self, total: f64) -> f64 {
total / self.days as f64
}
/// Every provider's window spend added up, in provider order.
pub fn grand_total(&self) -> f64 {
let mut grand = 0.0;
for total in self.provider_totals.values() {
grand += total;
}
grand
}
/// Integer mean of the finished runs' seconds.
pub fn mean_generation_secs(&self) -> Option<i64> {
if self.durations.is_empty() {
None
} else {
Some(self.durations.iter().sum::<i64>() / self.durations.len() as i64)
}
}
} }
/// `daily-epub stats [--days N]` as text: the whole evaluation framework /// `daily-epub stats [--days N]` as text: the whole evaluation framework
/// (§15.3). One fact per line, nothing wider than 80 columns. /// (§15.3). One fact per line, nothing wider than 80 columns.
pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String> { pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String> {
Ok(render_stats_text(&stats_data(db, days, now).await?))
}
/// Finished non-dry runs as sparkline points, oldest first: those started
/// at or after `since` (RFC3339) and/or the newest `limit`.
pub async fn run_series(
db: &Db,
since: Option<&str>,
limit: Option<i64>,
) -> Result<Vec<RunPoint>, sqlx::Error> {
let rows = sqlx::query(
"SELECT id, date, started_at, finished_at, status, cost_usd, selected FROM runs
WHERE status != 'dry_run' AND finished_at IS NOT NULL
AND (? IS NULL OR started_at >= ?)
ORDER BY id DESC LIMIT ?",
)
.bind(since)
.bind(since)
.bind(limit.unwrap_or(i64::MAX))
.fetch_all(db.pool())
.await?;
let mut points: Vec<RunPoint> = rows
.iter()
.map(|row| {
let started_at: String = row.get("started_at");
let finished_at: Option<String> = row.get("finished_at");
let duration_secs = match (
started_at.parse::<Timestamp>(),
finished_at.as_deref().map(str::parse::<Timestamp>),
) {
(Ok(started), Some(Ok(finished))) => {
Some((finished.as_second() - started.as_second()).max(0))
}
_ => None,
};
RunPoint {
run_id: row.get("id"),
date: row.get("date"),
started_at,
status: row.get("status"),
cost_usd: row.get("cost_usd"),
selected: row.get("selected"),
duration_secs,
}
})
.collect();
points.reverse();
Ok(points)
}
/// The Monday (UTC) of the week containing `ts`, as `YYYY-MM-DD`.
pub fn week_start(ts: Timestamp) -> String {
let date = ts.to_zoned(jiff::tz::TimeZone::UTC).date();
let offset = i64::from(date.weekday().to_monday_zero_offset());
date.checked_sub(jiff::Span::new().days(offset))
.unwrap_or(date)
.to_string()
}
/// Gather every figure of [`StatsData`] for the last `days` days.
pub async fn stats_data(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<StatsData> {
let days = days.max(1); let days = days.max(1);
let since_ts = now let since_ts = now
.checked_sub(jiff::Span::new().hours(days.saturating_mul(24))) .checked_sub(jiff::Span::new().hours(days.saturating_mul(24)))
@@ -641,30 +804,35 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
let since = fmt_ts(since_ts); let since = fmt_ts(since_ts);
let utc = jiff::tz::TimeZone::UTC; let utc = jiff::tz::TimeZone::UTC;
let since_date = since_ts.to_zoned(utc.clone()).date().to_string(); let since_date = since_ts.to_zoned(utc.clone()).date().to_string();
let today = now.to_zoned(utc).date().to_string(); let today = now.to_zoned(utc.clone()).date().to_string();
let mut out = String::new(); let mut data = StatsData {
let _ = writeln!(out, "stats: last {days} days ({since_date} → {today})"); days,
since_date: since_date.clone(),
today,
..StatsData::default()
};
// --- issues and articles --- // --- issues and articles ---
let issues: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM issues WHERE date >= ?") data.issues = sqlx::query_scalar("SELECT COUNT(*) FROM issues WHERE date >= ?")
.bind(&since_date) .bind(&since_date)
.fetch_one(db.pool()) .fetch_one(db.pool())
.await?; .await?;
let published: i64 = data.published =
sqlx::query_scalar("SELECT COUNT(*) FROM issue_articles WHERE issue_date >= ?") sqlx::query_scalar("SELECT COUNT(*) FROM issue_articles WHERE issue_date >= ?")
.bind(&since_date) .bind(&since_date)
.fetch_one(db.pool()) .fetch_one(db.pool())
.await?; .await?;
let _ = writeln!(out, "issues: {issues}"); let per_issue_rows = sqlx::query(
let _ = writeln!(out, "articles published: {published}"); "SELECT issue_date, COUNT(*) AS n FROM issue_articles
let per_issue = |n: i64| { WHERE issue_date >= ? GROUP BY issue_date ORDER BY issue_date",
if issues > 0 { )
format!("{:.1}", n as f64 / issues as f64) .bind(&since_date)
} else { .fetch_all(db.pool())
"n/a".to_string() .await?;
} data.selected_per_issue = per_issue_rows
}; .iter()
let _ = writeln!(out, "mean issue size: {} articles", per_issue(published)); .map(|row| (row.get::<String, _>("issue_date"), row.get::<i64, _>("n")))
.collect();
// --- explicit ratings by label --- // --- explicit ratings by label ---
let labels = sqlx::query( let labels = sqlx::query(
@@ -674,21 +842,32 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
.bind(&since) .bind(&since)
.fetch_all(db.pool()) .fetch_all(db.pool())
.await?; .await?;
let mut total_ratings = 0i64;
let mut by_label = Vec::new();
for row in &labels { for row in &labels {
let label = row.get::<String, _>("label"); let label = row.get::<String, _>("label");
let n = row.get::<i64, _>("n"); let n = row.get::<i64, _>("n");
if label != "cleared" { if label != "cleared" {
total_ratings += n; data.total_ratings += n;
} }
by_label.push((label, n)); data.ratings_by_label.push((label, n));
} }
let _ = writeln!(out, "explicit ratings: {total_ratings}"); let events = sqlx::query(
for (label, n) in &by_label { "SELECT label, event_at FROM rating_events
let _ = writeln!(out, "explicit ratings ({label}): {n}"); WHERE kind = 'explicit' AND event_at >= ? ORDER BY event_at",
)
.bind(&since)
.fetch_all(db.pool())
.await?;
for row in &events {
let Ok(event_at) = row.get::<String, _>("event_at").parse::<Timestamp>() else {
continue;
};
*data
.ratings_per_week
.entry(week_start(event_at))
.or_default()
.entry(row.get::<String, _>("label"))
.or_insert(0) += 1;
} }
let _ = writeln!(out, "ratings per issue: {}", per_issue(total_ratings));
// --- up/down per admitting retriever, from rated picks --- // --- up/down per admitting retriever, from rated picks ---
let rated_picks = sqlx::query( let rated_picks = sqlx::query(
@@ -712,8 +891,6 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
.bind(&since) .bind(&since)
.fetch_all(db.pool()) .fetch_all(db.pool())
.await?; .await?;
let mut per_retriever: BTreeMap<String, UpDown> = BTreeMap::new();
let mut exploration_positive = 0i64;
let mut seen: Option<ArticleId> = None; let mut seen: Option<ArticleId> = None;
for row in &rated_picks { for row in &rated_picks {
let article_id = row.get::<ArticleId, _>("article_id"); let article_id = row.get::<ArticleId, _>("article_id");
@@ -723,7 +900,7 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
seen = Some(article_id); seen = Some(article_id);
let value = row.get::<f64, _>("value"); let value = row.get::<f64, _>("value");
let retriever = first_retriever(row.get::<Option<String>, _>("admitted_by").as_deref()); let retriever = first_retriever(row.get::<Option<String>, _>("admitted_by").as_deref());
let entry = per_retriever.entry(retriever).or_default(); let entry = data.per_retriever.entry(retriever).or_default();
entry.rated += 1; entry.rated += 1;
if value > 0.0 { if value > 0.0 {
entry.up += 1; entry.up += 1;
@@ -735,24 +912,9 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
.map(|signals| signals.exploration) .map(|signals| signals.exploration)
.unwrap_or(false); .unwrap_or(false);
if exploration && value > 0.0 { if exploration && value > 0.0 {
exploration_positive += 1; data.exploration_positive += 1;
} }
} }
if per_retriever.is_empty() {
let _ = writeln!(out, "rated picks by admitting retriever: none");
}
for (retriever, counts) in &per_retriever {
let ratio = if counts.rated > 0 {
format!("{:.0}% up", 100.0 * counts.up as f64 / counts.rated as f64)
} else {
"n/a".to_string()
};
let _ = writeln!(
out,
"admitted by {retriever}: {} rated · {} up · {} down · {ratio}",
counts.rated, counts.up, counts.down
);
}
// --- exploration yield --- // --- exploration yield ---
let exploration_rows = sqlx::query( let exploration_rows = sqlx::query(
@@ -764,31 +926,25 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
.bind(&since) .bind(&since)
.fetch_all(db.pool()) .fetch_all(db.pool())
.await?; .await?;
let mut exploration_admitted = 0i64;
let mut exploration_selected = 0i64;
for row in &exploration_rows { for row in &exploration_rows {
match row.get::<String, _>("stage").as_str() { match row.get::<String, _>("stage").as_str() {
"selected" => { "selected" => {
exploration_admitted += 1; data.exploration_admitted += 1;
exploration_selected += 1; data.exploration_selected += 1;
} }
"admitted" | "assessed" | "shortlisted" => exploration_admitted += 1, "admitted" | "assessed" | "shortlisted" => data.exploration_admitted += 1,
_ => {} _ => {}
} }
} }
let _ = writeln!(out, "exploration admitted: {exploration_admitted}");
let _ = writeln!(out, "exploration selected: {exploration_selected}");
let _ = writeln!(out, "exploration rated positively: {exploration_positive}");
// --- cost per day per provider (§7.6) --- // --- cost per day per provider (§7.6) ---
let cost_rows = sqlx::query( let cost_rows = sqlx::query(
"SELECT provider_costs_json FROM runs "SELECT started_at, provider_costs_json FROM runs
WHERE started_at >= ? AND provider_costs_json IS NOT NULL", WHERE started_at >= ? AND provider_costs_json IS NOT NULL",
) )
.bind(&since) .bind(&since)
.fetch_all(db.pool()) .fetch_all(db.pool())
.await?; .await?;
let mut totals: BTreeMap<String, f64> = BTreeMap::new();
for row in &cost_rows { for row in &cost_rows {
let raw = row.get::<String, _>("provider_costs_json"); let raw = row.get::<String, _>("provider_costs_json");
let Ok(providers) = let Ok(providers) =
@@ -796,20 +952,21 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
else { else {
continue; continue;
}; };
let day = row
.get::<String, _>("started_at")
.parse::<Timestamp>()
.map(|ts| ts.to_zoned(utc.clone()).date().to_string())
.unwrap_or_else(|_| since_date.clone());
for (provider, usage) in providers { for (provider, usage) in providers {
*totals.entry(provider).or_insert(0.0) += usage.cost_usd; *data.provider_totals.entry(provider.clone()).or_insert(0.0) += usage.cost_usd;
*data
.cost_by_day
.entry(day.clone())
.or_default()
.entry(provider)
.or_insert(0.0) += usage.cost_usd;
} }
} }
let mut grand = 0.0;
for (provider, total) in &totals {
grand += total;
let _ = writeln!(
out,
"cost per day ({provider}): ${:.3}",
total / days as f64
);
}
let _ = writeln!(out, "cost per day (total): ${:.3}", grand / days as f64);
// --- mean generation time --- // --- mean generation time ---
let run_rows = sqlx::query( let run_rows = sqlx::query(
@@ -819,26 +976,81 @@ pub async fn stats(db: &Db, days: i64, now: Timestamp) -> anyhow::Result<String>
.bind(&since) .bind(&since)
.fetch_all(db.pool()) .fetch_all(db.pool())
.await?; .await?;
let mut durations = Vec::new();
for row in &run_rows { for row in &run_rows {
let started = row.get::<String, _>("started_at").parse::<Timestamp>(); let started = row.get::<String, _>("started_at").parse::<Timestamp>();
let finished = row.get::<String, _>("finished_at").parse::<Timestamp>(); let finished = row.get::<String, _>("finished_at").parse::<Timestamp>();
if let (Ok(started), Ok(finished)) = (started, finished) { if let (Ok(started), Ok(finished)) = (started, finished) {
durations.push((finished.as_second() - started.as_second()).max(0)); data.durations
.push((finished.as_second() - started.as_second()).max(0));
} }
} }
if durations.is_empty() {
let _ = writeln!(out, "mean generation time: n/a (0 runs)"); data.runs = run_series(db, Some(&since), None).await?;
} else { Ok(data)
let mean = durations.iter().sum::<i64>() / durations.len() as i64; }
/// The CLI text of [`StatsData`], one fact per line.
pub fn render_stats_text(data: &StatsData) -> String {
let mut out = String::new();
let _ = writeln!(
out,
"stats: last {} days ({} → {})",
data.days, data.since_date, data.today
);
let _ = writeln!(out, "issues: {}", data.issues);
let _ = writeln!(out, "articles published: {}", data.published);
let _ = writeln!(out, "mean issue size: {} articles", data.mean_issue_size());
let _ = writeln!(out, "explicit ratings: {}", data.total_ratings);
for (label, n) in &data.ratings_by_label {
let _ = writeln!(out, "explicit ratings ({label}): {n}");
}
let _ = writeln!(out, "ratings per issue: {}", data.ratings_per_issue());
if data.per_retriever.is_empty() {
let _ = writeln!(out, "rated picks by admitting retriever: none");
}
for (retriever, counts) in &data.per_retriever {
let _ = writeln!( let _ = writeln!(
out, out,
"mean generation time: {} ({} runs)", "admitted by {retriever}: {} rated · {} up · {} down · {}",
RunReport::format_duration(mean), counts.rated,
durations.len() counts.up,
counts.down,
counts.ratio()
); );
} }
Ok(out) let _ = writeln!(out, "exploration admitted: {}", data.exploration_admitted);
let _ = writeln!(out, "exploration selected: {}", data.exploration_selected);
let _ = writeln!(
out,
"exploration rated positively: {}",
data.exploration_positive
);
for (provider, total) in &data.provider_totals {
let _ = writeln!(
out,
"cost per day ({provider}): ${:.3}",
data.per_day(*total)
);
}
let _ = writeln!(
out,
"cost per day (total): ${:.3}",
data.per_day(data.grand_total())
);
match data.mean_generation_secs() {
None => {
let _ = writeln!(out, "mean generation time: n/a (0 runs)");
}
Some(mean) => {
let _ = writeln!(
out,
"mean generation time: {} ({} runs)",
RunReport::format_duration(mean),
data.durations.len()
);
}
}
out
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -1523,6 +1735,41 @@ mod tests {
text.lines().all(|line| line.chars().count() <= 80), text.lines().all(|line| line.chars().count() <= 80),
"no line wider than 80 columns" "no line wider than 80 columns"
); );
// Dashboard plan §12: the CLI output is byte-identical before and
// after the `stats_data` / `render_stats_text` split.
assert_eq!(text, STATS_TEXT_BEFORE_REFACTOR);
let data = stats_data(&db, 14, now).await.unwrap();
assert_eq!(render_stats_text(&data), text);
// The figures the stats page adds on top of the text.
assert_eq!(data.issues, 2);
assert_eq!(data.total_ratings, 4);
assert_eq!(
data.selected_per_issue,
vec![("2026-08-30".to_string(), 2), ("2026-09-01".to_string(), 1)]
);
assert_eq!(
data.cost_by_day["2026-08-30"]["deepseek"],
0.10 + 0.04,
"both 08-30 runs land on the same day"
);
assert_eq!(data.cost_by_day["2026-09-01"]["gemini"], 0.07);
assert_eq!(data.ratings_per_week["2026-08-31"]["loved"], 2);
assert_eq!(data.ratings_per_week["2026-08-31"]["not_for_me"], 1);
assert_eq!(data.ratings_per_week["2026-08-17"]["cleared"], 1);
assert_eq!(data.runs.len(), 3, "three finished non-dry runs");
assert_eq!(data.runs[0].run_id, run_ids[0], "oldest first");
assert_eq!(data.runs[0].duration_secs, Some(1200));
assert_eq!(data.runs[2].date, "2026-09-01");
assert_eq!(data.per_retriever["knn"].ratio(), "100% up");
assert_eq!(data.mean_generation_secs(), Some(900));
// The newest-N form the overview uses.
let last_two = run_series(&db, None, Some(2)).await.unwrap();
assert_eq!(
last_two.iter().map(|p| p.run_id).collect::<Vec<_>>(),
vec![run_ids[1], run_ids[2]]
);
// An empty database still prints every heading. // An empty database still prints every heading.
let (_dir, empty) = db_with_articles(&[]).await; let (_dir, empty) = db_with_articles(&[]).await;
@@ -1537,6 +1784,69 @@ mod tests {
] { ] {
assert!(text.contains(line), "missing {line:?} in:\n{text}"); assert!(text.contains(line), "missing {line:?} in:\n{text}");
} }
assert_eq!(text, STATS_TEXT_EMPTY_BEFORE_REFACTOR);
}
/// `stats(db, 14, 2026-09-02T12:00:00Z)` over the seed above, captured
/// from the pre-refactor implementation.
const STATS_TEXT_BEFORE_REFACTOR: &str = "\
stats: last 14 days (2026-08-19 → 2026-09-02)
issues: 2
articles published: 3
mean issue size: 1.5 articles
explicit ratings: 4
explicit ratings (cleared): 1
explicit ratings (good): 1
explicit ratings (loved): 2
explicit ratings (not_for_me): 1
ratings per issue: 2.0
admitted by blend: 1 rated · 0 up · 1 down · 0% up
admitted by exploration: 1 rated · 1 up · 0 down · 100% up
admitted by knn: 1 rated · 1 up · 0 down · 100% up
exploration admitted: 2
exploration selected: 1
exploration rated positively: 1
cost per day (anthropic): $0.043
cost per day (deepseek): $0.020
cost per day (gemini): $0.005
cost per day (voyage): $0.001
cost per day (total): $0.069
mean generation time: 15m00s (3 runs)
";
const STATS_TEXT_EMPTY_BEFORE_REFACTOR: &str = "\
stats: last 7 days (2026-08-26 → 2026-09-02)
issues: 0
articles published: 0
mean issue size: n/a articles
explicit ratings: 0
ratings per issue: n/a
rated picks by admitting retriever: none
exploration admitted: 0
exploration selected: 0
exploration rated positively: 0
cost per day (total): $0.000
mean generation time: n/a (0 runs)
";
#[test]
fn week_start_is_the_utc_monday() {
assert_eq!(
week_start("2026-09-02T12:00:00Z".parse().unwrap()),
"2026-08-31"
);
assert_eq!(
week_start("2026-08-31T00:00:00Z".parse().unwrap()),
"2026-08-31"
);
assert_eq!(
week_start("2026-09-06T23:59:59Z".parse().unwrap()),
"2026-08-31"
);
assert_eq!(
week_start("2026-09-07T00:00:00Z".parse().unwrap()),
"2026-09-07"
);
} }
#[tokio::test] #[tokio::test]
+606
View File
@@ -0,0 +1,606 @@
//! Operator jobs (dashboard plan §14): the fixed catalogue the Jobs page can
//! start, the `jobs` table lifecycle shared by the page and `daily-epub job
//! run`, and the systemd-backed [`JobRunner`].
//!
//! A job is a `daily-epub-job@<name>.service` instance. The web server never
//! runs the pipeline in-process: it inserts a `requested` row, asks systemd to
//! start the unit (polkit allows `start` on exactly that unit pattern), and the
//! unit's `job run <name>` claims the row, does the work and finishes it.
use std::time::Duration;
use async_trait::async_trait;
use jiff::Timestamp;
use jiff::civil::Date;
use sqlx::Row as _;
use crate::db::{Db, fmt_ts};
use crate::web::{JobRunner, UnitStatus};
/// Prefix of every job unit; the polkit rule matches the same pattern.
pub const UNIT_PREFIX: &str = "daily-epub-job@";
/// Marker written by the page's 30-second rule (§14.4).
pub const EXITED_BEFORE_START: &str = "unit exited before the job started; see the log";
// ---------------------------------------------------------------------------
// Catalogue (§14.1)
// ---------------------------------------------------------------------------
/// The jobs an admin can start from the dashboard (§14.1).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Job {
/// `generate` (today) or `generate-YYYY-MM-DD`.
Generate { date: Option<Date> },
/// `dry-run` → `generate --dry-run`.
DryRun,
/// `profile-rebuild` → `profile rebuild`.
ProfileRebuild,
/// `features-backfill` → `features backfill --days 30 --yes`.
FeaturesBackfill,
/// `backfill-social` → `backfill-social --days 7`.
BackfillSocial,
/// `features-prune` → `features prune`.
FeaturesPrune,
}
impl Job {
/// The catalogue in the order the Jobs page lists it.
pub const CATALOGUE: [Job; 6] = [
Job::Generate { date: None },
Job::DryRun,
Job::ProfileRebuild,
Job::FeaturesBackfill,
Job::BackfillSocial,
Job::FeaturesPrune,
];
/// `^[a-z0-9-]+$`: the only characters a job (and so a unit instance) name
/// may contain.
pub fn valid_name(name: &str) -> bool {
!name.is_empty()
&& name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-')
}
/// Parse a catalogue name or the dated `generate-YYYY-MM-DD` form; anything
/// else (including `../x`, uppercase, unknown names) is `None`.
pub fn parse(name: &str) -> Option<Job> {
if !Self::valid_name(name) {
return None;
}
match name {
"generate" => Some(Job::Generate { date: None }),
"dry-run" => Some(Job::DryRun),
"profile-rebuild" => Some(Job::ProfileRebuild),
"features-backfill" => Some(Job::FeaturesBackfill),
"backfill-social" => Some(Job::BackfillSocial),
"features-prune" => Some(Job::FeaturesPrune),
_ => {
let date = name.strip_prefix("generate-")?;
// Exactly `YYYY-MM-DD`; the round trip rejects `2026-9-3`.
let parsed: Date = date.parse().ok()?;
(parsed.to_string() == date).then_some(Job::Generate { date: Some(parsed) })
}
}
}
pub fn name(&self) -> String {
match self {
Job::Generate { date: None } => "generate".into(),
Job::Generate { date: Some(date) } => format!("generate-{date}"),
Job::DryRun => "dry-run".into(),
Job::ProfileRebuild => "profile-rebuild".into(),
Job::FeaturesBackfill => "features-backfill".into(),
Job::BackfillSocial => "backfill-social".into(),
Job::FeaturesPrune => "features-prune".into(),
}
}
/// `daily-epub-job@<name>.service`.
pub fn unit(&self) -> String {
unit_for(&self.name())
}
pub fn description(&self) -> &'static str {
match self {
Job::Generate { date: None } => {
"Build and publish today's issue, exactly as the morning timer does."
}
Job::Generate { date: Some(_) } => {
"Build and publish the issue of a given date, republishing it if it exists."
}
Job::DryRun => "Run the whole pipeline but publish nothing and record no issue.",
Job::ProfileRebuild => {
"Regenerate the learned taste adjustments from ratings with the editor model."
}
Job::FeaturesBackfill => {
"Embed rated and recently published articles (30 days) and interests into the cache."
}
Job::BackfillSocial => "Re-poll social scores for the last 7 days of entries.",
Job::FeaturesPrune => {
"Drop stale embeddings, old candidate telemetry and old assessments per the retention config."
}
}
}
/// The run-lock name `main::lock_holder` uses for the same command, when
/// the job takes the lock at all.
pub fn takes_lock(&self) -> Option<&'static str> {
match self {
Job::Generate { .. } | Job::DryRun => Some("generate"),
Job::ProfileRebuild => Some("profile rebuild"),
Job::FeaturesBackfill => Some("features backfill"),
Job::BackfillSocial => Some("backfill-social"),
Job::FeaturesPrune => None,
}
}
/// Needs a confirmation dialog: `generate` republishes an issue.
pub fn dangerous(&self) -> bool {
matches!(self, Job::Generate { .. })
}
}
/// `daily-epub-job@<name>.service` for a validated job name.
pub fn unit_for(name: &str) -> String {
format!("{UNIT_PREFIX}{name}.service")
}
// ---------------------------------------------------------------------------
// `jobs` table lifecycle (§14.2, §14.4)
// ---------------------------------------------------------------------------
/// One `jobs` row.
#[derive(Debug, Clone)]
pub struct JobRow {
pub id: i64,
pub name: String,
pub unit: String,
pub requested_by: Option<i64>,
/// The requester's username, when the user still exists.
pub requested_by_name: Option<String>,
pub requested_at: String,
pub started_at: Option<String>,
pub finished_at: Option<String>,
pub status: String,
pub message: Option<String>,
pub run_id: Option<i64>,
}
impl JobRow {
pub fn is_active(&self) -> bool {
matches!(self.status.as_str(), "requested" | "running")
}
}
const ROW_SELECT: &str =
"SELECT j.id, j.name, j.unit, j.requested_by, u.username AS requested_by_name,
j.requested_at, j.started_at, j.finished_at, j.status, j.message, j.run_id
FROM jobs j LEFT JOIN users u ON u.id = j.requested_by";
fn row_from(row: &sqlx::sqlite::SqliteRow) -> JobRow {
JobRow {
id: row.get("id"),
name: row.get("name"),
unit: row.get("unit"),
requested_by: row.get("requested_by"),
requested_by_name: row.get("requested_by_name"),
requested_at: row.get("requested_at"),
started_at: row.get("started_at"),
finished_at: row.get("finished_at"),
status: row.get("status"),
message: row.get("message"),
run_id: row.get("run_id"),
}
}
/// One job by id.
pub async fn get(db: &Db, id: i64) -> Result<Option<JobRow>, sqlx::Error> {
let row = sqlx::query(sqlx::AssertSqlSafe(format!("{ROW_SELECT} WHERE j.id = ?")))
.bind(id)
.fetch_optional(db.pool())
.await?;
Ok(row.as_ref().map(row_from))
}
/// The newest `limit` jobs.
pub async fn list(db: &Db, limit: i64) -> Result<Vec<JobRow>, sqlx::Error> {
let rows = sqlx::query(sqlx::AssertSqlSafe(format!(
"{ROW_SELECT} ORDER BY j.id DESC LIMIT ?"
)))
.bind(limit)
.fetch_all(db.pool())
.await?;
Ok(rows.iter().map(row_from).collect())
}
/// The id of a `requested`/`running` job for this unit, if any (the page
/// refuses a duplicate start, §14.4).
pub async fn active_for_unit(db: &Db, unit: &str) -> Result<Option<i64>, sqlx::Error> {
sqlx::query_scalar(
"SELECT id FROM jobs WHERE unit = ? AND status IN ('requested', 'running')
ORDER BY id DESC LIMIT 1",
)
.bind(unit)
.fetch_optional(db.pool())
.await
}
/// Insert a `requested` row (the dashboard, or `job run` when nobody asked).
pub async fn insert_requested(
db: &Db,
job: &Job,
requested_by: Option<i64>,
now: Timestamp,
) -> Result<i64, sqlx::Error> {
let row = sqlx::query(
"INSERT INTO jobs (name, unit, requested_by, requested_at, status)
VALUES (?, ?, ?, ?, 'requested') RETURNING id",
)
.bind(job.name())
.bind(job.unit())
.bind(requested_by)
.bind(fmt_ts(now))
.fetch_one(db.pool())
.await?;
Ok(row.get::<i64, _>("id"))
}
/// `job run`'s first step (§14.2): the newest `requested` row with this name,
/// or a fresh one with `requested_by NULL`, flipped to `running`.
pub async fn claim(db: &Db, job: &Job, now: Timestamp) -> Result<i64, sqlx::Error> {
let existing: Option<i64> = sqlx::query_scalar(
"SELECT id FROM jobs WHERE name = ? AND status = 'requested' ORDER BY id DESC LIMIT 1",
)
.bind(job.name())
.fetch_optional(db.pool())
.await?;
let id = match existing {
Some(id) => id,
None => insert_requested(db, job, None, now).await?,
};
sqlx::query("UPDATE jobs SET status = 'running', started_at = ? WHERE id = ?")
.bind(fmt_ts(now))
.bind(id)
.execute(db.pool())
.await?;
Ok(id)
}
/// Terminal state of a job.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Outcome {
Ok,
Failed,
}
impl Outcome {
pub fn as_str(self) -> &'static str {
match self {
Outcome::Ok => "ok",
Outcome::Failed => "failed",
}
}
}
/// Finish a job: status, `finished_at`, the one-line message and, for a
/// generate job, the run it produced.
pub async fn finish(
db: &Db,
id: i64,
outcome: Outcome,
message: &str,
run_id: Option<i64>,
now: Timestamp,
) -> Result<(), sqlx::Error> {
let message = message.split_whitespace().collect::<Vec<_>>().join(" ");
sqlx::query(
"UPDATE jobs SET status = ?, finished_at = ?, message = ?, run_id = ? WHERE id = ?",
)
.bind(outcome.as_str())
.bind(fmt_ts(now))
.bind(message)
.bind(run_id)
.bind(id)
.execute(db.pool())
.await?;
Ok(())
}
// ---------------------------------------------------------------------------
// SystemdRunner (§14.4)
// ---------------------------------------------------------------------------
/// Talks to systemd with `systemctl`/`journalctl` (10 s timeout each, stderr
/// in the error). Production runner when `server.jobs_enabled` is true.
#[derive(Debug, Clone)]
pub struct SystemdRunner {
timeout: Duration,
}
impl Default for SystemdRunner {
fn default() -> Self {
Self {
timeout: Duration::from_secs(10),
}
}
}
const SHOW_PROPERTIES: &str =
"ActiveState,SubState,Result,ExecMainStatus,ExecMainStartTimestamp,ExecMainExitTimestamp";
impl SystemdRunner {
async fn run(&self, program: &str, args: &[&str]) -> Result<String, String> {
let command = format!("{program} {}", args.join(" "));
let output = tokio::time::timeout(
self.timeout,
tokio::process::Command::new(program)
.args(args)
.kill_on_drop(true)
.output(),
)
.await
.map_err(|_| format!("{command}: timed out after {:?}", self.timeout))?
.map_err(|error| format!("{command}: {error}"))?;
let stdout = String::from_utf8_lossy(&output.stdout).into_owned();
if output.status.success() {
Ok(stdout)
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
Err(format!(
"{command}: exit {}: {}",
output
.status
.code()
.map(|code| code.to_string())
.unwrap_or_else(|| "signal".into()),
stderr.trim()
))
}
}
}
/// Parse `systemctl show -p …` output (`Key=Value` lines).
pub fn parse_unit_status(text: &str) -> UnitStatus {
let mut status = UnitStatus::default();
for line in text.lines() {
let Some((key, value)) = line.split_once('=') else {
continue;
};
let value = value.trim();
match key.trim() {
"ActiveState" => status.active_state = value.to_string(),
"SubState" => status.sub_state = value.to_string(),
"Result" => status.result = value.to_string(),
"ExecMainStatus" => status.exit_status = value.parse().ok(),
"ExecMainStartTimestamp" => {
status.started = (!value.is_empty()).then(|| value.to_string())
}
"ExecMainExitTimestamp" => {
status.exited = (!value.is_empty()).then(|| value.to_string())
}
_ => {}
}
}
status
}
#[async_trait]
impl JobRunner for SystemdRunner {
async fn start(&self, unit: &str) -> Result<(), String> {
self.run("systemctl", &["start", "--no-block", unit])
.await
.map(|_| ())
}
async fn status(&self, unit: &str) -> Result<UnitStatus, String> {
self.run("systemctl", &["show", "-p", SHOW_PROPERTIES, unit])
.await
.map(|text| parse_unit_status(&text))
}
async fn log(&self, unit: &str, lines: usize) -> Result<String, String> {
let lines = lines.to_string();
self.run(
"journalctl",
&["-u", unit, "-n", &lines, "--no-pager", "-o", "short-iso"],
)
.await
}
}
#[cfg(test)]
mod tests {
use super::*;
const POLKIT_RULES: &str = include_str!("../systemd/50-daily-epub.rules");
const JOB_UNIT: &str = include_str!("../systemd/daily-epub-job@.service");
const SERVER_UNIT: &str = include_str!("../systemd/daily-epub.service");
const POLKIT_UNIT_REGEX: &str = r"/^daily-epub-job@[a-z0-9-]+\.service$/";
/// The polkit rule's regex, by hand: `^daily-epub-job@[a-z0-9-]+\.service$`.
fn polkit_regex_matches(unit: &str) -> bool {
unit.strip_prefix(UNIT_PREFIX)
.and_then(|rest| rest.strip_suffix(".service"))
.is_some_and(Job::valid_name)
}
#[test]
fn parse_accepts_the_catalogue_and_the_dated_form() {
for job in Job::CATALOGUE {
assert_eq!(Job::parse(&job.name()), Some(job), "{}", job.name());
}
assert_eq!(
Job::parse("generate-2026-09-03"),
Some(Job::Generate {
date: Some("2026-09-03".parse().unwrap())
})
);
assert_eq!(
Job::parse("generate-2026-09-03").unwrap().unit(),
"daily-epub-job@generate-2026-09-03.service"
);
assert_eq!(
Job::parse("generate").unwrap().takes_lock(),
Some("generate")
);
assert_eq!(Job::parse("features-prune").unwrap().takes_lock(), None);
assert!(Job::parse("generate-2026-09-03").unwrap().dangerous());
assert!(!Job::parse("dry-run").unwrap().dangerous());
}
#[test]
fn parse_rejects_traversal_uppercase_and_unknown_names() {
for name in [
"../x",
"Generate",
"GENERATE",
"generate ",
"",
"backup",
"generate-2026-9-3",
"generate-2026-13-01",
"generate-",
"generate-2026-09-03.service",
"dry_run",
"features-prune;rm",
] {
assert_eq!(Job::parse(name), None, "{name:?}");
}
}
#[test]
fn polkit_rule_is_present_and_matches_the_unit_names() {
assert!(
POLKIT_RULES.contains(POLKIT_UNIT_REGEX),
"the rule must carry the unit regex verbatim"
);
assert!(POLKIT_RULES.contains(r#"action.lookup("verb") == "start""#));
assert!(POLKIT_RULES.contains(r#"subject.user == "daily-epub""#));
assert!(POLKIT_RULES.contains("org.freedesktop.systemd1.manage-units"));
for job in Job::CATALOGUE {
assert!(polkit_regex_matches(&job.unit()), "{}", job.unit());
}
assert!(polkit_regex_matches(
&Job::parse("generate-2026-09-03").unwrap().unit()
));
for unit in [
"daily-epub-job@../x.service",
"daily-epub-job@.service",
"daily-epub-job@Generate.service",
"daily-epub.service",
"daily-epub-generate.service",
"daily-epub-job@generate.service.d",
] {
assert!(!polkit_regex_matches(unit), "{unit}");
}
}
#[test]
fn unit_files_carry_the_job_template_and_journal_group() {
assert!(JOB_UNIT.contains("Description=The Daily EPUB job %i"));
assert!(JOB_UNIT.contains(
"ExecStart=/usr/local/bin/daily-epub --config /etc/daily-epub/config.toml job run %i"
));
assert!(JOB_UNIT.contains("TimeoutStartSec=45min"));
assert!(JOB_UNIT.contains("ProtectSystem=strict"));
assert!(SERVER_UNIT.contains("SupplementaryGroups=systemd-journal"));
}
#[test]
fn unit_status_parses_systemctl_show_output() {
let status = parse_unit_status(
"ActiveState=inactive\nSubState=dead\nResult=exit-code\nExecMainStatus=1\n\
ExecMainStartTimestamp=Thu 2026-09-03 05:30:01 EDT\nExecMainExitTimestamp=\n",
);
assert_eq!(status.active_state, "inactive");
assert_eq!(status.sub_state, "dead");
assert_eq!(status.result, "exit-code");
assert_eq!(status.exit_status, Some(1));
assert_eq!(
status.started.as_deref(),
Some("Thu 2026-09-03 05:30:01 EDT")
);
assert_eq!(status.exited, None);
assert_eq!(parse_unit_status("garbage").active_state, "");
}
#[tokio::test]
async fn claim_reuses_the_requested_row_and_finish_records_the_run() {
let dir = tempfile::tempdir().unwrap();
let db = Db::open_and_migrate(&dir.path().join("jobs.db"))
.await
.unwrap();
let now: Timestamp = "2026-09-03T10:00:00Z".parse().unwrap();
let job = Job::Generate {
date: Some("2026-09-03".parse().unwrap()),
};
let requested = insert_requested(&db, &job, None, now).await.unwrap();
assert_eq!(
active_for_unit(&db, &job.unit()).await.unwrap(),
Some(requested)
);
let claimed = claim(&db, &job, now).await.unwrap();
assert_eq!(
claimed, requested,
"the requested row is claimed, not duplicated"
);
let row = get(&db, claimed).await.unwrap().unwrap();
assert_eq!(row.status, "running");
assert_eq!(row.started_at.as_deref(), Some("2026-09-03T10:00:00Z"));
assert!(row.is_active());
let run_id = db.start_run(job_date(&job), now).await.unwrap();
finish(
&db,
claimed,
Outcome::Ok,
"curation: done",
Some(run_id),
now,
)
.await
.unwrap();
let row = get(&db, claimed).await.unwrap().unwrap();
assert_eq!(row.status, "ok");
assert_eq!(row.run_id, Some(run_id));
assert_eq!(row.message.as_deref(), Some("curation: done"));
assert_eq!(row.finished_at.as_deref(), Some("2026-09-03T10:00:00Z"));
assert_eq!(active_for_unit(&db, &job.unit()).await.unwrap(), None);
// Nothing requested: `claim` inserts a row with no requester.
let fresh = claim(&db, &Job::FeaturesPrune, now).await.unwrap();
let row = get(&db, fresh).await.unwrap().unwrap();
assert_eq!(row.status, "running");
assert_eq!(row.requested_by, None);
assert_eq!(row.name, "features-prune");
assert_eq!(list(&db, 10).await.unwrap().len(), 2);
assert_eq!(list(&db, 10).await.unwrap()[0].id, fresh);
finish(
&db,
fresh,
Outcome::Failed,
"provider failed\n after retry",
None,
now,
)
.await
.unwrap();
assert_eq!(
get(&db, fresh).await.unwrap().unwrap().message.as_deref(),
Some("provider failed after retry"),
"job messages are always one line"
);
}
fn job_date(job: &Job) -> Date {
match job {
Job::Generate { date: Some(date) } => *date,
_ => panic!("expected a dated generate"),
}
}
}
+1
View File
@@ -23,6 +23,7 @@ pub mod extract;
pub mod html; pub mod html;
pub mod http; pub mod http;
pub mod images; pub mod images;
pub mod jobs;
pub mod lock; pub mod lock;
pub mod miniflux; pub mod miniflux;
pub mod pipeline; pub mod pipeline;
+242 -22
View File
@@ -17,7 +17,7 @@ use daily_epub::db::Db;
use daily_epub::pipeline::{self, GenerateOptions, GenerateOutcome}; use daily_epub::pipeline::{self, GenerateOptions, GenerateOutcome};
use daily_epub::report::{RunReport, VOYAGE_PROVIDER}; use daily_epub::report::{RunReport, VOYAGE_PROVIDER};
use daily_epub::types::{ArticleId, RatingEvent, Vote}; use daily_epub::types::{ArticleId, RatingEvent, Vote};
use daily_epub::{curate, http, lock, server, social}; use daily_epub::{curate, http, jobs, lock, server, social};
/// A personalized daily newspaper, delivered as an EPUB. /// A personalized daily newspaper, delivered as an EPUB.
#[derive(Debug, Parser)] #[derive(Debug, Parser)]
@@ -61,6 +61,19 @@ enum Command {
/// Manage dashboard users without taking the pipeline run lock. /// Manage dashboard users without taking the pipeline run lock.
#[command(subcommand)] #[command(subcommand)]
Users(UsersCommand), Users(UsersCommand),
/// Operator jobs (what `daily-epub-job@<name>.service` runs).
#[command(subcommand)]
Job(JobCommand),
}
#[derive(Debug, Subcommand)]
enum JobCommand {
/// Run one catalogue job in-process and record it in the `jobs` table.
Run {
/// `generate`, `generate-YYYY-MM-DD`, `dry-run`, `profile-rebuild`,
/// `features-backfill`, `backfill-social` or `features-prune`.
name: String,
},
} }
#[derive(Debug, Subcommand)] #[derive(Debug, Subcommand)]
@@ -328,7 +341,7 @@ async fn main() -> Result<()> {
} }
Command::Profile(ProfileCommand::Rebuild) => { Command::Profile(ProfileCommand::Rebuild) => {
let db = Db::open_and_migrate(&config.database_path).await?; let db = Db::open_and_migrate(&config.database_path).await?;
cmd_profile_rebuild(&config, &db).await?; println!("{}", cmd_profile_rebuild(&config, &db).await?);
} }
Command::Ratings(command) => { Command::Ratings(command) => {
let db = Db::open_and_migrate(&config.database_path).await?; let db = Db::open_and_migrate(&config.database_path).await?;
@@ -345,11 +358,11 @@ async fn main() -> Result<()> {
} }
Command::Features(command) => { Command::Features(command) => {
let db = Db::open_and_migrate(&config.database_path).await?; let db = Db::open_and_migrate(&config.database_path).await?;
cmd_features(&config, &db, command).await?; println!("{}", cmd_features(&config, &db, command).await?);
} }
Command::BackfillSocial(args) => { Command::BackfillSocial(args) => {
let db = Db::open_and_migrate(&config.database_path).await?; let db = Db::open_and_migrate(&config.database_path).await?;
cmd_backfill_social(&db, args.days).await?; println!("{}", cmd_backfill_social(&db, args.days).await?);
} }
Command::Db(DbCommand::Migrate) => { Command::Db(DbCommand::Migrate) => {
let db = Db::open(&config.database_path).await?; let db = Db::open(&config.database_path).await?;
@@ -368,19 +381,44 @@ async fn main() -> Result<()> {
let db = Db::open_and_migrate(&config.database_path).await?; let db = Db::open_and_migrate(&config.database_path).await?;
cmd_users(&db, command).await?; cmd_users(&db, command).await?;
} }
Command::Job(JobCommand::Run { name }) => {
let Some(job) = jobs::Job::parse(&name) else {
eprintln!("unknown job {name:?}; the catalogue is:");
for job in jobs::Job::CATALOGUE {
eprintln!(" {:<20} {}", job.name(), job.description());
}
eprintln!(
" {:<20} {}",
"generate-YYYY-MM-DD",
jobs::Job::Generate {
date: Some(jiff::civil::Date::default())
}
.description()
);
std::process::exit(2);
};
let db = Db::open_and_migrate(&config.database_path).await?;
cmd_job_run(&config, &db, &job).await?;
}
} }
Ok(()) Ok(())
} }
/// The commands that write the database and provider budgets and so hold the /// The commands that write the database and provider budgets and so hold the
/// run lock (§5): `generate`, `profile rebuild`, `features backfill`, /// run lock (§5): `generate`, `profile rebuild`, `features backfill`,
/// `backfill-social`. Everything else is read-only or its own writer. /// `backfill-social`, and a `job run` of any of them. Everything else is
/// read-only or its own writer.
fn lock_holder(command: &Command) -> Option<&'static str> { fn lock_holder(command: &Command) -> Option<&'static str> {
match command { match command {
Command::Generate(_) => Some("generate"), Command::Generate(_) => Some("generate"),
Command::Profile(ProfileCommand::Rebuild) => Some("profile rebuild"), Command::Profile(ProfileCommand::Rebuild) => Some("profile rebuild"),
Command::Features(FeaturesCommand::Backfill(_)) => Some("features backfill"), Command::Features(FeaturesCommand::Backfill(_)) => Some("features backfill"),
Command::BackfillSocial(_) => Some("backfill-social"), Command::BackfillSocial(_) => Some("backfill-social"),
// An unknown name takes no lock; the dispatch exits 2 before opening
// the database.
Command::Job(JobCommand::Run { name }) => {
jobs::Job::parse(name).and_then(|job| job.takes_lock())
}
Command::Serve Command::Serve
| Command::Ratings(_) | Command::Ratings(_)
| Command::Explain(_) | Command::Explain(_)
@@ -603,7 +641,8 @@ fn print_lineup(issue: &daily_epub::types::Issue) {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
/// `profile rebuild` runs on the editor when configured, else bulk (§14.3). /// `profile rebuild` runs on the editor when configured, else bulk (§14.3).
async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> { /// Returns the one-line summary the CLI prints and `job run` records.
async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<String> {
use curate::llm::{Llms, provider_meters}; use curate::llm::{Llms, provider_meters};
let profile = curate::profile::load_or_build( let profile = curate::profile::load_or_build(
db, db,
@@ -637,12 +676,11 @@ async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> {
config.curation.feedback.verdicts_in_prompt, config.curation.feedback.verdicts_in_prompt,
) )
.await?; .await?;
println!( Ok(format!(
"taste profile rebuilt (version {}, {} chars)", "taste profile rebuilt (version {}, {} chars)",
rebuilt.version, rebuilt.version,
rebuilt.text.len() rebuilt.text.len()
); ))
Ok(())
} }
async fn resolve_rating_article( async fn resolve_rating_article(
@@ -765,7 +803,9 @@ async fn cmd_explain(db: &Db, args: ExplainArgs) -> Result<()> {
Ok(()) Ok(())
} }
async fn cmd_features(config: &Config, db: &Db, command: FeaturesCommand) -> Result<()> { /// `features backfill` / `features prune`; returns the final summary line
/// (progress lines are printed as they happen).
async fn cmd_features(config: &Config, db: &Db, command: FeaturesCommand) -> Result<String> {
match command { match command {
FeaturesCommand::Backfill(args) => { FeaturesCommand::Backfill(args) => {
if !config.voyage.enabled { if !config.voyage.enabled {
@@ -788,8 +828,7 @@ async fn cmd_features(config: &Config, db: &Db, command: FeaturesCommand) -> Res
plan.cached plan.cached
); );
if plan.is_empty() { if plan.is_empty() {
println!("cache is warm; nothing to do"); return Ok("cache is warm; nothing to do".into());
return Ok(());
} }
println!( println!(
"estimate: ~{} tokens ≈ ${:.4} with {} at ${:.2}/M", "estimate: ~{} tokens ≈ ${:.4} with {} at ${:.2}/M",
@@ -802,17 +841,16 @@ async fn cmd_features(config: &Config, db: &Db, command: FeaturesCommand) -> Res
&& !args.yes && !args.yes
&& !confirm("continue?")? && !confirm("continue?")?
{ {
println!("aborted"); return Ok("aborted".into());
return Ok(());
} }
let outcome = embedding::run_backfill(&service, &plan).await?; let outcome = embedding::run_backfill(&service, &plan).await?;
println!( Ok(format!(
"embedded {} articles and {} interests · {} tokens · ${:.4}", "embedded {} articles and {} interests · {} tokens · ${:.4}",
outcome.articles_embedded, outcome.articles_embedded,
outcome.interests_embedded, outcome.interests_embedded,
outcome.tokens, outcome.tokens,
outcome.cost_usd outcome.cost_usd
); ))
} }
FeaturesCommand::Prune => { FeaturesCommand::Prune => {
let ranking = &config.curation.ranking; let ranking = &config.curation.ranking;
@@ -823,17 +861,16 @@ async fn cmd_features(config: &Config, db: &Db, command: FeaturesCommand) -> Res
jiff::Timestamp::now(), jiff::Timestamp::now(),
) )
.await?; .await?;
println!( Ok(format!(
"pruned {} embeddings older than {} days, {} candidate rows and {} assessments older than {} days", "pruned {} embeddings older than {} days, {} candidate rows and {} assessments older than {} days",
pruned.embeddings, pruned.embeddings,
ranking.embedding_retention_days, ranking.embedding_retention_days,
pruned.telemetry, pruned.telemetry,
pruned.assessments, pruned.assessments,
ranking.telemetry_retention_days ranking.telemetry_retention_days
); ))
} }
} }
Ok(())
} }
/// A y/N question on stdin; anything but a leading `y` is a no. /// A y/N question on stdin; anything but a leading `y` is a no.
@@ -845,12 +882,94 @@ fn confirm(question: &str) -> Result<bool> {
Ok(answer.trim().to_lowercase().starts_with('y')) Ok(answer.trim().to_lowercase().starts_with('y'))
} }
async fn cmd_backfill_social(db: &Db, days: u32) -> Result<()> { async fn cmd_backfill_social(db: &Db, days: u32) -> Result<String> {
let http = http::build_client(http::DEFAULT_TIMEOUT)?; let http = http::build_client(http::DEFAULT_TIMEOUT)?;
let enricher = social::SocialEnricher::new(http, db.clone()); let enricher = social::SocialEnricher::new(http, db.clone());
let updated = enricher.backfill(days).await?; let updated = enricher.backfill(days).await?;
println!("refreshed social scores for {updated} articles"); Ok(format!("refreshed social scores for {updated} articles"))
Ok(()) }
// ---------------------------------------------------------------------------
// `job run` (dashboard plan §14.2)
// ---------------------------------------------------------------------------
/// `job run <name>`: claim the newest `requested` row of this job (or insert
/// one when the unit was started by hand), run the mapped command in-process
/// with the same functions the plain subcommands use, and record `ok` /
/// `failed` with a one-line message and, for generate, the run id. A failure
/// propagates so the unit exits non-zero (`Result=exit-code`).
async fn cmd_job_run(config: &Config, db: &Db, job: &jobs::Job) -> Result<()> {
let id = jobs::claim(db, job, jiff::Timestamp::now())
.await
.context("claiming the jobs row")?;
tracing::info!(job = %job.name(), job_id = id, "job started");
let result = run_job(config, db, job).await;
let now = jiff::Timestamp::now();
match result {
Ok((message, run_id)) => {
jobs::finish(db, id, jobs::Outcome::Ok, &message, run_id, now)
.await
.context("recording the job outcome")?;
tracing::info!(job = %job.name(), job_id = id, %message, "job finished");
Ok(())
}
Err(error) => {
let message = format!("{error:#}");
jobs::finish(db, id, jobs::Outcome::Failed, &message, None, now)
.await
.context("recording the job failure")?;
tracing::error!(job = %job.name(), job_id = id, %message, "job failed");
Err(error)
}
}
}
/// The command each catalogue job maps to (§14.1), returning its one-line
/// message and the run id it produced.
async fn run_job(config: &Config, db: &Db, job: &jobs::Job) -> Result<(String, Option<i64>)> {
match job {
jobs::Job::Generate { date } => {
generate_job(config, db, date.map(|date| date.to_string()), false).await
}
jobs::Job::DryRun => generate_job(config, db, None, true).await,
jobs::Job::ProfileRebuild => Ok((cmd_profile_rebuild(config, db).await?, None)),
jobs::Job::FeaturesBackfill => {
let args = BackfillArgs {
days: 30,
rated_only: false,
all: false,
yes: true,
};
Ok((
cmd_features(config, db, FeaturesCommand::Backfill(args)).await?,
None,
))
}
jobs::Job::BackfillSocial => Ok((cmd_backfill_social(db, 7).await?, None)),
jobs::Job::FeaturesPrune => Ok((
cmd_features(config, db, FeaturesCommand::Prune).await?,
None,
)),
}
}
/// `generate [--date D] [--dry-run]` as a job: the `curation:` line is the
/// message, the run id links the job to its run.
async fn generate_job(
config: &Config,
db: &Db,
date: Option<String>,
dry_run: bool,
) -> Result<(String, Option<i64>)> {
let opts = GenerateOptions {
date,
dry_run,
..GenerateOptions::default()
};
let outcome = pipeline::generate(config, db, &opts).await?;
print_outcome(&outcome);
let [curation, ..] = outcome.report.info_block();
Ok((curation, Some(outcome.run_id)))
} }
#[cfg(test)] #[cfg(test)]
@@ -982,9 +1101,110 @@ mod tests {
vec!["db", "migrate"], vec!["db", "migrate"],
vec!["features", "prune"], vec!["features", "prune"],
vec!["config", "check"], vec!["config", "check"],
vec!["job", "run", "features-prune"],
vec!["job", "run", "not-a-job"],
] { ] {
assert_eq!(lock_holder(&parse(&args)), None, "{args:?}"); assert_eq!(lock_holder(&parse(&args)), None, "{args:?}");
} }
// `job run` takes the same lock as the command it maps to.
assert_eq!(
lock_holder(&parse(&["job", "run", "generate"])),
Some("generate")
);
assert_eq!(
lock_holder(&parse(&["job", "run", "generate-2026-09-03"])),
Some("generate")
);
assert_eq!(
lock_holder(&parse(&["job", "run", "dry-run"])),
Some("generate")
);
assert_eq!(
lock_holder(&parse(&["job", "run", "profile-rebuild"])),
Some("profile rebuild")
);
assert_eq!(
lock_holder(&parse(&["job", "run", "features-backfill"])),
Some("features backfill")
);
assert_eq!(
lock_holder(&parse(&["job", "run", "backfill-social"])),
Some("backfill-social")
);
}
#[test]
fn parses_job_run() {
match Cli::try_parse_from(["daily-epub", "job", "run", "features-prune"])
.unwrap()
.command
{
Command::Job(JobCommand::Run { name }) => assert_eq!(name, "features-prune"),
other => panic!("expected job run, got {other:?}"),
}
assert!(Cli::try_parse_from(["daily-epub", "job", "run"]).is_err());
assert!(Cli::try_parse_from(["daily-epub", "job"]).is_err());
}
/// Dashboard plan §17 "Jobs": `job run` flips the dashboard's `requested`
/// row to `running` and then `ok` with the command's message, in-process
/// and without systemd.
#[tokio::test]
async fn job_run_flips_requested_to_running_to_ok() {
use sqlx::Row as _;
let dir = tempfile::tempdir().unwrap();
let mut config = Config {
database_path: dir.path().join("jobs.db"),
..Config::default()
};
let db = Db::open_and_migrate(&config.database_path).await.unwrap();
let now = jiff::Timestamp::now();
let job = jobs::Job::FeaturesPrune;
let requested = jobs::insert_requested(&db, &job, None, now).await.unwrap();
assert_eq!(
jobs::get(&db, requested).await.unwrap().unwrap().status,
"requested"
);
cmd_job_run(&config, &db, &job).await.unwrap();
let row = jobs::get(&db, requested).await.unwrap().unwrap();
assert_eq!(row.status, "ok", "{row:?}");
assert!(row.started_at.is_some());
assert!(row.finished_at.is_some());
assert!(
row.message
.as_deref()
.unwrap_or_default()
.starts_with("pruned 0 embeddings"),
"{row:?}"
);
assert_eq!(row.run_id, None);
let rows: i64 = sqlx::query("SELECT COUNT(*) AS n FROM jobs")
.fetch_one(db.pool())
.await
.unwrap()
.get("n");
assert_eq!(rows, 1, "the requested row was claimed, not duplicated");
// A failing command records `failed` with the error and propagates it.
config.voyage.enabled = false;
let backfill = jobs::Job::FeaturesBackfill;
let error = cmd_job_run(&config, &db, &backfill).await.unwrap_err();
assert!(error.to_string().contains("voyage.enabled is false"));
let failed = jobs::list(&db, 1).await.unwrap().remove(0);
assert_eq!(failed.name, "features-backfill");
assert_eq!(failed.status, "failed");
assert_eq!(failed.requested_by, None, "started by hand: no requester");
assert!(
failed
.message
.as_deref()
.unwrap_or_default()
.contains("voyage.enabled is false"),
"{failed:?}"
);
} }
#[test] #[test]
+27 -1
View File
@@ -101,6 +101,23 @@ impl AppState {
} }
} }
/// `new`, with a specific [`crate::web::JobRunner`]: `SystemdRunner` in
/// `serve`, a `MockRunner` in router tests (dashboard plan §14.4).
pub fn with_jobs(
db: Db,
config: Config,
config_path: Option<PathBuf>,
jobs: Arc<dyn crate::web::JobRunner>,
) -> Self {
let mut state = Self::new(db, config, config_path);
state.web = Arc::new(crate::web::WebState {
jobs,
started_at: Timestamp::now(),
config_mtime: std::sync::Mutex::new(None),
});
state
}
pub fn config(&self) -> Arc<Config> { pub fn config(&self) -> Arc<Config> {
match self.config.read() { match self.config.read() {
Ok(config) => Arc::clone(&config), Ok(config) => Arc::clone(&config),
@@ -199,7 +216,16 @@ pub async fn serve(
} }
}); });
let app = router(AppState::new(db, config, config_path)); // The Jobs page starts `daily-epub-job@<name>.service` through systemd +
// polkit (§14); `server.jobs_enabled = false` swaps in the runner whose
// page says so.
let jobs: Arc<dyn crate::web::JobRunner> = if config.server.jobs_enabled {
Arc::new(crate::jobs::SystemdRunner::default())
} else {
tracing::info!("server.jobs_enabled is false; the Jobs page cannot start units");
Arc::new(crate::web::DisabledRunner)
};
let app = router(AppState::with_jobs(db, config, config_path, jobs));
axum::serve( axum::serve(
listener, listener,
app.into_make_service_with_connect_info::<std::net::SocketAddr>(), app.into_make_service_with_connect_info::<std::net::SocketAddr>(),
+626 -1
View File
@@ -1,10 +1,635 @@
//! Dashboard: jobs pages. Filled in by web dashboard plan step 6. //! Dashboard: the Jobs pages (`/dashboard/jobs`, dashboard plan §14.4).
//!
//! The catalogue as cards, the `jobs` table, `POST /dashboard/jobs/{name}`
//! (insert `requested`, ask the runner to start the unit) and the job page
//! with the live unit status and the journal tail. The server never runs the
//! pipeline in-process: `job run` inside the unit does the work (§14.2).
use askama::Template;
use axum::Router; use axum::Router;
use axum::body::Bytes;
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Redirect, Response};
use axum::routing::get;
use axum_login::tower_sessions::Session;
use jiff::Timestamp;
use crate::config::Config;
use crate::jobs::{self, EXITED_BEFORE_START, Job, JobRow};
use crate::server::AppState; use crate::server::AppState;
use crate::web::session::{AuthSession, Viewer};
use crate::web::{Flash, Html, Page, UnitStatus, WebError, take_flash};
use super::{db_err, duration_between, fmt_duration, fmt_stored_time};
/// Routes contributed by this page group (merged by `dashboard::router`). /// Routes contributed by this page group (merged by `dashboard::router`).
pub fn routes() -> Router<AppState> { pub fn routes() -> Router<AppState> {
Router::new() Router::new()
.route("/dashboard/jobs", get(index))
.route("/dashboard/jobs/{key}", get(show).post(start))
}
/// How many rows the jobs table shows.
const TABLE_ROWS: i64 = 200;
/// A `requested` row whose unit already stopped with a failure this long after
/// the request is marked failed (§14.4).
const EXIT_GRACE_SECS: i64 = 30;
#[derive(Debug, Clone)]
struct JobCard {
name: String,
description: &'static str,
dangerous: bool,
/// The `generate` card carries the date input for `generate-YYYY-MM-DD`.
dated: bool,
lock: Option<&'static str>,
}
#[derive(Debug, Clone)]
struct JobLine {
id: i64,
name: String,
requested_by: String,
requested: String,
started: String,
finished: String,
duration: String,
status: String,
message: Option<String>,
run_id: Option<i64>,
}
#[derive(Template)]
#[template(path = "dashboard/jobs.html")]
struct JobsTemplate {
page: Page,
jobs_enabled: bool,
cards: Vec<JobCard>,
jobs: Vec<JobLine>,
today: String,
}
#[derive(Template)]
#[template(path = "dashboard/job.html")]
struct JobTemplate {
page: Page,
job: JobLine,
unit: String,
description: &'static str,
refresh: bool,
status: Option<UnitStatus>,
status_error: Option<String>,
log: String,
log_error: Option<String>,
log_lines: u32,
}
fn cards() -> Vec<JobCard> {
Job::CATALOGUE
.iter()
.map(|job| JobCard {
name: job.name(),
description: job.description(),
dangerous: job.dangerous(),
dated: matches!(job, Job::Generate { .. }),
lock: job.takes_lock(),
})
.collect()
}
fn job_line(row: &JobRow, config: &Config) -> JobLine {
JobLine {
id: row.id,
name: row.name.clone(),
requested_by: match (&row.requested_by_name, row.requested_by) {
(Some(name), _) => name.clone(),
(None, Some(id)) => format!("user {id}"),
(None, None) => "by hand".into(),
},
requested: fmt_stored_time(Some(&row.requested_at), config),
started: fmt_stored_time(row.started_at.as_deref(), config),
finished: fmt_stored_time(row.finished_at.as_deref(), config),
duration: fmt_duration(
row.started_at
.as_deref()
.and_then(|started| duration_between(started, row.finished_at.as_deref())),
),
status: row.status.clone(),
message: row.message.clone(),
run_id: row.run_id,
}
}
async fn jobs_template(
state: &AppState,
viewer: Option<Viewer>,
flash: Option<Flash>,
) -> Result<JobsTemplate, WebError> {
let config = state.config();
let rows = jobs::list(&state.db, TABLE_ROWS).await.map_err(db_err)?;
let mut page = Page::new("Jobs", viewer, "dashboard");
page.flash = flash;
let today = config
.tz()
.map(|tz| Timestamp::now().to_zoned(tz).date().to_string())
.unwrap_or_else(|_| {
Timestamp::now()
.to_zoned(jiff::tz::TimeZone::UTC)
.date()
.to_string()
});
Ok(JobsTemplate {
page,
jobs_enabled: config.server.jobs_enabled,
cards: cards(),
jobs: rows.iter().map(|row| job_line(row, &config)).collect(),
today,
})
}
/// `GET /dashboard/jobs`: the catalogue cards and the jobs table.
async fn index(
State(state): State<AppState>,
auth: AuthSession,
Extension(session): Extension<Session>,
) -> Result<Response, WebError> {
let viewer = auth.user().await.map(Viewer::from);
let flash = take_flash(&session).await?;
Ok(Html(jobs_template(&state, viewer, flash).await?).into_response())
}
/// The `date` field of the start form, when present and non-empty.
fn form_date(body: &[u8]) -> Option<String> {
url::form_urlencoded::parse(body)
.find(|(key, _)| key == "date")
.map(|(_, value)| value.trim().to_string())
.filter(|value| !value.is_empty())
}
async fn set_flash(session: &Session, kind: &str, text: String) -> Result<(), WebError> {
session
.insert(
"flash",
Flash {
kind: kind.into(),
text,
},
)
.await
.map_err(|error| WebError::Internal(error.into()))
}
/// `POST /dashboard/jobs/{name}` (admin, origin-checked): parse the name,
/// refuse a duplicate `requested`/`running` unit (409), insert `requested`,
/// start the unit; a failed start marks the row `failed`.
async fn start(
State(state): State<AppState>,
auth: AuthSession,
Extension(session): Extension<Session>,
Path(name): Path<String>,
body: Bytes,
) -> Result<Response, WebError> {
let viewer = auth
.user()
.await
.map(Viewer::from)
.ok_or_else(|| WebError::Unauthenticated {
next: "/dashboard/jobs".into(),
})?;
let mut job = Job::parse(&name).ok_or(WebError::NotFound)?;
if let (Job::Generate { date: None }, Some(date)) = (job, form_date(&body)) {
let parsed = date
.parse::<jiff::civil::Date>()
.map_err(|_| WebError::BadRequest(format!("invalid date {date:?}")))?;
job = Job::Generate { date: Some(parsed) };
}
let config = state.config();
if !config.server.jobs_enabled {
set_flash(
&session,
"error",
"Jobs are disabled on this server (server.jobs_enabled = false).".into(),
)
.await?;
return Ok(Redirect::to("/dashboard/jobs").into_response());
}
let unit = job.unit();
if let Some(active) = jobs::active_for_unit(&state.db, &unit)
.await
.map_err(db_err)?
{
let flash = Flash {
kind: "error".into(),
text: format!(
"{} is already requested or running (job {active}).",
job.name()
),
};
let template = jobs_template(&state, Some(viewer), Some(flash)).await?;
return Ok((StatusCode::CONFLICT, Html(template)).into_response());
}
// TODO(step 5 merge): reload_if_changed — pick up a hand-edited config.toml
// before the unit starts (§4.2); step 5 adds the helper in web/mod.rs.
let now = Timestamp::now();
let id = jobs::insert_requested(&state.db, &job, Some(viewer.id), now)
.await
.map_err(db_err)?;
match state.web.jobs.start(&unit).await {
Ok(()) => {
tracing::info!(user = %viewer.username, job = %job.name(), job_id = id, %unit, "job requested");
set_flash(&session, "success", format!("Started {}.", job.name())).await?;
}
Err(error) => {
tracing::warn!(user = %viewer.username, job = %job.name(), job_id = id, %unit, %error, "job start failed");
jobs::finish(
&state.db,
id,
jobs::Outcome::Failed,
&format!("could not start {unit}: {error}"),
None,
Timestamp::now(),
)
.await
.map_err(db_err)?;
set_flash(
&session,
"error",
format!("Could not start {}: {error}", job.name()),
)
.await?;
}
}
Ok(Redirect::to(&format!("/dashboard/jobs/{id}")).into_response())
}
/// `GET /dashboard/jobs/{id}`: the row, the live unit status and the journal
/// tail; applies the 30-second "unit exited before the job started" rule.
async fn show(
State(state): State<AppState>,
auth: AuthSession,
Extension(session): Extension<Session>,
Path(key): Path<String>,
) -> Result<Response, WebError> {
let viewer = auth.user().await.map(Viewer::from);
let id: i64 = key.parse().map_err(|_| WebError::NotFound)?;
let config = state.config();
let db = &state.db;
let mut row = jobs::get(db, id)
.await
.map_err(db_err)?
.ok_or(WebError::NotFound)?;
let job = Job::parse(&row.name);
let (status, status_error) = match state.web.jobs.status(&row.unit).await {
Ok(status) => (Some(status), None),
Err(error) => (None, Some(error)),
};
if let Some(status) = &status
&& row.status == "requested"
&& status.exited_unsuccessfully()
&& requested_secs_ago(&row, Timestamp::now()) >= EXIT_GRACE_SECS
{
jobs::finish(
db,
id,
jobs::Outcome::Failed,
EXITED_BEFORE_START,
None,
Timestamp::now(),
)
.await
.map_err(db_err)?;
row = jobs::get(db, id)
.await
.map_err(db_err)?
.ok_or(WebError::NotFound)?;
}
let lines = config.server.journal_lines;
let (log, log_error) = match state.web.jobs.log(&row.unit, lines as usize).await {
Ok(log) => (log, None),
Err(error) => (String::new(), Some(error)),
};
let mut page = Page::new(format!("Job {id} · {}", row.name), viewer, "dashboard");
page.flash = take_flash(&session).await?;
Ok(Html(JobTemplate {
page,
refresh: row.is_active(),
unit: row.unit.clone(),
description: job.map(|job| job.description()).unwrap_or(""),
job: job_line(&row, &config),
status,
status_error,
log,
log_error,
log_lines: lines,
})
.into_response())
}
fn requested_secs_ago(row: &JobRow, now: Timestamp) -> i64 {
row.requested_at
.parse::<Timestamp>()
.map(|requested| now.as_second() - requested.as_second())
.unwrap_or(i64::MAX)
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use axum::body::Body;
use axum::http::{Method, Request, header};
use tower::ServiceExt;
use super::*;
use crate::db::Db;
use crate::server::router;
use crate::web::MockRunner;
use crate::web::dashboard::tests::{assert_admin_only, get, login_cookie, response_text};
async fn app_with_runner(
config: Config,
runner: Arc<dyn crate::web::JobRunner>,
) -> (tempfile::TempDir, Db, axum::Router) {
let dir = tempfile::tempdir().unwrap();
let db = Db::open_and_migrate(&dir.path().join("db.sqlite"))
.await
.unwrap();
crate::web::users::add(&db, "reader", "correct horse battery", false)
.await
.unwrap();
crate::web::users::add(&db, "admin", "correct horse battery", true)
.await
.unwrap();
let app = router(AppState::with_jobs(db.clone(), config, None, runner));
(dir, db, app)
}
async fn post(app: &axum::Router, uri: &str, body: &str, cookie: &str) -> Response {
app.clone()
.oneshot(
Request::builder()
.method(Method::POST)
.uri(uri)
.header(header::COOKIE, cookie)
.header(header::CONTENT_TYPE, "application/x-www-form-urlencoded")
.header("sec-fetch-site", "same-origin")
.body(Body::from(body.to_string()))
.unwrap(),
)
.await
.unwrap()
}
fn location(response: &Response) -> String {
response
.headers()
.get(header::LOCATION)
.unwrap()
.to_str()
.unwrap()
.to_string()
}
#[test]
fn form_date_reads_the_optional_field() {
assert_eq!(form_date(b"date=2026-09-03"), Some("2026-09-03".into()));
assert_eq!(form_date(b"date=+2026-09-03+"), Some("2026-09-03".into()));
assert_eq!(form_date(b"date="), None);
assert_eq!(form_date(b""), None);
assert_eq!(form_date(b"other=1"), None);
}
#[tokio::test]
async fn jobs_page_lists_the_catalogue_and_the_table() {
let runner = Arc::new(MockRunner::default());
let (_dir, db, app) = app_with_runner(Config::default(), runner).await;
let now: Timestamp = "2026-09-03T10:00:00Z".parse().unwrap();
let id = jobs::insert_requested(&db, &Job::FeaturesPrune, None, now)
.await
.unwrap();
jobs::finish(&db, id, jobs::Outcome::Ok, "pruned 0 embeddings", None, now)
.await
.unwrap();
let body = assert_admin_only(&app, "/dashboard/jobs").await;
for job in Job::CATALOGUE {
assert!(
body.contains(&format!("action=\"/dashboard/jobs/{}\"", job.name())),
"{}: {body}",
job.name()
);
let escaped_description = job.description().replace('\'', "&#39;");
assert!(body.contains(&escaped_description), "{}", job.name());
}
assert!(body.contains("data-confirm"), "generate needs confirmation");
assert!(body.contains("type=\"date\""), "the dated generate input");
assert!(body.contains("pruned 0 embeddings"), "{body}");
assert!(body.contains(&format!("/dashboard/jobs/{id}")), "{body}");
assert!(!body.contains("style=\""), "no inline styles under the CSP");
}
#[tokio::test]
async fn starting_a_job_inserts_a_row_and_starts_the_unit() {
let runner = Arc::new(MockRunner::default());
let (_dir, db, app) = app_with_runner(
Config::default(),
runner.clone() as Arc<dyn crate::web::JobRunner>,
)
.await;
let admin = login_cookie(&app, "admin", "correct horse battery").await;
let response = post(&app, "/dashboard/jobs/features-prune", "", &admin).await;
assert_eq!(response.status(), StatusCode::SEE_OTHER);
let target = location(&response);
assert!(target.starts_with("/dashboard/jobs/"), "{target}");
let row = jobs::list(&db, 10).await.unwrap().remove(0);
assert_eq!(target, format!("/dashboard/jobs/{}", row.id));
assert_eq!(row.name, "features-prune");
assert_eq!(row.unit, "daily-epub-job@features-prune.service");
assert_eq!(row.status, "requested");
assert_eq!(row.requested_by_name.as_deref(), Some("admin"));
assert_eq!(
runner.calls(),
vec!["start daily-epub-job@features-prune.service".to_string()]
);
// The job page shows the row, the unit and refreshes while active.
let page = get(&app, &target, Some(&admin)).await;
assert_eq!(page.status(), StatusCode::OK);
let body = response_text(page).await;
assert!(
body.contains("daily-epub-job@features-prune.service"),
"{body}"
);
assert!(body.contains("data-refresh=\"5\""), "{body}");
assert!(body.contains("badge requested"), "{body}");
assert!(body.contains("Started features-prune."), "the flash");
// Simulate the unit claiming the request, then prove a running
// duplicate is refused with 409 and no second row.
assert_eq!(
jobs::claim(&db, &Job::FeaturesPrune, Timestamp::now())
.await
.unwrap(),
row.id
);
let duplicate = post(&app, "/dashboard/jobs/features-prune", "", &admin).await;
assert_eq!(duplicate.status(), StatusCode::CONFLICT);
let body = response_text(duplicate).await;
assert!(body.contains("already requested or running"), "{body}");
assert_eq!(jobs::list(&db, 10).await.unwrap().len(), 1);
assert_eq!(
runner
.calls()
.iter()
.filter(|call| call.starts_with("start "))
.count(),
1,
"no second start"
);
// The dated generate form starts the dated unit.
let dated = post(&app, "/dashboard/jobs/generate", "date=2026-09-03", &admin).await;
assert_eq!(dated.status(), StatusCode::SEE_OTHER);
let row = jobs::list(&db, 10).await.unwrap().remove(0);
assert_eq!(row.name, "generate-2026-09-03");
assert_eq!(
runner.calls().last().unwrap(),
"start daily-epub-job@generate-2026-09-03.service"
);
let bad_date = post(&app, "/dashboard/jobs/generate", "date=soon", &admin).await;
assert_eq!(bad_date.status(), StatusCode::BAD_REQUEST);
// Unknown names never reach the runner.
for name in ["../x", "Generate", "backup"] {
let unknown = post(&app, &format!("/dashboard/jobs/{name}"), "", &admin).await;
assert_eq!(unknown.status(), StatusCode::NOT_FOUND, "{name}");
}
assert_eq!(
runner
.calls()
.iter()
.filter(|call| call.starts_with("start "))
.count(),
2
);
// Readers cannot start jobs.
let reader = login_cookie(&app, "reader", "correct horse battery").await;
let forbidden = post(&app, "/dashboard/jobs/features-prune", "", &reader).await;
assert_eq!(forbidden.status(), StatusCode::FORBIDDEN);
}
#[tokio::test]
async fn a_failed_start_marks_the_row_failed() {
let runner = Arc::new(MockRunner::default());
runner.fail_starts(Some("polkit: access denied"));
let (_dir, db, app) = app_with_runner(
Config::default(),
runner.clone() as Arc<dyn crate::web::JobRunner>,
)
.await;
let admin = login_cookie(&app, "admin", "correct horse battery").await;
let response = post(&app, "/dashboard/jobs/dry-run", "", &admin).await;
assert_eq!(response.status(), StatusCode::SEE_OTHER);
let row = jobs::list(&db, 10).await.unwrap().remove(0);
assert_eq!(row.status, "failed");
assert!(row.finished_at.is_some());
assert_eq!(
row.message.as_deref(),
Some("could not start daily-epub-job@dry-run.service: polkit: access denied")
);
let body = response_text(get(&app, &location(&response), Some(&admin)).await).await;
assert!(body.contains("polkit: access denied"), "{body}");
assert!(
!body.contains("data-refresh"),
"a finished job does not refresh"
);
// The unit is free again.
runner.fail_starts(None);
let again = post(&app, "/dashboard/jobs/dry-run", "", &admin).await;
assert_eq!(again.status(), StatusCode::SEE_OTHER);
assert_eq!(jobs::list(&db, 10).await.unwrap()[0].status, "requested");
}
#[tokio::test]
async fn job_page_marks_a_unit_that_exited_before_the_job_started() {
let runner = Arc::new(MockRunner::default());
let unit = Job::ProfileRebuild.unit();
runner.set_status(
&unit,
UnitStatus {
active_state: "inactive".into(),
sub_state: "dead".into(),
result: "exit-code".into(),
exit_status: Some(1),
started: Some("Thu 2026-09-03 05:30:01 EDT".into()),
exited: Some("Thu 2026-09-03 05:30:02 EDT".into()),
},
);
runner.set_log("Sep 03 05:30:01 daily-epub[1]: Error: profile rebuild is already running");
let (_dir, db, app) = app_with_runner(
Config::default(),
runner.clone() as Arc<dyn crate::web::JobRunner>,
)
.await;
let admin = login_cookie(&app, "admin", "correct horse battery").await;
// Requested a minute ago and never claimed.
let requested_at = Timestamp::now() - jiff::Span::new().seconds(60);
let id = jobs::insert_requested(&db, &Job::ProfileRebuild, None, requested_at)
.await
.unwrap();
let body =
response_text(get(&app, &format!("/dashboard/jobs/{id}"), Some(&admin)).await).await;
assert!(body.contains(EXITED_BEFORE_START), "{body}");
assert!(body.contains("badge failed"), "{body}");
assert!(body.contains("exit-code"), "the live unit status");
assert!(body.contains("is already running"), "the journal tail");
assert_eq!(jobs::get(&db, id).await.unwrap().unwrap().status, "failed");
assert!(runner.calls().contains(&format!(
"log {unit} {}",
Config::default().server.journal_lines
)));
// A fresh request whose unit has not run yet is left alone.
runner.set_status(
&Job::FeaturesPrune.unit(),
UnitStatus {
active_state: "inactive".into(),
sub_state: "dead".into(),
result: "success".into(),
..UnitStatus::default()
},
);
let fresh = jobs::insert_requested(&db, &Job::FeaturesPrune, None, Timestamp::now())
.await
.unwrap();
let body =
response_text(get(&app, &format!("/dashboard/jobs/{fresh}"), Some(&admin)).await).await;
assert!(body.contains("badge requested"), "{body}");
assert_eq!(
jobs::get(&db, fresh).await.unwrap().unwrap().status,
"requested"
);
let missing = get(&app, "/dashboard/jobs/999999", Some(&admin)).await;
assert_eq!(missing.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn disabled_jobs_refuse_starts_without_a_row() {
let mut config = Config::default();
config.server.jobs_enabled = false;
let (_dir, db, app) = app_with_runner(config, Arc::new(crate::web::DisabledRunner)).await;
let admin = login_cookie(&app, "admin", "correct horse battery").await;
let body = response_text(get(&app, "/dashboard/jobs", Some(&admin)).await).await;
assert!(body.contains("Jobs are disabled"), "{body}");
let response = post(&app, "/dashboard/jobs/features-prune", "", &admin).await;
assert_eq!(response.status(), StatusCode::SEE_OTHER);
assert_eq!(location(&response), "/dashboard/jobs");
assert!(jobs::list(&db, 10).await.unwrap().is_empty());
}
} }
+40 -1
View File
@@ -425,9 +425,10 @@ struct OverviewTemplate {
active_jobs: Vec<JobLine>, active_jobs: Vec<JobLine>,
finished_jobs: Vec<JobLine>, finished_jobs: Vec<JobLine>,
config_warnings: Vec<String>, config_warnings: Vec<String>,
sparklines: Vec<stats::Sparkline>,
} }
/// `GET /dashboard` — the overview (§9.1). Sparklines arrive with step 6. /// `GET /dashboard` — the overview (§9.1).
async fn overview( async fn overview(
State(state): State<AppState>, State(state): State<AppState>,
auth: AuthSession, auth: AuthSession,
@@ -443,6 +444,7 @@ async fn overview(
let (ratings, ratings_total) = ratings_this_week(db, now).await?; let (ratings, ratings_total) = ratings_this_week(db, now).await?;
let unrated = unrated_picks(db).await?; let unrated = unrated_picks(db).await?;
let (active_jobs, finished_jobs) = jobs_summary(db, &config).await?; let (active_jobs, finished_jobs) = jobs_summary(db, &config).await?;
let sparklines = overview_sparklines(db).await?;
let config_warnings = config let config_warnings = config
.check_report(state.config_path.as_deref()) .check_report(state.config_path.as_deref())
.into_iter() .into_iter()
@@ -461,10 +463,41 @@ async fn overview(
active_jobs, active_jobs,
finished_jobs, finished_jobs,
config_warnings, config_warnings,
sparklines,
}) })
.into_response()) .into_response())
} }
/// How many finished non-dry runs the overview sparklines cover (§9.1).
const SPARKLINE_RUNS: i64 = 30;
/// Cost per run, selected per run and generation seconds over the last 30
/// non-dry runs (§9.1), drawn by `dashboard/_sparkline.html`.
async fn overview_sparklines(db: &Db) -> Result<Vec<stats::Sparkline>, WebError> {
let runs = crate::curate::telemetry::run_series(db, None, Some(SPARKLINE_RUNS))
.await
.map_err(db_err)?;
let labels = (
runs.first().map(|run| run.date.as_str()).unwrap_or(""),
runs.last().map(|run| run.date.as_str()).unwrap_or(""),
);
let costs: Vec<f64> = runs.iter().map(|run| run.cost_usd).collect();
let selected: Vec<f64> = runs.iter().map(|run| run.selected as f64).collect();
let seconds: Vec<f64> = runs
.iter()
.map(|run| run.duration_secs.unwrap_or(0) as f64)
.collect();
Ok(vec![
stats::Sparkline::line("Cost per run", &costs, labels, "runs", fmt_usd),
stats::Sparkline::line("Selected per run", &selected, labels, "runs", |n| {
format!("{n:.0}")
}),
stats::Sparkline::line("Generation time", &seconds, labels, "runs", |secs| {
RunReport::format_duration(secs as i64)
}),
])
}
async fn last_run_card(db: &Db, config: &Config) -> Result<Option<LastRunCard>, WebError> { async fn last_run_card(db: &Db, config: &Config) -> Result<Option<LastRunCard>, WebError> {
let Some(row) = sqlx::query( let Some(row) = sqlx::query(
"SELECT id, date, status, started_at, finished_at, entries_fetched, candidates, "SELECT id, date, status, started_at, finished_at, entries_fetched, candidates,
@@ -1145,5 +1178,11 @@ pub(crate) mod tests {
assert!(body.contains("deepseek"), "{body}"); assert!(body.contains("deepseek"), "{body}");
assert!(body.contains("voyage"), "{body}"); assert!(body.contains("voyage"), "{body}");
assert!(body.contains("Ratings this week"), "{body}"); assert!(body.contains("Ratings this week"), "{body}");
// Step 6: three sparklines over the last 30 runs (two finished here).
assert!(body.contains("Cost per run"), "{body}");
assert!(body.contains("Generation time"), "{body}");
assert_eq!(body.matches("<polyline").count(), 3, "{body}");
assert!(body.contains("2 runs · max $0.11"), "{body}");
assert!(!body.contains("style=\""), "no inline styles under the CSP");
} }
} }
+639 -2
View File
@@ -1,10 +1,647 @@
//! Dashboard: stats pages. Filled in by web dashboard plan step 6. //! Dashboard: the stats page (`/dashboard/stats`, dashboard plan §12) and the
//! server-rendered SVG sparkline the overview shares (§9.1).
//!
//! The figures come from `telemetry::stats_data`, the same source as
//! `daily-epub stats`; the page shows them as tables plus three sparklines
//! (cost per day stacked per provider, selected per issue, ratings per week by
//! label) and the retriever yield with its up/down ratio.
use std::collections::BTreeMap;
use std::fmt::Write as _;
use askama::Template;
use axum::Router; use axum::Router;
use axum::extract::{Extension, Query, State};
use axum::response::{IntoResponse, Response};
use axum::routing::get;
use axum_login::tower_sessions::Session;
use jiff::Timestamp;
use serde::Deserialize;
use crate::curate::telemetry::{self, StatsData};
use crate::report::RunReport;
use crate::server::AppState; use crate::server::AppState;
use crate::web::session::{AuthSession, Viewer};
use crate::web::{Html, Page, WebError, take_flash};
use super::fmt_usd;
/// Routes contributed by this page group (merged by `dashboard::router`). /// Routes contributed by this page group (merged by `dashboard::router`).
pub fn routes() -> Router<AppState> { pub fn routes() -> Router<AppState> {
Router::new() Router::new().route("/dashboard/stats", get(stats))
}
/// The windows the page offers; anything else falls back to the first.
pub const WINDOWS: [i64; 3] = [14, 30, 90];
// ---------------------------------------------------------------------------
// Sparklines (§9.1, §12)
// ---------------------------------------------------------------------------
const SPARK_WIDTH: f64 = 240.0;
const SPARK_HEIGHT: f64 = 48.0;
const SPARK_PAD: f64 = 2.0;
/// Fill classes cycle through this many series colours (`.spark .s0` …).
pub const SERIES_CLASSES: usize = 6;
/// One stacked-bar segment: pre-formatted SVG coordinates, a series index for
/// its fill class and a `<title>` label.
#[derive(Debug, Clone, PartialEq)]
pub struct Bar {
pub x: String,
pub y: String,
pub w: String,
pub h: String,
pub series: usize,
pub label: String,
}
#[derive(Debug, Clone, PartialEq)]
pub struct Legend {
pub name: String,
pub series: usize,
}
/// A small SVG chart rendered by `dashboard/_sparkline.html` from numbers
/// only — presentation attributes and classes, no inline styles (the CSP is
/// `style-src 'self'`) and no `|safe`.
#[derive(Debug, Clone, PartialEq)]
pub struct Sparkline {
pub title: String,
pub caption: String,
pub width: u32,
pub height: u32,
/// `<polyline points>` of a line chart; empty for a bar chart.
pub points: String,
pub bars: Vec<Bar>,
pub legend: Vec<Legend>,
pub first_label: String,
pub last_label: String,
pub empty: bool,
}
fn coord(value: f64) -> String {
let text = format!("{value:.1}");
text.strip_suffix(".0").map(str::to_string).unwrap_or(text)
}
impl Sparkline {
fn blank(title: &str) -> Self {
Self {
title: title.to_string(),
caption: String::new(),
width: SPARK_WIDTH as u32,
height: SPARK_HEIGHT as u32,
points: String::new(),
bars: Vec::new(),
legend: Vec::new(),
first_label: String::new(),
last_label: String::new(),
empty: true,
}
}
/// A line over `values` (baseline 0), labelled with the first and last
/// point's label; the caption reports the count and the maximum.
pub fn line(
title: &str,
values: &[f64],
labels: (&str, &str),
unit: &str,
format_max: impl Fn(f64) -> String,
) -> Self {
let mut spark = Self::blank(title);
if values.is_empty() {
return spark;
}
let max = values.iter().copied().fold(0.0_f64, f64::max);
let inner_w = SPARK_WIDTH - 2.0 * SPARK_PAD;
let inner_h = SPARK_HEIGHT - 2.0 * SPARK_PAD;
let step = if values.len() > 1 {
inner_w / (values.len() - 1) as f64
} else {
0.0
};
let mut points = String::new();
for (index, value) in values.iter().enumerate() {
let x = if values.len() > 1 {
SPARK_PAD + step * index as f64
} else {
SPARK_WIDTH / 2.0
};
let y = if max > 0.0 {
SPARK_HEIGHT - SPARK_PAD - value.max(0.0) / max * inner_h
} else {
SPARK_HEIGHT - SPARK_PAD
};
if !points.is_empty() {
points.push(' ');
}
let _ = write!(points, "{},{}", coord(x), coord(y));
}
spark.points = points;
spark.caption = format!("{} {unit} · max {}", values.len(), format_max(max));
spark.first_label = labels.0.to_string();
spark.last_label = labels.1.to_string();
spark.empty = false;
spark
}
/// Stacked bars: one column per `(label, segments)` where each segment is
/// `(series name, value)`; series get fill classes in first-seen order.
pub fn stacked(
title: &str,
columns: &[(String, Vec<(String, f64)>)],
unit: &str,
format_max: impl Fn(f64) -> String,
) -> Self {
let mut spark = Self::blank(title);
if columns.is_empty() {
return spark;
}
let mut series: Vec<String> = Vec::new();
for (_, segments) in columns {
for (name, _) in segments {
if !series.contains(name) {
series.push(name.clone());
}
}
}
let max = columns
.iter()
.map(|(_, segments)| segments.iter().map(|(_, v)| v.max(0.0)).sum::<f64>())
.fold(0.0_f64, f64::max);
let inner_w = SPARK_WIDTH - 2.0 * SPARK_PAD;
let inner_h = SPARK_HEIGHT - 2.0 * SPARK_PAD;
let slot = inner_w / columns.len() as f64;
let gap = if slot > 3.0 { 1.0 } else { 0.0 };
let bar_w = (slot - gap).max(0.5);
let mut bars = Vec::new();
for (index, (label, segments)) in columns.iter().enumerate() {
let x = SPARK_PAD + slot * index as f64;
let mut top = SPARK_HEIGHT - SPARK_PAD;
for (name, value) in segments {
let value = value.max(0.0);
if value <= 0.0 || max <= 0.0 {
continue;
}
let h = value / max * inner_h;
top -= h;
let series_index = series.iter().position(|s| s == name).unwrap_or(0);
bars.push(Bar {
x: coord(x),
y: coord(top),
w: coord(bar_w),
h: coord(h),
series: series_index % SERIES_CLASSES,
label: format!("{label} · {name}: {}", format_max(value)),
});
}
}
spark.bars = bars;
spark.legend = series
.iter()
.enumerate()
.map(|(index, name)| Legend {
name: name.clone(),
series: index % SERIES_CLASSES,
})
.collect();
spark.caption = format!("{} {unit} · max {}", columns.len(), format_max(max));
spark.first_label = columns[0].0.clone();
spark.last_label = columns[columns.len() - 1].0.clone();
spark.empty = false;
spark
}
}
// ---------------------------------------------------------------------------
// Stats page (§12)
// ---------------------------------------------------------------------------
#[derive(Debug, Deserialize)]
pub struct StatsQuery {
days: Option<i64>,
}
#[derive(Debug, Clone)]
struct KeyValue {
key: String,
value: String,
}
#[derive(Debug, Clone)]
struct RetrieverLine {
retriever: String,
rated: i64,
up: i64,
down: i64,
ratio: String,
}
#[derive(Debug, Clone)]
struct CostRow {
date: String,
cells: Vec<String>,
total: String,
}
#[derive(Debug, Clone)]
struct RunLine {
run_id: i64,
date: String,
status: String,
cost: String,
selected: i64,
duration: String,
}
#[derive(Template)]
#[template(path = "dashboard/stats.html")]
struct StatsTemplate {
page: Page,
days: i64,
windows: Vec<i64>,
since_date: String,
today: String,
summary: Vec<KeyValue>,
ratings: Vec<KeyValue>,
retrievers: Vec<RetrieverLine>,
exploration: Vec<KeyValue>,
providers: Vec<String>,
cost_rows: Vec<CostRow>,
cost_per_day: Vec<KeyValue>,
runs: Vec<RunLine>,
sparklines: Vec<Sparkline>,
text: String,
}
/// `?days=` clamped to one of [`WINDOWS`].
pub fn window(days: Option<i64>) -> i64 {
days.filter(|days| WINDOWS.contains(days))
.unwrap_or(WINDOWS[0])
}
/// `GET /dashboard/stats?days=14|30|90`.
async fn stats(
State(state): State<AppState>,
auth: AuthSession,
Extension(session): Extension<Session>,
Query(query): Query<StatsQuery>,
) -> Result<Response, WebError> {
let viewer = auth.user().await.map(Viewer::from);
let days = window(query.days);
let now = Timestamp::now();
let data = telemetry::stats_data(&state.db, days, now)
.await
.map_err(WebError::Internal)?;
let text = telemetry::render_stats_text(&data);
let summary = vec![
KeyValue {
key: "Issues".into(),
value: data.issues.to_string(),
},
KeyValue {
key: "Articles published".into(),
value: data.published.to_string(),
},
KeyValue {
key: "Mean issue size".into(),
value: format!("{} articles", data.mean_issue_size()),
},
KeyValue {
key: "Explicit ratings".into(),
value: data.total_ratings.to_string(),
},
KeyValue {
key: "Ratings per issue".into(),
value: data.ratings_per_issue(),
},
KeyValue {
key: "Mean generation time".into(),
value: match data.mean_generation_secs() {
Some(secs) => format!(
"{} ({} runs)",
RunReport::format_duration(secs),
data.durations.len()
),
None => "n/a (0 runs)".into(),
},
},
];
let ratings = data
.ratings_by_label
.iter()
.map(|(label, n)| KeyValue {
key: label.clone(),
value: n.to_string(),
})
.collect();
let retrievers = data
.per_retriever
.iter()
.map(|(retriever, counts)| RetrieverLine {
retriever: retriever.clone(),
rated: counts.rated,
up: counts.up,
down: counts.down,
ratio: counts.ratio(),
})
.collect();
let exploration = vec![
KeyValue {
key: "Admitted".into(),
value: data.exploration_admitted.to_string(),
},
KeyValue {
key: "Selected".into(),
value: data.exploration_selected.to_string(),
},
KeyValue {
key: "Rated positively".into(),
value: data.exploration_positive.to_string(),
},
];
let providers: Vec<String> = data.provider_totals.keys().cloned().collect();
let cost_rows = data
.cost_by_day
.iter()
.map(|(date, by_provider)| CostRow {
date: date.clone(),
cells: providers
.iter()
.map(|provider| {
by_provider
.get(provider)
.map(|usd| format!("{usd:.3}"))
.unwrap_or_else(|| "".into())
})
.collect(),
total: format!("{:.3}", by_provider.values().sum::<f64>()),
})
.collect();
let mut cost_per_day: Vec<KeyValue> = data
.provider_totals
.iter()
.map(|(provider, total)| KeyValue {
key: provider.clone(),
value: format!("${:.3}", data.per_day(*total)),
})
.collect();
cost_per_day.push(KeyValue {
key: "total".into(),
value: format!("${:.3}", data.per_day(data.grand_total())),
});
let runs = data
.runs
.iter()
.map(|run| RunLine {
run_id: run.run_id,
date: run.date.clone(),
status: run.status.clone(),
cost: fmt_usd(run.cost_usd),
selected: run.selected,
duration: super::fmt_duration(run.duration_secs),
})
.collect();
let mut page = Page::new("Stats", viewer, "dashboard");
page.flash = take_flash(&session).await?;
Ok(Html(StatsTemplate {
page,
days,
windows: WINDOWS.to_vec(),
since_date: data.since_date.clone(),
today: data.today.clone(),
summary,
ratings,
retrievers,
exploration,
providers,
cost_rows,
cost_per_day,
runs,
sparklines: stats_sparklines(&data),
text,
})
.into_response())
}
/// Every UTC date from `since` to `today` inclusive.
fn dates_between(since: &str, today: &str) -> Vec<String> {
let (Ok(mut date), Ok(end)) = (
since.parse::<jiff::civil::Date>(),
today.parse::<jiff::civil::Date>(),
) else {
return Vec::new();
};
let mut dates = Vec::new();
while date <= end && dates.len() < 400 {
dates.push(date.to_string());
match date.tomorrow() {
Ok(next) => date = next,
Err(_) => break,
}
}
dates
}
/// The stats page's three charts (§12).
fn stats_sparklines(data: &StatsData) -> Vec<Sparkline> {
let providers: Vec<String> = data.provider_totals.keys().cloned().collect();
let empty = BTreeMap::new();
let cost_columns: Vec<(String, Vec<(String, f64)>)> =
dates_between(&data.since_date, &data.today)
.into_iter()
.map(|date| {
let by_provider = data.cost_by_day.get(&date).unwrap_or(&empty);
let segments = providers
.iter()
.map(|provider| {
(
provider.clone(),
by_provider.get(provider).copied().unwrap_or(0.0),
)
})
.collect();
(date, segments)
})
.collect();
let cost = if data.cost_by_day.is_empty() {
Sparkline::stacked("Cost per day", &[], "days", fmt_usd)
} else {
Sparkline::stacked("Cost per day", &cost_columns, "days", |usd| {
format!("${usd:.3}")
})
};
let selected_values: Vec<f64> = data
.selected_per_issue
.iter()
.map(|(_, n)| *n as f64)
.collect();
let selected = Sparkline::line(
"Selected per issue",
&selected_values,
(
data.selected_per_issue
.first()
.map(|(date, _)| date.as_str())
.unwrap_or(""),
data.selected_per_issue
.last()
.map(|(date, _)| date.as_str())
.unwrap_or(""),
),
"issues",
|n| format!("{n:.0}"),
);
let week_columns: Vec<(String, Vec<(String, f64)>)> = data
.ratings_per_week
.iter()
.map(|(week, by_label)| {
(
week.clone(),
by_label
.iter()
.map(|(label, n)| (label.clone(), *n as f64))
.collect(),
)
})
.collect();
let weekly = Sparkline::stacked("Ratings per week", &week_columns, "weeks", |n| {
format!("{n:.0}")
});
vec![cost, selected, weekly]
}
#[cfg(test)]
mod tests {
use super::*;
use crate::web::dashboard::tests::{
app_with_users, assert_admin_only, get, login_cookie, response_text, seed,
};
#[test]
fn window_falls_back_to_fourteen_days() {
assert_eq!(window(None), 14);
assert_eq!(window(Some(30)), 30);
assert_eq!(window(Some(90)), 90);
assert_eq!(window(Some(7)), 14);
assert_eq!(window(Some(-1)), 14);
}
#[test]
fn line_sparkline_scales_to_the_maximum_and_marks_empties() {
let spark = Sparkline::line(
"Cost",
&[0.0, 2.0, 1.0],
("2026-09-01", "2026-09-03"),
"runs",
fmt_usd,
);
assert!(!spark.empty);
assert_eq!(spark.points, "2,46 120,2 238,24");
assert_eq!(spark.caption, "3 runs · max $2.00");
assert_eq!(spark.first_label, "2026-09-01");
assert_eq!(spark.last_label, "2026-09-03");
assert!(spark.bars.is_empty());
let flat = Sparkline::line("Flat", &[0.0, 0.0], ("a", "b"), "runs", fmt_usd);
assert_eq!(flat.points, "2,46 238,46");
let none = Sparkline::line("None", &[], ("", ""), "runs", fmt_usd);
assert!(none.empty);
assert!(none.points.is_empty());
}
#[test]
fn stacked_sparkline_assigns_series_classes_in_first_seen_order() {
let columns = vec![
(
"2026-09-01".to_string(),
vec![("deepseek".to_string(), 0.1), ("voyage".to_string(), 0.1)],
),
(
"2026-09-02".to_string(),
vec![("deepseek".to_string(), 0.0), ("voyage".to_string(), 0.4)],
),
];
let spark =
Sparkline::stacked("Cost per day", &columns, "days", |usd| format!("${usd:.3}"));
assert!(!spark.empty);
assert_eq!(spark.legend.len(), 2);
assert_eq!(spark.legend[0].name, "deepseek");
assert_eq!(spark.legend[0].series, 0);
assert_eq!(spark.legend[1].series, 1);
// Three visible segments: the zero-valued one is skipped.
assert_eq!(spark.bars.len(), 3);
assert_eq!(spark.bars[0].series, 0);
assert_eq!(spark.bars[0].h, "11");
assert_eq!(spark.bars[0].y, "35");
assert_eq!(spark.bars[1].y, "24", "stacked on top of the first");
assert_eq!(spark.bars[2].h, "44");
assert_eq!(spark.bars[2].label, "2026-09-02 · voyage: $0.400");
assert_eq!(spark.caption, "2 days · max $0.400");
let none = Sparkline::stacked("None", &[], "days", fmt_usd);
assert!(none.empty);
}
#[test]
fn dates_between_is_inclusive() {
assert_eq!(
dates_between("2026-08-30", "2026-09-01"),
vec!["2026-08-30", "2026-08-31", "2026-09-01"]
);
assert!(dates_between("2026-09-02", "2026-09-01").is_empty());
assert!(dates_between("bad", "2026-09-01").is_empty());
}
#[tokio::test]
async fn stats_page_shows_tables_sparklines_and_the_text() {
let seed = seed().await;
// The seed is dated 2026-09-02; the page windows on `now`, so move the
// runs and rating events into the last hour to keep the test stable.
let now = Timestamp::now();
let earlier = crate::db::fmt_ts(now - jiff::Span::new().hours(1));
sqlx::query("UPDATE runs SET started_at = ?, finished_at = ?")
.bind(&earlier)
.bind(crate::db::fmt_ts(now))
.execute(seed.db.pool())
.await
.unwrap();
sqlx::query("UPDATE rating_events SET event_at = ?")
.bind(&earlier)
.execute(seed.db.pool())
.await
.unwrap();
let app = app_with_users(&seed.db).await;
let body = assert_admin_only(&app, "/dashboard/stats").await;
assert!(body.contains("?days=30"), "{body}");
assert!(body.contains("?days=90"), "{body}");
assert!(body.contains("Retriever yield"), "{body}");
assert!(body.contains("<rect"), "{body}");
assert!(body.contains("Ratings per week"), "{body}");
assert!(body.contains("badge loved"), "{body}");
assert!(body.contains("deepseek"), "{body}");
assert!(
body.contains("mean generation time:"),
"the CLI text is included"
);
assert!(!body.contains("style=\""), "no inline styles under the CSP");
assert!(
body.contains(&format!("/dashboard/runs/{}", seed.run_id)),
"{body}"
);
// An out-of-catalogue window falls back to 14 days.
let admin = login_cookie(&app, "admin", "correct horse battery").await;
let fallback = get(&app, "/dashboard/stats?days=7", Some(&admin)).await;
assert_eq!(fallback.status(), axum::http::StatusCode::OK);
let body = response_text(fallback).await;
assert!(body.contains("stats: last 14 days"), "{body}");
}
} }
+55 -5
View File
@@ -35,6 +35,20 @@ pub struct UnitStatus {
pub sub_state: String, pub sub_state: String,
pub result: String, pub result: String,
pub exit_status: Option<i32>, pub exit_status: Option<i32>,
/// `ExecMainStartTimestamp`, as systemd prints it; empty when never run.
pub started: Option<String>,
/// `ExecMainExitTimestamp`, as systemd prints it.
pub exited: Option<String>,
}
impl UnitStatus {
/// The unit ran and stopped with a failure result (`exit-code`, `failed`,
/// `signal`, `timeout`, …); a never-started unit reports `success`.
pub fn exited_unsuccessfully(&self) -> bool {
matches!(self.active_state.as_str(), "inactive" | "failed")
&& !self.result.is_empty()
&& self.result != "success"
}
} }
#[derive(Debug, Default)] #[derive(Debug, Default)]
@@ -55,11 +69,15 @@ impl JobRunner for DisabledRunner {
} }
} }
/// In-memory runner for router tests. Step 6 will add scripted results alongside /// In-memory runner for router tests: records every call and answers with
/// these recorded calls when the jobs pages begin invoking the runner. /// scripted results (`start` succeeds, `status` is inactive/success and `log`
/// is empty unless told otherwise).
#[derive(Debug, Default)] #[derive(Debug, Default)]
pub struct MockRunner { pub struct MockRunner {
calls: Mutex<Vec<String>>, calls: Mutex<Vec<String>>,
start_error: Mutex<Option<String>>,
statuses: Mutex<std::collections::HashMap<String, UnitStatus>>,
log_text: Mutex<String>,
} }
impl MockRunner { impl MockRunner {
@@ -67,6 +85,24 @@ impl MockRunner {
self.calls.lock().expect("mock runner lock").clone() self.calls.lock().expect("mock runner lock").clone()
} }
/// Make every `start` fail with `message` (`None` restores success).
pub fn fail_starts(&self, message: Option<&str>) {
*self.start_error.lock().expect("mock runner lock") = message.map(str::to_string);
}
/// Script the `status` answer for one unit.
pub fn set_status(&self, unit: &str, status: UnitStatus) {
self.statuses
.lock()
.expect("mock runner lock")
.insert(unit.to_string(), status);
}
/// Script the journal text every `log` call returns.
pub fn set_log(&self, text: &str) {
*self.log_text.lock().expect("mock runner lock") = text.to_string();
}
fn record(&self, call: String) { fn record(&self, call: String) {
self.calls.lock().expect("mock runner lock").push(call); self.calls.lock().expect("mock runner lock").push(call);
} }
@@ -76,17 +112,31 @@ impl MockRunner {
impl JobRunner for MockRunner { impl JobRunner for MockRunner {
async fn start(&self, unit: &str) -> Result<(), String> { async fn start(&self, unit: &str) -> Result<(), String> {
self.record(format!("start {unit}")); self.record(format!("start {unit}"));
Ok(()) match self.start_error.lock().expect("mock runner lock").clone() {
Some(message) => Err(message),
None => Ok(()),
}
} }
async fn status(&self, unit: &str) -> Result<UnitStatus, String> { async fn status(&self, unit: &str) -> Result<UnitStatus, String> {
self.record(format!("status {unit}")); self.record(format!("status {unit}"));
Ok(UnitStatus::default()) Ok(self
.statuses
.lock()
.expect("mock runner lock")
.get(unit)
.cloned()
.unwrap_or_else(|| UnitStatus {
active_state: "inactive".into(),
sub_state: "dead".into(),
result: "success".into(),
..UnitStatus::default()
}))
} }
async fn log(&self, unit: &str, lines: usize) -> Result<String, String> { async fn log(&self, unit: &str, lines: usize) -> Result<String, String> {
self.record(format!("log {unit} {lines}")); self.record(format!("log {unit} {lines}"));
Ok(String::new()) Ok(self.log_text.lock().expect("mock runner lock").clone())
} }
} }
+12 -1
View File
@@ -73,7 +73,6 @@ pre.preview { white-space:pre-wrap; overflow-wrap:anywhere; font:.85rem/1.4 ui-m
.versions pre.preview { max-height:6rem; border:0; padding:0; } .versions pre.preview { max-height:6rem; border:0; padding:0; }
.versions form { margin:0; } .versions form { margin:0; }
@media (max-width:60rem) { .profile-grid { display:block; } } @media (max-width:60rem) { .profile-grid { display:block; } }
||||||| 849231e
/* step 3: dashboard reads (overview, runs, articles) */ /* step 3: dashboard reads (overview, runs, articles) */
.dashboard h1 { font-size:1.5rem; margin:.5rem 0; } .dashboard h1 { font-size:1.5rem; margin:.5rem 0; }
.dashboard h1 a { color:inherit; } .dashboard h1 a { color:inherit; }
@@ -107,3 +106,15 @@ details.explain { margin:1rem 0; }
.picks li { border-bottom:1px solid var(--rule); padding:.5rem 0; } .picks li { border-bottom:1px solid var(--rule); padding:.5rem 0; }
.budget meter { width:60%; max-width:14rem; height:.8rem; vertical-align:middle; margin-right:.5rem; } .budget meter { width:60%; max-width:14rem; height:.8rem; vertical-align:middle; margin-right:.5rem; }
.table-filter { margin:.5rem 0; padding:.3rem; width:20rem; max-width:100%; } .table-filter { margin:.5rem 0; padding:.3rem; width:20rem; max-width:100%; }
/* step 6: jobs and stats */
.sparklines { display:grid; grid-template-columns:repeat(auto-fit,minmax(16rem,1fr)); gap:1rem 1.5rem; margin:1rem 0; }
.spark-figure { margin:0; min-width:0; } .spark-figure figcaption { font-size:.9rem; margin-bottom:.2rem; }
.spark { display:block; width:100%; max-width:26rem; height:auto; border-bottom:1px solid var(--rule); }
.spark .s0 { fill:var(--good); } .spark .s1 { fill:var(--loved); } .spark .s2 { fill:var(--down); } .spark .s3 { fill:var(--accent); } .spark .s4 { fill:var(--muted); } .spark .s5 { fill:var(--fg); }
.spark .line { stroke:var(--accent); }
.spark-axis { display:flex; justify-content:space-between; max-width:26rem; font-size:.75rem; }
.spark-legend { list-style:none; padding:0; margin:.3rem 0 0; display:flex; flex-wrap:wrap; gap:.2rem .8rem; font-size:.8rem; }
.spark-legend svg { vertical-align:middle; }
.job-cards form.inline { margin:.4rem 0 0; } .job-cards label { display:inline-grid; }
td.message,dd.message { overflow-wrap:anywhere; max-width:32rem; }
pre.journal { max-height:40rem; }
+5
View File
@@ -57,3 +57,8 @@ document.querySelectorAll("table[data-filter]").forEach((table) => {
}); });
}); });
}); });
/* step 6: reload a job page every N seconds while its job is requested/running */
document.querySelectorAll("[data-refresh]").forEach((element) => {
const seconds = Number(element.dataset.refresh);
if (seconds > 0) setTimeout(() => window.location.reload(), seconds * 1000);
});
@@ -0,0 +1,8 @@
<figure class="spark-figure"><figcaption>{{ spark.title }}{% if !spark.caption.is_empty() %} <span class="muted">· {{ spark.caption }}</span>{% endif %}</figcaption>
{% if spark.empty %}<p class="muted">No data yet.</p>{% else %}<svg class="spark" viewBox="0 0 {{ spark.width }} {{ spark.height }}" width="{{ spark.width }}" height="{{ spark.height }}" role="img" aria-label="{{ spark.title }}">
{% for bar in spark.bars %}<rect class="s{{ bar.series }}" x="{{ bar.x }}" y="{{ bar.y }}" width="{{ bar.w }}" height="{{ bar.h }}"><title>{{ bar.label }}</title></rect>
{% endfor %}{% if !spark.points.is_empty() %}<polyline class="line" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linejoin="round" stroke-linecap="round" points="{{ spark.points }}"/>{% endif %}
</svg>
<div class="spark-axis muted"><span>{{ spark.first_label }}</span><span>{{ spark.last_label }}</span></div>
{% if !spark.legend.is_empty() %}<ul class="spark-legend">{% for item in spark.legend %}<li><svg width="10" height="10" viewBox="0 0 10 10" aria-hidden="true"><rect class="s{{ item.series }}" width="10" height="10"/></svg> {{ item.name }}</li>{% endfor %}</ul>{% endif %}
{% endif %}</figure>
+32
View File
@@ -0,0 +1,32 @@
{% extends "layout.html" %}{% block content %}<section class="dashboard"{% if refresh %} data-refresh="5"{% endif %}>
<h1><a href="/dashboard/jobs">Jobs</a> Job {{ job.id }} · {{ job.name }} <span class="badge {{ job.status }}">{{ job.status }}</span></h1>
{% if !description.is_empty() %}<p class="muted">{{ description }}</p>{% endif %}
<div class="cards">
<section class="card"><h2>Job</h2>
<dl class="kv">
<dt>Unit</dt><dd><code>{{ unit }}</code></dd>
<dt>Requested</dt><dd>{{ job.requested }} · {{ job.requested_by }}</dd>
<dt>Started</dt><dd>{{ job.started }}</dd>
<dt>Finished</dt><dd>{{ job.finished }} · {{ job.duration }}</dd>
<dt>Status</dt><dd><span class="badge {{ job.status }}">{{ job.status }}</span></dd>
{% if let Some(message) = job.message %}<dt>Message</dt><dd class="message">{{ message }}</dd>{% endif %}
{% if let Some(run_id) = job.run_id %}<dt>Run</dt><dd><a href="/dashboard/runs/{{ run_id }}">Run {{ run_id }}</a></dd>{% endif %}
</dl>
{% if refresh %}<p class="muted">This page reloads every 5 seconds while the job is requested or running.</p>{% endif %}
</section>
<section class="card"><h2>Unit</h2>
{% if let Some(status) = status %}<dl class="kv">
<dt>Active</dt><dd>{{ status.active_state }}{% if !status.sub_state.is_empty() %} ({{ status.sub_state }}){% endif %}</dd>
<dt>Result</dt><dd>{{ status.result }}{% if let Some(code) = status.exit_status %} · exit status {{ code }}{% endif %}</dd>
{% if let Some(started) = status.started %}<dt>Main started</dt><dd>{{ started }}</dd>{% endif %}
{% if let Some(exited) = status.exited %}<dt>Main exited</dt><dd>{{ exited }}</dd>{% endif %}
</dl>{% endif %}
{% if let Some(error) = status_error %}<p class="error">Unit status unavailable: {{ error }}</p>{% endif %}
<p class="muted">From <code>systemctl show {{ unit }}</code>.</p>
</section>
</div>
<h2>Log</h2>
{% if let Some(error) = log_error %}<p class="error">Journal unavailable: {{ error }}</p>{% endif %}
<p class="muted">Last {{ log_lines }} lines of <code>journalctl -u {{ unit }}</code>.</p>
<pre class="preview journal">{{ log }}</pre>
</section>{% endblock %}
+28
View File
@@ -0,0 +1,28 @@
{% extends "layout.html" %}{% block content %}<section class="dashboard">
<h1>Jobs</h1>
{% if jobs_enabled %}<p class="muted">Each job starts <code>daily-epub-job@&lt;name&gt;.service</code> through systemd; the unit runs <code>daily-epub job run &lt;name&gt;</code> and records itself here. Jobs that take the run lock wait for nothing: a second <code>generate</code> while one is running fails immediately.</p>{% else %}<p class="notice">Jobs are disabled on this server (<code>server.jobs_enabled = false</code>); run the commands by hand instead.</p>{% endif %}
<div class="cards job-cards">{% for card in cards %}<section class="card">
<h2>{{ card.name }}</h2>
<p>{{ card.description }}</p>
{% if let Some(lock) = card.lock %}<p class="muted">Takes the <code>{{ lock }}</code> lock.</p>{% endif %}
<form method="post" action="/dashboard/jobs/{{ card.name }}" class="inline"{% if card.dangerous %} data-confirm="Start {{ card.name }}? This republishes the issue of that date."{% endif %}>
{% if card.dated %}<label>Date <input type="date" name="date" value="{{ today }}"></label>{% endif %}
<button type="submit"{% if !jobs_enabled %} disabled{% endif %}>Start</button>
</form>
</section>{% endfor %}</div>
<h2>History</h2>
{% if jobs.is_empty() %}<p class="muted">No jobs recorded yet.</p>{% else %}<div class="scroll-x"><table data-filter>
<thead><tr><th>job</th><th>name</th><th>requested by</th><th>requested</th><th>started</th><th>finished</th><th class="num">duration</th><th>status</th><th>message</th><th>run</th></tr></thead>
<tbody>{% for job in jobs %}<tr>
<td><a href="/dashboard/jobs/{{ job.id }}">{{ job.id }}</a></td>
<td>{{ job.name }}</td>
<td>{{ job.requested_by }}</td>
<td>{{ job.requested }}</td>
<td>{{ job.started }}</td>
<td>{{ job.finished }}</td>
<td class="num">{{ job.duration }}</td>
<td><span class="badge {{ job.status }}">{{ job.status }}</span></td>
<td class="message">{% if let Some(message) = job.message %}{{ message }}{% endif %}</td>
<td>{% if let Some(run_id) = job.run_id %}<a href="/dashboard/runs/{{ run_id }}">{{ run_id }}</a>{% endif %}</td>
</tr>{% endfor %}</tbody></table></div>{% endif %}
</section>{% endblock %}
+2 -1
View File
@@ -32,7 +32,8 @@
<p><a href="/dashboard/settings">Settings</a></p> <p><a href="/dashboard/settings">Settings</a></p>
</section>{% endif %} </section>{% endif %}
</div> </div>
<!-- step 6: sparklines (cost per run, selected per run, generation seconds; last 30 non-dry runs) --> <h2>Last 30 runs</h2>
<div class="sparklines">{% for item in sparklines %}{% let spark = item %}{% include "dashboard/_sparkline.html" %}{% endfor %}</div>
<h2>Unrated picks</h2> <h2>Unrated picks</h2>
{% if unrated.is_empty() %}<p class="muted">Every pick from the last three issues has a verdict.</p>{% else %}<p class="muted">Picks from the last three issues without a verdict yet.</p> {% if unrated.is_empty() %}<p class="muted">Every pick from the last three issues has a verdict.</p>{% else %}<p class="muted">Picks from the last three issues without a verdict yet.</p>
<ul class="picks">{% for pick in unrated %}<li><a href="{{ pick.issue_href }}">{{ pick.title }}</a> <span class="muted">· {{ pick.feed }} · {{ pick.issue_date }}</span> <ul class="picks">{% for pick in unrated %}<li><a href="{{ pick.issue_href }}">{{ pick.title }}</a> <span class="muted">· {{ pick.feed }} · {{ pick.issue_date }}</span>
+31
View File
@@ -0,0 +1,31 @@
{% extends "layout.html" %}{% block content %}<section class="dashboard">
<h1>Stats</h1>
<p class="muted">Last {{ days }} days ({{ since_date }} → {{ today }}) · window: {% for window in windows %}{% if *window == days %}<strong>{{ window }}</strong>{% else %}<a href="/dashboard/stats?days={{ window }}">{{ window }}</a>{% endif %}{% if !loop.last %} · {% endif %}{% endfor %} days · the same figures as <code>daily-epub stats --days {{ days }}</code>.</p>
<div class="sparklines">{% for item in sparklines %}{% let spark = item %}{% include "dashboard/_sparkline.html" %}{% endfor %}</div>
<div class="cards">
<section class="card"><h2>Issues and ratings</h2>
<dl class="kv">{% for line in summary %}<dt>{{ line.key }}</dt><dd>{{ line.value }}</dd>{% endfor %}</dl>
{% if ratings.is_empty() %}<p class="muted">No explicit ratings in the window.</p>{% else %}<p>{% for line in ratings %}<span class="badge {{ line.key }}">{{ line.key }}</span> {{ line.value }}{% if !loop.last %} · {% endif %}{% endfor %}</p>{% endif %}
</section>
<section class="card"><h2>Retriever yield</h2>
{% if retrievers.is_empty() %}<p class="muted">No rated picks by admitting retriever in the window.</p>{% else %}<div class="scroll-x"><table>
<thead><tr><th>admitted by</th><th class="num">rated</th><th class="num">up</th><th class="num">down</th><th class="num">ratio</th></tr></thead>
<tbody>{% for line in retrievers %}<tr><td>{{ line.retriever }}</td><td class="num">{{ line.rated }}</td><td class="num">{{ line.up }}</td><td class="num">{{ line.down }}</td><td class="num">{{ line.ratio }}</td></tr>{% endfor %}</tbody></table></div>{% endif %}
<h3>Exploration</h3>
<dl class="kv">{% for line in exploration %}<dt>{{ line.key }}</dt><dd>{{ line.value }}</dd>{% endfor %}</dl>
</section>
<section class="card"><h2>Cost per day</h2>
<dl class="kv">{% for line in cost_per_day %}<dt>{{ line.key }}</dt><dd>{{ line.value }}</dd>{% endfor %}</dl>
<p class="muted">Window spend divided by {{ days }} days, per provider.</p>
</section>
</div>
<h2>Spend by day</h2>
{% if cost_rows.is_empty() %}<p class="muted">No provider costs recorded in the window.</p>{% else %}<div class="scroll-x"><table data-filter>
<thead><tr><th>day (UTC)</th>{% for provider in providers %}<th class="num">{{ provider }}</th>{% endfor %}<th class="num">total</th></tr></thead>
<tbody>{% for row in cost_rows %}<tr><td>{{ row.date }}</td>{% for cell in row.cells %}<td class="num">{{ cell }}</td>{% endfor %}<td class="num">{{ row.total }}</td></tr>{% endfor %}</tbody></table></div>{% endif %}
<h2>Runs in the window</h2>
{% if runs.is_empty() %}<p class="muted">No finished runs in the window.</p>{% else %}<div class="scroll-x"><table data-filter>
<thead><tr><th>run</th><th>date</th><th>status</th><th class="num">cost</th><th class="num">selected</th><th class="num">duration</th></tr></thead>
<tbody>{% for run in runs %}<tr><td><a href="/dashboard/runs/{{ run.run_id }}">{{ run.run_id }}</a></td><td><a href="/issues/{{ run.date }}">{{ run.date }}</a></td><td><span class="badge {{ run.status }}">{{ run.status }}</span></td><td class="num">{{ run.cost }}</td><td class="num">{{ run.selected }}</td><td class="num">{{ run.duration }}</td></tr>{% endfor %}</tbody></table></div>{% endif %}
<details class="explain"><summary>As text (<code>daily-epub stats --days {{ days }}</code>)</summary><pre class="preview">{{ text }}</pre></details>
</section>{% endblock %}
+8
View File
@@ -0,0 +1,8 @@
polkit.addRule(function (action, subject) {
if (action.id == "org.freedesktop.systemd1.manage-units" &&
subject.user == "daily-epub" &&
action.lookup("verb") == "start" &&
/^daily-epub-job@[a-z0-9-]+\.service$/.test(action.lookup("unit"))) {
return polkit.Result.YES;
}
});
+69
View File
@@ -0,0 +1,69 @@
# The Daily EPUB — one operator job (dashboard plan §14): an instance of this
# template runs `daily-epub job run <name>` for a catalogue name such as
# `generate`, `generate-2026-09-03`, `dry-run`, `profile-rebuild`,
# `features-backfill`, `backfill-social` or `features-prune`. The Jobs page
# starts instances through systemd + polkit (systemd/50-daily-epub.rules); by
# hand: `sudo systemctl start daily-epub-job@features-prune`.
#
# Install: sudo install -m0644 systemd/daily-epub-job@.service /etc/systemd/system/
# (same binary, config and env file as daily-epub.service).
# Logs: journalctl -u daily-epub-job@<name> -f
[Unit]
Description=The Daily EPUB job %i
Documentation=https://github.com/thallada/the-daily-epub
After=network-online.target miniflux.service
Wants=network-online.target
[Service]
Type=oneshot
User=daily-epub
Group=daily-epub
ExecStart=/usr/local/bin/daily-epub --config /etc/daily-epub/config.toml job run %i
EnvironmentFile=-/etc/daily-epub/env
Environment=RUST_LOG=info,sqlx=warn,hyper=warn
# Same ceiling as the timer-driven generate unit: a generate job is the same
# network-bound run, and nothing else in the catalogue takes longer.
TimeoutStartSec=45min
Nice=10
IOSchedulingClass=idle
# --- state + writable paths ---------------------------------------------
StateDirectory=daily-epub
StateDirectoryMode=0750
WorkingDirectory=/var/lib/daily-epub
# The publish dirs from [publish] in config.toml — keep these in sync.
# Every path listed here must exist at start, or the unit fails with 226/NAMESPACE.
ReadWritePaths=/home/thallada/bookorbit/books/daily-epub /var/lib/daily-epub/xtc
# --- hardening (spec §3.15) ---------------------------------------------
ProtectSystem=strict
# read-only (not yes): the BookOrbit publish dir lives under /home, and
# ProtectHome=yes would mask it even with the ReadWritePaths entry above.
ProtectHome=read-only
PrivateTmp=yes
PrivateDevices=yes
NoNewPrivileges=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectHostname=yes
ProtectProc=invisible
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
LockPersonality=yes
# No MemoryDenyWriteExecute here: this unit spawns Node (epub-to-xtc-converter),
# whose JIT needs W+X pages (§3.11).
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
UMask=0027
# Template instances are normally started on demand by the dashboard.
[Install]
WantedBy=multi-user.target
+2
View File
@@ -23,6 +23,8 @@ Wants=network-online.target
Type=exec Type=exec
User=daily-epub User=daily-epub
Group=daily-epub Group=daily-epub
# The Jobs page reads `journalctl -u daily-epub-job@<name>` (dashboard plan §14.3).
SupplementaryGroups=systemd-journal
ExecStart=/usr/local/bin/daily-epub --config /etc/daily-epub/config.toml serve ExecStart=/usr/local/bin/daily-epub --config /etc/daily-epub/config.toml serve
EnvironmentFile=-/etc/daily-epub/env EnvironmentFile=-/etc/daily-epub/env
Environment=RUST_LOG=info,sqlx=warn,hyper=warn Environment=RUST_LOG=info,sqlx=warn,hyper=warn