diff --git a/README.md b/README.md index 87b1bd0..c35b816 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ select and introduce 15–25 of them. It assembles two EPUB editions (a standard and one tuned for the Xteink X4 e-ink reader), converts the X4 edition to XTC, and publishes the lot over its own OPDS catalog — which doubles as a [BookOrbit](https://github.com/thallada/bookorbit) watched folder if you run one. -Each article chapter ends with 👍/👎 links that feed back into tomorrow's curation. +Each article chapter ends with Loved it / Good / Not for me links that feed back into tomorrow's curation. Steady-state cost is roughly **$0.05–0.30/day** in DeepSeek tokens, hard-capped by `max_daily_usd`. @@ -48,13 +48,13 @@ selects, feed excerpts stand in for summaries) instead of losing the day's issue | Rust (2024 edition toolchain) | building | `cargo build --release` | | **Miniflux** with an API key | the only content source | Settings → API Keys. The client is read-only and never mutates read state. | | **DeepSeek API key** | curation + editorial | . Optional: `--skip-llm` runs the whole pipeline without it. | -| A 32+ byte random secret | signs the 👍/👎 rating links | `openssl rand -hex 32` | +| A 32+ byte random secret | signs the article rating links | `openssl rand -hex 32` | | **BookOrbit** library + watched folder | *optional* — a richer library UI on top of the same folder | Delivery does not need it: `daily-epub serve` has its own OPDS catalog over `publish.epub_dir`. If you do run it, create a dedicated "The Daily EPUB" library, enable *Watch folders*, and point `publish.epub_dir` at it. | | **Node.js 18+** and a clone of [`epub-to-xtc-converter`](https://github.com/bigbag/epub-to-xtc-converter) | XTC/XTCH output for the Xteink X4 | Optional (`xtc.enabled = false` turns it off). Needs `npm install` **inside `cli/`**, and a settings JSON naming a real TTF/OTF — see below. It has **no global npm bin** — it is invoked as `node /cli/index.js convert …`, which is why `xtc.command`/`xtc.args` are fully general. | | A reverse proxy for `daily.hallada.net` → `127.0.0.1:3499` | rating links must be reachable from e-readers on the internet | TLS via your existing setup. | -`data/scour-interests.opml` (the ~220 Scour interests the taste profile is seeded -from) must be readable at the path in `interests_opml`. +`data/profile.md` is the hand-maintained reader profile; its optional interests are merged +with `data/scour-interests.opml`. Both paths are configurable. --- @@ -72,7 +72,10 @@ sudo install -m0755 target/release/daily-epub /usr/local/bin/ ``` daily-epub generate [--date YYYY-MM-DD] [--dry-run] [--out DIR] [--max-articles N] [--skip-llm] daily-epub serve # rating endpoints + OPDS catalog + downloads -daily-epub profile rebuild # regenerate the taste profile from ratings (weekly inside generate) +daily-epub profile rebuild # regenerate learned profile adjustments +daily-epub ratings list --days 90 +daily-epub ratings set --article 42 --label loved --note "excellent" +daily-epub ratings clear --url https://example.com/article daily-epub backfill-social # re-poll social scores for recent articles daily-epub db migrate # run migrations (also automatic on every start) ``` @@ -114,7 +117,8 @@ Secrets belong in the environment file, never in the TOML. | `world_briefing` | `true` | Include the Wikipedia Current Events section. | | `database_path` | `/var/lib/daily-epub/daily-epub.db` | SQLite file; parent dirs are created. | | `out_dir` | `/var/lib/daily-epub/out` | Where `generate` writes artifacts before publishing. | -| `interests_opml` | `data/scour-interests.opml` | Scour interest export used to seed the taste profile. | +| `profile_path` | `data/profile.md` | Hand-maintained reader profile, loaded every run. | +| `interests_opml` | `data/scour-interests.opml` | Scour interests merged with the profile interests. | | `miniflux.base_url` | `http://127.0.0.1:8082` | Miniflux root (no `/v1`). | | `miniflux.api_key` | — | **`DAILY_EPUB_MINIFLUX__API_KEY`**. Required. | | `miniflux.page_limit` | `250` | Entries per page; Miniflux caps this at 250. | @@ -131,6 +135,10 @@ Secrets belong in the environment file, never in the TOML. | `curation.blocked_domains` | `[]` | Hosts excluded outright. | | `curation.paywall_domains` | `[]` | Extra paywalled hosts, merged with the built-in list (nytimes, wsj, ft, economist, …). | | `curation.sections` | 8 sections | The **only** section names the model may use. `World Briefing` is reserved and never offered. | +| `curation.feedback.loved_value` | `1.0` | Weight for a Loved it verdict. | +| `curation.feedback.good_value` | `0.35` | Weight for a Good verdict. | +| `curation.feedback.not_for_me_value` | `-1.0` | Weight for a Not for me verdict. | +| `curation.feedback.verdicts_in_prompt` | `60` | Recent explicit verdicts included in the system prompt. | | `publish.epub_dir` | `/srv/bookorbit/libraries/daily-epub` | Both EPUB editions land here by atomic copy, and this is the directory the OPDS feed lists. The editions are distinguished by a `(X4)` tag in **both** the filename and `dc:title` — libraries and OPDS clients list books by title, so the filename alone would make them look identical. Point a BookOrbit watched folder at it if you want its UI too. **Renamed from `bookorbit_dir`**; the old key is a hard config error. | | `publish.xtc_dir` | `/var/lib/daily-epub/xtc` | XTC artifacts. **Not** listed in the OPDS feed — CrossPoint cannot acquire them — but downloadable at `/files/xtc/` for sideloading. | | `xtc.enabled` | `true` | Set `false` to skip the converter entirely. | @@ -356,9 +364,8 @@ curl -s https://daily.hallada.net/healthz curl -s https://daily.hallada.net/opds/daily.xml | head curl -s https://daily.hallada.net/issues.json | jq '.[0]' -# 7. Feedback loop: tap 👍 in KOReader, then -sqlite3 /var/lib/daily-epub/daily-epub.db 'select * from ratings;' -sqlite3 /var/lib/daily-epub/daily-epub.db 'select * from feed_priors;' +# 7. Feedback loop: tap Loved it / Good / Not for me in KOReader, then +sqlite3 /var/lib/daily-epub/daily-epub.db 'select * from rating_events order by event_at desc;' # 8. Watch cost and quality for a week sqlite3 /var/lib/daily-epub/daily-epub.db \ diff --git a/config.example.toml b/config.example.toml index 3dc7a07..74eeb33 100644 --- a/config.example.toml +++ b/config.example.toml @@ -22,7 +22,8 @@ database_path = "/var/lib/daily-epub/daily-epub.db" # Default output directory for generated artifacts (overridden by `--out`). out_dir = "/var/lib/daily-epub/out" -# Path to the Scour interests OPML used to seed the taste profile (§3.6). +# Hand-maintained reader profile and Scour interests merged into the system prompt. +profile_path = "data/profile.md" interests_opml = "data/scour-interests.opml" [miniflux] @@ -59,6 +60,12 @@ sections = [ "From the Blogroll", ] +[curation.feedback] +loved_value = 1.0 +good_value = 0.35 +not_for_me_value = -1.0 +verdicts_in_prompt = 60 + [publish] # Where both EPUB editions land, and what the OPDS feed lists. BookOrbit is # optional — it just watches this folder if you run it. diff --git a/data/profile.md b/data/profile.md new file mode 100644 index 0000000..defb301 --- /dev/null +++ b/data/profile.md @@ -0,0 +1,30 @@ +# Reader profile + +## Who he is +A software engineer in the Boston area who reads on e-ink in the morning. He would +rather read six excellent long pieces than thirty adequate short ones. He reads across +an unusually wide range of subjects and does not need a topic to be professionally +useful to enjoy it. + +## What he wants +- Long-form and high-effort above all: essays, deep dives, post-mortems, field notes, + annotated experiments, thorough explainers, personal narratives with real specificity. + Length is a proxy, not the goal. +- Any topic, if the writing is excellent. +- Social proof is evidence a critical audience read it, not a verdict. +- Boston and New England local news: city government, transit, universities, civic stories. +- Ultra-niche community news: small scenes with their own vocabulary. +- World and US news kept light and neutral (the World Briefing covers it separately). + +## What he does not want +Press releases and funding announcements dressed as news; SEO listicles; link roundups; +changelogs without analysis; sponsored content; crypto and engagement bait; rewrites of a +story he can read at the source; culture-war outrage; one paragraph stretched to five. + +## How to judge +Would he still be glad he read this an hour later? Reward specificity, first-hand +experience, honest uncertainty, and prose with a human behind it. Penalize padding, +unsourced confidence, and summaries of other people's work. + +## Interests +(optional: one per line; merged with data/scour-interests.opml) diff --git a/migrations/0002_curation_v2.sql b/migrations/0002_curation_v2.sql new file mode 100644 index 0000000..7424013 --- /dev/null +++ b/migrations/0002_curation_v2.sql @@ -0,0 +1,96 @@ +-- Personalized curation v2 schema (plan 2026-09-02 §7). +-- `scores` intentionally remains until migration 0003 (step 4). + +CREATE TABLE rating_events ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + article_id INTEGER NOT NULL REFERENCES articles(id) ON DELETE CASCADE, + issue_date TEXT, + kind TEXT NOT NULL CHECK (kind IN ('explicit', 'implicit')), + source TEXT NOT NULL, + label TEXT NOT NULL, + value REAL NOT NULL, + note TEXT, + event_at TEXT NOT NULL +); + +CREATE INDEX idx_rating_events_article ON rating_events(article_id, event_at); +CREATE INDEX idx_rating_events_at ON rating_events(event_at); + +-- Filled in by step 3. +CREATE TABLE article_embeddings ( + article_id INTEGER PRIMARY KEY REFERENCES articles(id) ON DELETE CASCADE, + model TEXT NOT NULL, + dimension INTEGER NOT NULL, + input_hash TEXT NOT NULL, + embedding BLOB NOT NULL, + created_at TEXT NOT NULL +); + +-- Filled in by step 3. +CREATE TABLE interest_embeddings ( + interest TEXT PRIMARY KEY, + model TEXT NOT NULL, + dimension INTEGER NOT NULL, + embedding BLOB NOT NULL, + created_at TEXT NOT NULL +); + +-- Triage rows are filled in by step 4; deep rows by step 5. +CREATE TABLE article_assessments ( + article_id INTEGER NOT NULL REFERENCES articles(id) ON DELETE CASCADE, + stage TEXT NOT NULL CHECK (stage IN ('triage', 'deep')), + model TEXT NOT NULL, + prompt_version INTEGER NOT NULL, + profile_version INTEGER, + score REAL, + fit REAL, + kind TEXT, + facets_json TEXT, + rationale TEXT, + category TEXT, + paywalled_guess INTEGER NOT NULL DEFAULT 0, + assessed_at TEXT NOT NULL, + PRIMARY KEY (article_id, stage) +); + +CREATE INDEX idx_article_assessments_at ON article_assessments(assessed_at); + +-- Filled in by step 3. +CREATE TABLE candidate_runs ( + run_id INTEGER NOT NULL REFERENCES runs(id) ON DELETE CASCADE, + article_id INTEGER NOT NULL REFERENCES articles(id) ON DELETE CASCADE, + stage TEXT NOT NULL, + excluded_reason TEXT, + admitted_by TEXT, + signals_json TEXT NOT NULL, + utility REAL, + rank_utility INTEGER, + cluster_id INTEGER, + cluster_rank INTEGER, + editor_why TEXT, + PRIMARY KEY (run_id, article_id) +); + +CREATE INDEX idx_candidate_runs_article ON candidate_runs(article_id); +CREATE INDEX idx_candidate_runs_run_stage ON candidate_runs(run_id, stage); + +-- Filled in by later telemetry/provider steps. +ALTER TABLE runs ADD COLUMN config_json TEXT; +ALTER TABLE runs ADD COLUMN provider_costs_json TEXT; +-- Filled in by step 2. +ALTER TABLE issue_articles ADD COLUMN why TEXT; + +INSERT INTO rating_events + (article_id, issue_date, kind, source, label, value, note, event_at) +SELECT article_id, + issue_date, + 'explicit', + 'migration', + CASE vote WHEN 1 THEN 'loved' ELSE 'not_for_me' END, + CASE vote WHEN 1 THEN 1.0 ELSE -1.0 END, + NULL, + rated_at +FROM ratings; + +DROP TABLE ratings; +DROP TABLE feed_priors; diff --git a/src/auth.rs b/src/auth.rs index 7bf5f26..cf45dad 100644 --- a/src/auth.rs +++ b/src/auth.rs @@ -5,14 +5,14 @@ //! both sides. It lives here and nowhere else: //! //! ```text -//! message = "{issue_date}/{article_id}/{up|down}" +//! message = "{issue_date}/{article_id}/{loved|good|down}" //! token = hex(hmac_sha256(secret, message))[..16] //! link = {public_url}/r/{issue_date}/{article_id}/{vote}?t={token} //! ``` //! //! Pinned test vector, asserted from three places (here, `epub::build`, //! `tests/m7_server.rs`): `secret = "test-secret"`, date `2026-08-15`, -//! article `42`, `up` → `3b314cf7e6d8f50f`. +//! article `42`, `loved` (with legacy `up` verification). use hmac::{Hmac, KeyInit, Mac}; use jiff::civil::Date; @@ -23,17 +23,26 @@ use crate::types::{ArticleId, Vote}; /// Characters of the hex HMAC kept in rating links (§3.9). pub const TOKEN_LEN: usize = 16; -/// The exact signed string: `{issue_date}/{article_id}/{up|down}` (§3.9). +/// The exact signed string: `{issue_date}/{article_id}/{loved|good|down}` (§3.9). pub fn rating_message(issue_date: Date, article_id: ArticleId, vote: Vote) -> String { format!("{issue_date}/{article_id}/{}", vote.as_str()) } /// `hex(hmac_sha256(secret, "{issue_date}/{article_id}/{vote}"))[..16]` (§3.9). pub fn rating_token(secret: &str, issue_date: Date, article_id: ArticleId, vote: Vote) -> String { + rating_token_for_segment(secret, issue_date, article_id, vote.as_str()) +} + +fn rating_token_for_segment( + secret: &str, + issue_date: Date, + article_id: ArticleId, + segment: &str, +) -> String { // `Hmac` derives a fixed-size key from any input length, so this never fails. let mut mac = as KeyInit>::new_from_slice(secret.as_bytes()) .expect("HMAC accepts keys of any length"); - mac.update(rating_message(issue_date, article_id, vote).as_bytes()); + mac.update(format!("{issue_date}/{article_id}/{segment}").as_bytes()); let digest = hex::encode(mac.finalize().into_bytes()); digest[..TOKEN_LEN].to_string() } @@ -46,10 +55,16 @@ pub fn verify_token( vote: Vote, token: &str, ) -> bool { - constant_time_eq( - rating_token(secret, issue_date, article_id, vote).as_bytes(), - token.as_bytes(), - ) + let current = rating_token(secret, issue_date, article_id, vote); + if constant_time_eq(current.as_bytes(), token.as_bytes()) { + return true; + } + // Already-published `up` links were signed over the literal legacy segment. + vote == Vote::Loved + && constant_time_eq( + rating_token_for_segment(secret, issue_date, article_id, "up").as_bytes(), + token.as_bytes(), + ) } /// Length-independent, data-independent byte comparison. @@ -67,7 +82,7 @@ pub fn constant_time_eq(a: &[u8], b: &[u8]) -> bool { } /// Full rating URL embedded in an article footer: -/// `{public_url}/r/{date}/{article_id}/{up|down}?t={token}` (§3.9). +/// `{public_url}/r/{date}/{article_id}/{loved|good|down}?t={token}` (§3.9). pub fn rating_url( public_url: &str, secret: &str, @@ -92,50 +107,55 @@ mod tests { } #[test] - fn token_matches_the_shared_vector() { - assert_eq!(rating_message(date(), 42, Vote::Up), "2026-08-15/42/up"); - assert_eq!( - rating_token("test-secret", date(), 42, Vote::Up), - "3b314cf7e6d8f50f" - ); - assert_eq!(rating_token("test-secret", date(), 42, Vote::Up).len(), 16); + fn all_three_tokens_verify_and_are_distinct() { + let votes = [Vote::Loved, Vote::Good, Vote::NotForMe]; + let tokens: Vec = votes + .iter() + .map(|vote| rating_token("test-secret", date(), 42, *vote)) + .collect(); + assert_eq!(tokens.len(), 3); + assert!(tokens.iter().all(|token| token.len() == TOKEN_LEN)); + assert_ne!(tokens[0], tokens[1]); + assert_ne!(tokens[1], tokens[2]); + for (vote, token) in votes.into_iter().zip(tokens) { + assert!(verify_token("test-secret", date(), 42, vote, &token)); + } } #[test] - fn tokens_are_per_article_and_per_vote() { - let up = rating_token("s", date(), 42, Vote::Up); - assert_ne!(up, rating_token("s", date(), 42, Vote::Down)); - assert_ne!(up, rating_token("s", date(), 43, Vote::Up)); - assert_ne!(up, rating_token("other", date(), 42, Vote::Up)); - let tomorrow: Date = "2026-08-16".parse().unwrap(); - assert_ne!(up, rating_token("s", tomorrow, 42, Vote::Up)); - } - - #[test] - fn verification_is_exact() { + fn legacy_up_token_still_verifies_as_loved() { + let legacy = rating_token_for_segment("test-secret", date(), 42, "up"); + assert_eq!(legacy, "3b314cf7e6d8f50f"); assert!(verify_token( - "s", + "test-secret", date(), 42, - Vote::Up, - &rating_token("s", date(), 42, Vote::Up) + Vote::Loved, + &legacy )); - assert!(!verify_token("s", date(), 42, Vote::Up, "deadbeefdeadbeef")); - assert!(!verify_token("s", date(), 42, Vote::Up, "")); - assert!(!verify_token("s", date(), 42, Vote::Up, "short")); + } + + #[test] + fn verification_rejects_tampering() { + let token = rating_token("s", date(), 42, Vote::Loved); + assert!(!verify_token("s", date(), 42, Vote::Good, &token)); + assert!(!verify_token("s", date(), 43, Vote::Loved, &token)); + assert!(!verify_token("other", date(), 42, Vote::Loved, &token)); + assert!(!verify_token("s", date(), 42, Vote::Loved, "short")); } #[test] fn url_shape_matches_the_spec() { + let token = rating_token("test-secret", date(), 42, Vote::Good); assert_eq!( rating_url( "https://daily.hallada.net/", "test-secret", date(), 42, - Vote::Up + Vote::Good ), - "https://daily.hallada.net/r/2026-08-15/42/up?t=3b314cf7e6d8f50f" + format!("https://daily.hallada.net/r/2026-08-15/42/good?t={token}") ); } } diff --git a/src/config.rs b/src/config.rs index 4a45614..3c42a58 100644 --- a/src/config.rs +++ b/src/config.rs @@ -69,6 +69,8 @@ pub struct Config { pub out_dir: PathBuf, /// Scour interests OPML used to seed the taste profile (§3.6). pub interests_opml: PathBuf, + /// Hand-maintained reader profile loaded for every curation run (§8.2). + pub profile_path: PathBuf, pub miniflux: MinifluxConfig, pub deepseek: DeepseekConfig, @@ -92,6 +94,7 @@ impl Default for Config { database_path: PathBuf::from("/var/lib/daily-epub/daily-epub.db"), out_dir: PathBuf::from("/var/lib/daily-epub/out"), interests_opml: PathBuf::from("data/scour-interests.opml"), + profile_path: PathBuf::from("data/profile.md"), miniflux: MinifluxConfig::default(), deepseek: DeepseekConfig::default(), curation: CurationConfig::default(), @@ -172,6 +175,7 @@ pub struct CurationConfig { pub paywall_domains: Vec, /// The only section names the LLM may use (§3.6 stage B). pub sections: Vec, + pub feedback: FeedbackConfig, } impl Default for CurationConfig { @@ -193,6 +197,28 @@ impl Default for CurationConfig { .iter() .map(|s| s.to_string()) .collect(), + feedback: FeedbackConfig::default(), + } + } +} + +/// `[curation.feedback]` — explicit verdict weights and prompt history (§6, §8.4). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields, default)] +pub struct FeedbackConfig { + pub loved_value: f64, + pub good_value: f64, + pub not_for_me_value: f64, + pub verdicts_in_prompt: usize, +} + +impl Default for FeedbackConfig { + fn default() -> Self { + Self { + loved_value: 1.0, + good_value: 0.35, + not_for_me_value: -1.0, + verdicts_in_prompt: 60, } } } @@ -382,6 +408,9 @@ mod tests { assert_eq!(c.max_daily_usd, 2.0); assert!(c.world_briefing); assert_eq!(c.deepseek.model, "deepseek-v4-flash"); + assert_eq!(c.profile_path, PathBuf::from("data/profile.md")); + assert_eq!(c.curation.feedback.good_value, 0.35); + assert_eq!(c.curation.feedback.verdicts_in_prompt, 60); assert_eq!(c.xtc.format, XtcFormat::Xtch); assert_eq!(c.curation.sections.len(), 8); c.validate().unwrap(); diff --git a/src/curate/prefilter.rs b/src/curate/prefilter.rs index e91215f..07ad7b0 100644 --- a/src/curate/prefilter.rs +++ b/src/curate/prefilter.rs @@ -12,15 +12,14 @@ //! | came via Scour | +8 | §3.5 (already matched a stated interest) | //! | came via HN frontpage | +8 | §3.5 | //! | carried by several feeds | 0 … +8 | §3.2 (multi-source *is* social proof) | -//! | feed prior | −12 … +12 | §3.9 beta-smoothed upvote rate, neutral at 0.5 | //! | excerpt only | −20 | §3.5 (penalized, never banned — §7) | //! | roundup/release-notes title | −15 | §3.5 | //! | blocked domain | excluded | §3.5 | -use std::collections::{HashMap, HashSet}; +use std::collections::HashSet; use crate::config::{Config, CurationConfig}; -use crate::types::{Article, ArticleId, FeedId, FeedPrior, ScoredArticle, SourceKind}; +use crate::types::{Article, ArticleId, FeedId, ScoredArticle, SourceKind}; /// Title patterns that mark low-effort posts: link roundups, release notes, /// sponsor posts (§3.5). @@ -64,7 +63,6 @@ pub const MAX_SOCIAL_POINTS: f64 = 25.0; pub const SCOUR_BONUS: f64 = 8.0; pub const HN_FRONTPAGE_BONUS: f64 = 8.0; pub const MAX_MULTI_SOURCE_POINTS: f64 = 8.0; -pub const MAX_FEED_PRIOR_POINTS: f64 = 12.0; pub const EXCERPT_ONLY_PENALTY: f64 = 20.0; pub const ROUNDUP_TITLE_PENALTY: f64 = 15.0; @@ -75,8 +73,6 @@ const SOCIAL_SATURATION: f64 = 6.0; /// Everything the pre-filter needs beyond the articles themselves (§3.5, §3.9). #[derive(Debug, Clone, Default)] pub struct PrefilterContext { - /// Per-feed Bayesian upvote rate from ratings history (§3.9). - pub feed_priors: HashMap, /// Article ids already published in a previous issue (§3.5). pub already_published: Vec, /// Article ids the LLM scored < [`STALE_LOW_SCORE`] recently (§3.5). @@ -94,43 +90,18 @@ impl PrefilterContext { let since = today .checked_sub(jiff::Span::new().days(STALE_LOOKBACK_DAYS)) .unwrap_or(today); - let feed_priors = db - .feed_priors() - .await? - .into_iter() - .map(|p| (p.feed_id, p)) - .collect(); let already_published = db.previously_published_ids().await?; let recently_rejected = db.recently_low_scored_ids(STALE_LOW_SCORE, since).await?; tracing::debug!( - priors = ?feed_priors_len(&feed_priors), published = already_published.len(), rejected = recently_rejected.len(), "loaded prefilter context" ); Ok(Self { - feed_priors, already_published, recently_rejected, }) } - - fn prior_for(&self, article: &Article) -> f64 { - // The cluster's feeds are all candidates; take the most favourable one, - // since a story carried by a well-rated feed is a better bet. - let mut best = self.feed_priors.get(&article.feed_id).map(FeedPrior::rate); - for source in &article.sources { - if let Some(p) = self.feed_priors.get(&source.feed_id) { - let rate = p.rate(); - best = Some(best.map_or(rate, |b: f64| b.max(rate))); - } - } - best.unwrap_or(0.5) - } -} - -fn feed_priors_len(m: &HashMap) -> usize { - m.len() } /// True when the article's feed is in `curation.always_include_feeds` (§3.5). @@ -226,9 +197,9 @@ pub fn social_points(social_score: f64) -> f64 { MAX_SOCIAL_POINTS * (social_score / SOCIAL_SATURATION).min(1.0).sqrt() } -/// Score one article 0–100 from word count, social proof, source signals, feed -/// prior, and the excerpt/roundup/blocklist penalties (§3.5). -pub fn score_article(article: &Article, ctx: &PrefilterContext, cfg: &Config) -> f64 { +/// Score one article 0–100 from word count, social proof, source signals, +/// and the excerpt/roundup/blocklist penalties (§3.5). +pub fn score_article(article: &Article, _ctx: &PrefilterContext, cfg: &Config) -> f64 { if is_blocked(article, &cfg.curation) { return 0.0; } @@ -245,9 +216,6 @@ pub fn score_article(article: &Article, ctx: &PrefilterContext, cfg: &Config) -> let extra_feeds = article.sources.len().saturating_sub(1) as f64; score += (extra_feeds * 4.0).min(MAX_MULTI_SOURCE_POINTS); - // Beta-smoothed upvote rate, neutral (0.5) contributing nothing (§3.9). - score += (ctx.prior_for(article) - 0.5) * 2.0 * MAX_FEED_PRIOR_POINTS; - if article.excerpt_only { score -= EXCERPT_ONLY_PENALTY; } @@ -288,12 +256,10 @@ pub fn run(articles: Vec
, ctx: &PrefilterContext, cfg: &Config) -> Vec< let prefilter_score = score_article(&article, ctx, cfg); let social_score = article.social_score(); - let feed_prior = ctx.prior_for(&article); scored.push(ScoredArticle { article, prefilter_score, social_score, - feed_prior, llm: None, auto_include, }); @@ -488,35 +454,6 @@ pub(crate) mod tests { ); } - #[test] - fn feed_prior_moves_the_score_both_ways() { - let cfg = cfg(); - let mut liked = PrefilterContext::default(); - liked.feed_priors.insert( - 7, - FeedPrior { - feed_id: 7, - upvotes: 18, - downvotes: 0, - included: 18, - }, - ); - let mut disliked = PrefilterContext::default(); - disliked.feed_priors.insert( - 7, - FeedPrior { - feed_id: 7, - upvotes: 0, - downvotes: 18, - included: 18, - }, - ); - let a = article(1, "Deep dive", 1200); - let neutral = score_article(&a, &PrefilterContext::default(), &cfg); - assert!(score_article(&a, &liked, &cfg) > neutral); - assert!(score_article(&a, &disliked, &cfg) < neutral); - } - #[test] fn blocked_domains_and_auto_includes_match_urls_and_ids() { let mut cfg = cfg(); @@ -563,7 +500,6 @@ pub(crate) mod tests { let ctx = PrefilterContext { already_published: vec![4], recently_rejected: vec![6], - ..PrefilterContext::default() }; let kept = run(articles, &ctx, &cfg); @@ -598,7 +534,6 @@ pub(crate) mod tests { let ctx = PrefilterContext { recently_rejected: vec![1], already_published: vec![2], - ..PrefilterContext::default() }; let kept = run(vec![a, b], &ctx, &cfg); let ids: Vec = kept.iter().map(|s| s.article.id).collect(); @@ -613,14 +548,6 @@ pub(crate) mod tests { .expect("db"); let date: jiff::civil::Date = "2026-08-15".parse().expect("date"); - db.upsert_feed_prior(&FeedPrior { - feed_id: 7, - upvotes: 4, - downvotes: 1, - included: 5, - }) - .await - .expect("prior"); sqlx::query( "INSERT INTO articles (id, canonical_url, title, first_seen) VALUES (42, 'https://example.com/42', 'Printed', '2026-08-14T00:00:00Z'), @@ -660,6 +587,5 @@ pub(crate) mod tests { let ctx = PrefilterContext::load(&db, date).await.expect("context"); assert_eq!(ctx.already_published, vec![42]); assert_eq!(ctx.recently_rejected, vec![43], "old rejects age out"); - assert!((ctx.feed_priors[&7].rate() - 5.0 / 7.0).abs() < 1e-12); } } diff --git a/src/curate/profile/mod.rs b/src/curate/profile/mod.rs index a7491b1..544ea82 100644 --- a/src/curate/profile/mod.rs +++ b/src/curate/profile/mod.rs @@ -1,13 +1,7 @@ -//! Taste profile construction (spec §3.6, §3.9). +//! Reader-profile and system-prompt construction (personalized curation v2 §8). //! -//! A ~600-word document assembled from (a) the interest names parsed out of -//! `data/scour-interests.opml`, grouped into themes, (b) hard-coded stated -//! preferences, and (c) a "learned adjustments" section regenerated weekly from -//! recent 👍/👎 ratings. Stored and versioned in `kv`. -//! -//! This document is the **system prompt** for every DeepSeek call in the run, so -//! it must be byte-identical between requests: DeepSeek's automatic prefix cache -//! is what makes the whole pipeline cost cents rather than dollars (§3.6). +//! Every run rebuilds one byte-stable prompt from the hand-maintained profile, +//! standing interests, stored weekly adjustments, and current explicit verdicts. use std::collections::BTreeSet; use std::fmt::Write as _; @@ -15,45 +9,26 @@ use std::path::Path; use anyhow::Context as _; use jiff::Timestamp; -use jiff::civil::Date; use serde::{Deserialize, Serialize}; -use sqlx::Row as _; use super::llm::LlmClient; use crate::db::{Db, KV_PROFILE_VERSION, KV_TASTE_PROFILE}; -use crate::types::{TasteProfile, Vote}; +use crate::types::{Facets, RatedArticle, TasteProfile}; -/// Rebuild cadence for the learned-adjustments section (§3.6). pub const REBUILD_INTERVAL_DAYS: i64 = 7; -/// Ratings lookback used when rewriting learned adjustments (§3.9). -pub const RATINGS_LOOKBACK_DAYS: i64 = 90; -/// `kv` key holding just the learned-adjustments block, so that re-parsing the -/// OPML never loses what the ratings taught us (§3.6). +pub const RATINGS_LOOKBACK_DAYS: i64 = 36_500; pub const KV_LEARNED_ADJUSTMENTS: &str = "taste_profile_learned"; -/// Ratings fed to one rebuild call. -const MAX_RATINGS_IN_PROMPT: usize = 400; +const MAX_RATINGS_IN_REBUILD: usize = 200; -/// Hard-coded stated preferences from the reader profile (spec §1). -pub const STATED_PREFERENCES: &str = "\ -Prefers long-form, high-effort, well-written articles on any topic. Uses social \ -proof (HN/Reddit/Lobsters upvotes and comment counts) as a quality proxy. Wants \ -tech news, light general/US world news (Wikipedia Current Events style, neutral), \ -Boston-area news, and ultra-niche community news."; +pub const NO_LEARNED_ADJUSTMENTS: &str = "No reader ratings have been collected yet. Judge purely on the stated preferences and interests above."; -/// Placeholder used until the first ratings arrive (§3.6c). -pub const NO_LEARNED_ADJUSTMENTS: &str = "No reader ratings have been collected yet. Judge purely on the stated \ - preferences and interests above."; +/// The only reader-profile prose that remains in code (§8.2). +const EDITOR_IN_CHIEF_FRAMING: &str = "You are the editor-in-chief of *The Daily EPUB*, a personal morning newspaper assembled every day for exactly one reader. Everything you are asked to do — score, select, place, summarize, introduce — serves his taste, not a general audience's. When a judgement call is close, re-read this profile and decide the way he would."; // --------------------------------------------------------------------------- -// OPML parsing (§3.6a) +// Interest and profile-file parsing // --------------------------------------------------------------------------- -/// Parse interest names out of the Scour OPML (§3.6). -/// -/// The file is one long line of `` elements; -/// we take every `text` attribute, XML-unescape it, trim it, and de-duplicate -/// case-insensitively (the export contains both `Self-hosting` and -/// `Self-Hosting`). Order follows the document so the result is deterministic. pub fn parse_interests(opml_path: &Path) -> anyhow::Result> { let raw = std::fs::read_to_string(opml_path) .with_context(|| format!("reading the interests OPML at {}", opml_path.display()))?; @@ -68,19 +43,15 @@ pub fn parse_interests(opml_path: &Path) -> anyhow::Result> { Ok(interests) } -/// [`parse_interests`] over an in-memory document (also the unit-test seam). pub fn parse_interests_str(raw: &str) -> Vec { - let mut seen: BTreeSet = BTreeSet::new(); + let mut seen = BTreeSet::new(); let mut out = Vec::new(); for chunk in raw.split("text=\"").skip(1) { let Some((value, _)) = chunk.split_once('"') else { continue; }; let name = xml_unescape(value).trim().to_string(); - if name.is_empty() { - continue; - } - if seen.insert(name.to_lowercase()) { + if !name.is_empty() && seen.insert(name.to_lowercase()) { out.push(name); } } @@ -99,102 +70,156 @@ fn xml_unescape(s: &str) -> String { .replace("&", "&") } -pub mod themes; +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ProfileFile { + /// Original Markdown with every `## Interests` section removed. + pub body: String, + pub interests: Vec, +} +/// Remove any `## Interests` section and parse its non-empty lines as interests. +/// A leading `- ` is stripped; all other profile bytes pass through unchanged. +pub fn parse_profile_str(raw: &str) -> ProfileFile { + let mut body = String::with_capacity(raw.len()); + let mut interests = Vec::new(); + let mut in_interests = false; + + for line in raw.split_inclusive('\n') { + let heading = line.trim_end_matches(['\r', '\n']).trim(); + if heading.eq_ignore_ascii_case("## Interests") { + in_interests = true; + continue; + } + if in_interests && heading.starts_with("## ") { + in_interests = false; + } + if in_interests { + let interest = heading.strip_prefix("- ").unwrap_or(heading).trim(); + if !interest.is_empty() + && !interest.eq_ignore_ascii_case( + "(optional: one per line; merged with data/scour-interests.opml)", + ) + { + interests.push(interest.to_string()); + } + } else { + body.push_str(line); + } + } + ProfileFile { body, interests } +} + +pub fn load_profile(path: &Path) -> anyhow::Result { + match std::fs::read_to_string(path) { + Ok(raw) => Ok(parse_profile_str(&raw)), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + tracing::warn!(path = %path.display(), "profile file is missing; using OPML interests only"); + Ok(ProfileFile { + body: String::new(), + interests: Vec::new(), + }) + } + Err(error) => { + Err(error).with_context(|| format!("reading the reader profile at {}", path.display())) + } + } +} + +fn union_interests(opml: Vec, profile: Vec) -> Vec { + let mut seen = BTreeSet::new(); + let mut out = Vec::new(); + for interest in opml.into_iter().chain(profile) { + let interest = interest.trim(); + if !interest.is_empty() && seen.insert(interest.to_lowercase()) { + out.push(interest.to_string()); + } + } + out +} + +pub mod themes; pub use themes::group_into_themes; // --------------------------------------------------------------------------- -// Document assembly (§3.6) +// Prompt assembly // --------------------------------------------------------------------------- -/// The invariant part of the profile: who the reader is and how to judge for him. -/// Kept as one constant so the prompt bytes never drift between calls (§3.6). -const PROFILE_PREAMBLE: &str = "\ -You are the editor-in-chief of *The Daily EPUB*, a personal morning newspaper \ -assembled every day for exactly one reader. Everything you are asked to do — \ -score, select, place, summarize, introduce — serves his taste, not a general \ -audience's. When a judgement call is close, re-read this profile and decide the \ -way he would. +fn verdict_label(label: &str) -> &str { + match label { + "loved" => "LOVED", + "good" => "GOOD", + "not_for_me" => "NOT FOR ME", + other => other, + } +} -## The reader +fn one_line(text: &str) -> String { + text.split_whitespace().collect::>().join(" ") +} -A software engineer in the Boston area who reads on e-ink in the morning. He \ -would rather read six excellent long pieces than thirty adequate short ones. He \ -reads across an unusually wide range of subjects and does not need a topic to be \ -professionally useful to enjoy it. +/// Assemble sections in the exact cache-friendly order required by §8.4. +pub fn build( + profile_body: &str, + interests: &[String], + learned_adjustments: &str, + ratings: &[RatedArticle], + verdict_limit: usize, +) -> String { + let mut doc = String::with_capacity(16 * 1024); + doc.push_str(EDITOR_IN_CHIEF_FRAMING); + doc.push_str("\n\n"); -## What he wants + if !profile_body.is_empty() { + doc.push_str(profile_body); + if !profile_body.ends_with('\n') { + doc.push('\n'); + } + doc.push('\n'); + } -- **Long-form and high-effort above all.** Essays, deep dives, post-mortems, \ -field notes, annotated experiments, thorough explainers, personal narratives with \ -real specificity. Length is a proxy, not the goal: what he is buying is evident \ -effort and a point of view. -- **Any topic, if the writing is excellent.** A brilliant piece on medieval \ -bookbinding beats a competent one on his favourite language. Do not reject \ -something merely because it sits outside the interest list below. -- **Social proof as a quality signal, not a ranking.** Hundreds of HN or Reddit \ -points and a busy comment thread mean the piece survived contact with a critical \ -audience — treat it as evidence, then judge the writing yourself. A quiet post \ -from a good blog can outrank a viral one. -- **Boston and New England local news** — city government, transit, universities, \ -neighbourhood and civic stories. -- **Ultra-niche community news.** Small scenes with their own vocabulary — a \ -mailing-list argument, a hobby project's release story, a subculture's internal \ -debate — are a feature of this paper, not a distraction. -- **World and US news kept light and neutral.** Wikipedia-Current-Events register: \ -what happened, who is involved, no outrage, no opinion columns. The World \ -Briefing section is compiled separately; do not fill the paper with wire copy. - -## What he does not want - -Press releases and funding announcements dressed as news; SEO listicles; \ -link-roundup and \"this week in X\" posts; changelogs and release notes without \ -analysis; sponsored content and thinly disguised marketing; crypto and \ -engagement-bait; rewrites of a story he can read at the source; culture-war \ -outrage; anything whose substance is one paragraph stretched to five. - -## How to judge - -Ask: *would he still be glad he read this an hour later?* Reward specificity, \ -first-hand experience, honest uncertainty, and prose with a human behind it. \ -Penalize padding, unsourced confidence, and summaries of other people's work. \ -Prefer the primary source over the aggregator when both are present."; - -/// Assemble the full profile document from interests, stated preferences and the -/// current learned-adjustments block (§3.6). -/// -/// Pure and deterministic: the same inputs always produce the same bytes. -pub fn build(interests: &[String], learned_adjustments: &str) -> String { - let mut doc = String::with_capacity(8 * 1024); - doc.push_str("# The Daily EPUB — reader taste profile\n\n"); - doc.push_str(PROFILE_PREAMBLE); - doc.push_str("\n\n## Stated preferences (verbatim)\n\n"); - doc.push_str(STATED_PREFERENCES); - doc.push_str("\n\n## Standing interests\n\n"); - doc.push_str( - "These are his ~220 subscribed interest topics, grouped. They raise the \ - floor for a match, but never cap the paper: an outstanding article on \ - none of these still belongs.\n\n", - ); + doc.push_str("## Standing interests\n\n"); + doc.push_str("These are his subscribed interest topics, grouped. They raise the floor for a match, but never cap the paper: an outstanding article on none of these still belongs.\n\n"); for (theme, members) in group_into_themes(interests) { let _ = writeln!(doc, "- **{}**: {}", theme, members.join(", ")); } - doc.push_str("\n## Learned adjustments (rebuilt weekly from 👍/👎 ratings)\n\n"); + + doc.push_str("\n## Learned adjustments (rebuilt weekly from ratings)\n\n"); let learned = learned_adjustments.trim(); doc.push_str(if learned.is_empty() { NO_LEARNED_ADJUSTMENTS } else { learned }); - doc.push('\n'); + + doc.push_str("\n\n## Recent verdicts\n\n"); + for rating in ratings.iter().take(verdict_limit) { + let summary = rating + .summary + .as_deref() + .map(one_line) + .filter(|summary| !summary.is_empty()) + .unwrap_or_else(|| "no summary available".to_string()); + let feed = if rating.feed_title.trim().is_empty() { + "unknown" + } else { + rating.feed_title.trim() + }; + let _ = writeln!( + doc, + "{} | {} | {} | {}", + verdict_label(&rating.label), + one_line(&rating.title), + one_line(feed), + summary + ); + } doc } // --------------------------------------------------------------------------- -// Persistence (§3.6 `kv`) +// Persistence and per-run loading // --------------------------------------------------------------------------- -/// `kv[profile_version]` payload. #[derive(Debug, Clone, Serialize, Deserialize)] struct ProfileVersion { version: i64, @@ -206,59 +231,81 @@ async fn stored_version(db: &Db) -> anyhow::Result> { return Ok(None); }; match serde_json::from_str::(&raw) { - Ok(v) => { - let built = v.built_at.parse::().unwrap_or_else(|_| { - tracing::warn!(value = %v.built_at, "unparseable profile build time"); + Ok(version) => { + let built_at = version.built_at.parse::().unwrap_or_else(|_| { + tracing::warn!(value = %version.built_at, "unparseable profile build time"); Timestamp::UNIX_EPOCH }); - Ok(Some((v.version, built))) + Ok(Some((version.version, built_at))) } - Err(e) => { - tracing::warn!(error = %e, "unparseable kv[profile_version]; treating as absent"); + Err(error) => { + tracing::warn!(%error, "unparseable kv[profile_version]; treating as absent"); Ok(None) } } } -async fn store(db: &Db, profile: &TasteProfile, learned: &str) -> anyhow::Result<()> { - db.kv_set(KV_TASTE_PROFILE, &profile.text).await?; - db.kv_set(KV_LEARNED_ADJUSTMENTS, learned).await?; - let version = serde_json::to_string(&ProfileVersion { - version: profile.version, - built_at: profile.built_at.to_string(), +async fn store_version(db: &Db, version: i64, built_at: Timestamp) -> anyhow::Result<()> { + let json = serde_json::to_string(&ProfileVersion { + version, + built_at: built_at.to_string(), })?; - db.kv_set(KV_PROFILE_VERSION, &version).await?; + db.kv_set(KV_PROFILE_VERSION, &json).await?; Ok(()) } -/// Load the stored profile, building a default one on first run (§3.6). -pub async fn load_or_build(db: &Db, opml_path: &Path) -> anyhow::Result { - if let Some(text) = db.kv_get(KV_TASTE_PROFILE).await?.filter(|t| !t.is_empty()) { - let (version, built_at) = stored_version(db).await?.unwrap_or((1, Timestamp::now())); - tracing::debug!(version, chars = text.len(), "loaded stored taste profile"); - return Ok(TasteProfile { - text, - version, - built_at, - }); - } - let interests = parse_interests(opml_path)?; +async fn prompt_inputs( + db: &Db, + opml_path: &Path, + profile_path: &Path, +) -> anyhow::Result<(ProfileFile, Vec, Vec, String)> { + let opml = parse_interests(opml_path)?; + let profile = load_profile(profile_path)?; + let interests = union_interests(opml, profile.interests.clone()); + let ratings = db.current_ratings(RATINGS_LOOKBACK_DAYS).await?; let learned = db.kv_get(KV_LEARNED_ADJUSTMENTS).await?.unwrap_or_default(); - let profile = TasteProfile { - text: build(&interests, &learned), - version: 1, - built_at: Timestamp::now(), + Ok((profile, interests, ratings, learned)) +} + +/// Rebuild the complete system prompt from its live inputs on every run. +pub async fn load_or_build( + db: &Db, + opml_path: &Path, + profile_path: &Path, + verdict_limit: usize, +) -> anyhow::Result { + let (profile_file, interests, ratings, learned) = + prompt_inputs(db, opml_path, profile_path).await?; + let (version, built_at) = match stored_version(db).await? { + Some(stored) => stored, + None => { + let built_at = Timestamp::now(); + store_version(db, 1, built_at).await?; + (1, built_at) + } }; - store(db, &profile, &learned).await?; - tracing::info!( + let profile = TasteProfile { + text: build( + &profile_file.body, + &interests, + &learned, + &ratings, + verdict_limit, + ), + version, + built_at, + }; + db.kv_set(KV_TASTE_PROFILE, &profile.text).await?; + tracing::debug!( + version, interests = interests.len(), + verdicts = ratings.len().min(verdict_limit), chars = profile.text.len(), - "built the initial taste profile" + "rebuilt the taste profile prompt" ); Ok(profile) } -/// True when the stored profile is older than [`REBUILD_INTERVAL_DAYS`] (§3.6). pub async fn is_stale(db: &Db) -> anyhow::Result { let Some((_, built_at)) = stored_version(db).await? else { return Ok(true); @@ -267,169 +314,112 @@ pub async fn is_stale(db: &Db) -> anyhow::Result { Ok(age_days >= REBUILD_INTERVAL_DAYS) } -/// The automatic weekly rebuild the pipeline calls before curating (§3.6). -/// -/// Rebuilds only when the stored profile is at least a week old *and* there are -/// ratings to learn from; otherwise returns the profile unchanged. Failures are -/// non-fatal — a stale profile still curates fine. pub async fn weekly_rebuild_if_due( db: &Db, llm: &LlmClient, opml_path: &Path, + profile_path: &Path, + verdict_limit: usize, ) -> anyhow::Result> { if !is_stale(db).await? { return Ok(None); } - if recent_ratings(db).await?.is_empty() { + if db.current_ratings(RATINGS_LOOKBACK_DAYS).await?.is_empty() { tracing::debug!("profile is stale but there are no ratings to learn from"); return Ok(None); } tracing::info!("taste profile is over a week old; rebuilding learned adjustments"); - Ok(Some(rebuild(db, llm, opml_path).await?)) + Ok(Some( + rebuild(db, llm, opml_path, profile_path, verdict_limit).await?, + )) } // --------------------------------------------------------------------------- -// Rebuild (§3.6c, §3.9) +// Weekly learned-adjustments rebuild // --------------------------------------------------------------------------- -/// A rated article as fed to the learned-adjustments prompt (§3.6c). -#[derive(Debug, Clone, PartialEq)] -pub struct RatedArticle { - pub vote: Vote, - pub title: String, - pub feed_title: String, - pub category: String, - /// The stage-A category the model itself assigned, when we have one. - pub llm_category: String, -} +pub const LEARNED_ADJUSTMENTS_PROMPT: &str = r#"TASK: rewrite the "Learned adjustments" section of the reader profile in your system prompt, using only the rating history below. -/// Recent ratings joined to article titles, feeds and categories (§3.6c). -/// -/// `db.rs` exposes `recent_ratings_detailed`, but it returns titles only; the -/// prompt is much more useful with the feed and category attached. -pub async fn recent_ratings(db: &Db) -> anyhow::Result> { - // Timestamp arithmetic only accepts uniform units, so days become hours. - let since = Timestamp::now() - .checked_sub(jiff::Span::new().hours(RATINGS_LOOKBACK_DAYS * 24)) - .unwrap_or(Timestamp::UNIX_EPOCH); - let since_date = since.to_zoned(jiff::tz::TimeZone::UTC).date(); - let rows = sqlx::query( - "SELECT r.vote AS vote, - COALESCE(a.title, '') AS title, - COALESCE(e.feed_title, '') AS feed_title, - COALESCE(e.category, '') AS category, - COALESCE((SELECT s.llm_category FROM scores s - WHERE s.article_id = a.id AND s.llm_category IS NOT NULL - ORDER BY s.run_date DESC LIMIT 1), '') AS llm_category - FROM ratings r - JOIN articles a ON a.id = r.article_id - LEFT JOIN entries e ON e.id = a.best_entry_id - WHERE r.issue_date >= ? - ORDER BY r.rated_at DESC - LIMIT ?", - ) - .bind(since_date.to_string()) - .bind(MAX_RATINGS_IN_PROMPT as i64) - .fetch_all(db.pool()) - .await - .context("loading recent ratings for the profile rebuild")?; +Each line is an explicit verdict with the article title, feed, summary, any deep-assessment facets, and the operator's note. - Ok(rows - .iter() - .map(|r| RatedArticle { - vote: if r.get::("vote") >= 0 { - Vote::Up - } else { - Vote::Down - }, - title: r.get("title"), - feed_title: r.get("feed_title"), - category: r.get("category"), - llm_category: r.get("llm_category"), - }) - .collect()) -} +Look for patterns, not one-offs. Treat the stated preferences as a strong prior, not a rule. When repeated, recent behaviour clearly conflicts with an older stated preference, say so. Do not override a stated preference on one or two ratings. -/// Instruction block for the weekly learned-adjustments rewrite (§3.6c). -pub const LEARNED_ADJUSTMENTS_PROMPT: &str = "\ -TASK: rewrite the \"Learned adjustments\" section of the reader profile in your \ -system prompt, using only the rating history below. +Write 120–200 words as 4–8 bullet points, each one imperative and usable while scoring. One bullet is required and must begin "Diversity check:": name any subject or format that is starting to dominate the loved list and should not crowd out the rest of the paper. Do not mention specific article titles, rating counts, or this instruction. If the history is too thin to support any pattern, say so in one sentence instead of inventing one, while still including the Diversity check bullet. -Each line is a thumbs-up or thumbs-down the reader gave an article that appeared \ -in a past issue, with the article's title, the feed it came from, and its \ -category. - -Look for patterns, not one-offs. Good adjustments name a *kind* of article and a \ -*reason*: \"consistently downvotes vendor engineering-blog posts that are really \ -product announcements\"; \"consistently upvotes database-internals deep dives, \ -even very long ones\"; \"lukewarm on AI-industry news, warm on hands-on LLM \ -tinkering\". Ignore patterns supported by fewer than two ratings, and never \ -contradict the stated preferences — refine them. - -Write 120–200 words as 4–8 bullet points, each one imperative and usable while \ -scoring (\"Rank X higher\", \"Be sceptical of Y\"). Do not mention specific \ -article titles, the rating counts, or this instruction. If the history is too \ -thin to support any pattern, say so in one sentence instead of inventing one. - -Return JSON exactly: {\"learned_adjustments\": \"\"} +Return JSON exactly: {"learned_adjustments": ""} RATING HISTORY (newest first): -"; +"#; -/// The weekly rewrite's JSON envelope. #[derive(Debug, Clone, Deserialize)] struct LearnedAdjustmentsResponse { #[serde(default)] learned_adjustments: String, } -/// Render the rating history block of the rebuild prompt (§3.6c). +fn facets_line(facets: Option<&Facets>) -> Option { + let facets = facets?; + let values = [ + facets.format.as_deref(), + facets.depth.as_deref(), + facets.evidence.as_deref(), + facets.technicality.as_deref(), + facets.topic_group.as_deref(), + ] + .into_iter() + .flatten() + .filter(|value| !value.trim().is_empty()) + .collect::>(); + (!values.is_empty()).then(|| values.join("/")) +} + pub fn build_rebuild_prompt(ratings: &[RatedArticle]) -> String { let mut prompt = String::from(LEARNED_ADJUSTMENTS_PROMPT); - let (mut up, mut down) = (0usize, 0usize); - for r in ratings { - match r.vote { - Vote::Up => up += 1, - Vote::Down => down += 1, + for rating in ratings.iter().take(MAX_RATINGS_IN_REBUILD) { + let mut parts = vec![ + verdict_label(&rating.label).to_string(), + one_line(&rating.title), + if rating.feed_title.trim().is_empty() { + "unknown".to_string() + } else { + one_line(&rating.feed_title) + }, + ]; + if let Some(summary) = rating + .summary + .as_deref() + .map(one_line) + .filter(|s| !s.is_empty()) + { + parts.push(summary); } - let category = if r.llm_category.is_empty() { - r.category.as_str() - } else { - r.llm_category.as_str() - }; - let _ = writeln!( - prompt, - "{} | {} | feed: {} | category: {}", - match r.vote { - Vote::Up => "UP ", - Vote::Down => "DOWN", - }, - r.title.trim(), - if r.feed_title.is_empty() { - "unknown" - } else { - r.feed_title.trim() - }, - if category.is_empty() { - "unknown" - } else { - category.trim() - }, - ); + if let Some(facets) = facets_line(rating.facets.as_ref()) { + parts.push(format!("facets: {facets}")); + } + if let Some(note) = rating + .note + .as_deref() + .map(one_line) + .filter(|s| !s.is_empty()) + { + parts.push(format!("note: {note}")); + } + let _ = writeln!(prompt, "{}", parts.join(" | ")); } - let _ = write!(prompt, "\n({up} up, {down} down)\n"); prompt } -/// `daily-epub profile rebuild` — summarize recent ratings into a new learned -/// adjustments section and store a new profile version (§3.6, §3.9). -pub async fn rebuild(db: &Db, llm: &LlmClient, opml_path: &Path) -> anyhow::Result { - let interests = parse_interests(opml_path)?; - let ratings = recent_ratings(db).await?; +pub async fn rebuild( + db: &Db, + llm: &LlmClient, + opml_path: &Path, + profile_path: &Path, + verdict_limit: usize, +) -> anyhow::Result { + let ratings = db.current_ratings(RATINGS_LOOKBACK_DAYS).await?; let previous = db.kv_get(KV_LEARNED_ADJUSTMENTS).await?.unwrap_or_default(); - let learned = if ratings.is_empty() { - tracing::info!("no ratings in the lookback window; keeping the existing adjustments"); + tracing::info!("no ratings available; keeping the existing adjustments"); previous } else { let prompt = build_rebuild_prompt(&ratings); @@ -437,365 +427,243 @@ pub async fn rebuild(db: &Db, llm: &LlmClient, opml_path: &Path) -> anyhow::Resu .complete_json::(&prompt, 0.4) .await { - Ok(resp) if !resp.learned_adjustments.trim().is_empty() => { - tracing::info!( - ratings = ratings.len(), - chars = resp.learned_adjustments.len(), - "rewrote the learned-adjustments section" - ); - resp.learned_adjustments.trim().to_string() + Ok(response) if !response.learned_adjustments.trim().is_empty() => { + response.learned_adjustments.trim().to_string() } Ok(_) => { tracing::warn!("the model returned empty adjustments; keeping the previous ones"); previous } - Err(e) => { - tracing::warn!(error = %e, "learned-adjustments rewrite failed; keeping the previous ones"); + Err(error) => { + tracing::warn!(%error, "learned-adjustments rewrite failed; keeping the previous ones"); previous } } }; - let next_version = stored_version(db).await?.map_or(1, |(v, _)| v + 1); + db.kv_set(KV_LEARNED_ADJUSTMENTS, &learned).await?; + let next_version = stored_version(db) + .await? + .map_or(1, |(version, _)| version + 1); + let built_at = Timestamp::now(); + store_version(db, next_version, built_at).await?; + + let opml = parse_interests(opml_path)?; + let profile_file = load_profile(profile_path)?; + let interests = union_interests(opml, profile_file.interests.clone()); + let current = db.current_ratings(RATINGS_LOOKBACK_DAYS).await?; let profile = TasteProfile { - text: build(&interests, &learned), + text: build( + &profile_file.body, + &interests, + &learned, + ¤t, + verdict_limit, + ), version: next_version, - built_at: Timestamp::now(), + built_at, }; - store(db, &profile, &learned).await?; + db.kv_set(KV_TASTE_PROFILE, &profile.text).await?; tracing::info!( - version = profile.version, + version = next_version, chars = profile.text.len(), "stored a new taste profile" ); Ok(profile) } -// --------------------------------------------------------------------------- -// Feed priors (§3.9a) -// --------------------------------------------------------------------------- - -/// Recompute per-feed beta-smoothed priors from the ratings table (§3.9). -/// -/// Returns the number of feeds written. `FeedPrior::rate()` does the smoothing; -/// this only maintains the raw counts plus how often the feed has been included. -pub async fn rebuild_feed_priors(db: &Db) -> anyhow::Result { - use std::collections::HashMap; - - use crate::types::{FeedId, FeedPrior}; - - let mut priors: HashMap = HashMap::new(); - for (feed_id, vote) in db.ratings_with_feed().await? { - let entry = priors.entry(feed_id).or_insert(FeedPrior { - feed_id, - ..FeedPrior::default() - }); - match vote { - Vote::Up => entry.upvotes += 1, - Vote::Down => entry.downvotes += 1, - } - } - - let rows = sqlx::query( - "SELECT e.feed_id AS feed_id, COUNT(*) AS included - FROM issue_articles ia - JOIN articles a ON a.id = ia.article_id - JOIN entries e ON e.id = a.best_entry_id - GROUP BY e.feed_id", - ) - .fetch_all(db.pool()) - .await - .context("counting per-feed inclusions")?; - for row in &rows { - let feed_id: FeedId = row.get("feed_id"); - let included: i64 = row.get("included"); - priors - .entry(feed_id) - .or_insert(FeedPrior { - feed_id, - ..FeedPrior::default() - }) - .included = included; - } - - for prior in priors.values() { - db.upsert_feed_prior(prior).await?; - } - tracing::info!(feeds = priors.len(), "rebuilt feed priors"); - Ok(priors.len()) -} - -/// Convenience for callers that only have a [`Date`]: the ratings lookback start. -pub fn ratings_since(today: Date) -> Date { - today - .checked_sub(jiff::Span::new().days(RATINGS_LOOKBACK_DAYS)) - .unwrap_or(today) -} - #[cfg(test)] mod tests { use super::*; - use crate::types::Rating; const OPML_PATH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/data/scour-interests.opml"); + const PROFILE_PATH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/data/profile.md"); - fn interests() -> Vec { - parse_interests(Path::new(OPML_PATH)).expect("the shipped OPML parses") + #[test] + fn profile_interests_are_removed_and_union_case_insensitively() { + let parsed = parse_profile_str( + "# P\n\n## Interests\n- Rust\nBoston Tech\n- rust\n\n## Notes\nKeep this.\n", + ); + assert_eq!(parsed.body, "# P\n\n## Notes\nKeep this.\n"); + assert_eq!(parsed.interests, ["Rust", "Boston Tech", "rust"]); + let union = union_interests(vec!["rust".into(), "E-Ink".into()], parsed.interests); + assert_eq!(union, ["rust", "E-Ink", "Boston Tech"]); } #[test] - fn parses_the_shipped_opml() { - let list = interests(); + fn prompt_sections_are_ordered_and_verdicts_have_required_labels() { + let rating = RatedArticle { + article_id: 1, + issue_date: None, + title: "A title".into(), + feed_title: "A feed".into(), + summary: Some("A summary\nwith whitespace.".into()), + facets: None, + note: None, + value: -1.0, + label: "not_for_me".into(), + event_at: "2026-08-15T12:00:00Z".parse().unwrap(), + }; + let prompt = build( + "# Reader profile\n\nProfile prose.", + &["Rust".into()], + "- Adjust.", + &[rating], + 60, + ); + let framing = prompt.find("editor-in-chief").unwrap(); + let profile = prompt.find("# Reader profile").unwrap(); + let interests = prompt.find("## Standing interests").unwrap(); + let learned = prompt.find("## Learned adjustments").unwrap(); + let verdicts = prompt.find("## Recent verdicts").unwrap(); assert!( - list.len() > 180, - "expected ~220 interests, got {}", - list.len() + framing < profile && profile < interests && interests < learned && learned < verdicts ); - assert!(list.iter().any(|i| i == "Rust")); - assert!(list.iter().any(|i| i == "Boston Tech")); - assert!(list.iter().any(|i| i == "E-Ink Displays")); - // Trailing whitespace is trimmed and case-duplicates collapse. - assert!(list.iter().any(|i| i == "photography")); - let lowered: Vec = list.iter().map(|i| i.to_lowercase()).collect(); - let unique: BTreeSet<&String> = lowered.iter().collect(); - assert_eq!(unique.len(), lowered.len(), "duplicates survived"); - assert!(!list.iter().any(|i| i.contains("scour.ing"))); + assert!(prompt.contains("NOT FOR ME | A title | A feed | A summary with whitespace.")); } #[test] - fn parsing_handles_entities_and_empties() { - let raw = r#" - - - - - "#; - assert_eq!( - parse_interests_str(raw), - vec!["Tea & Coffee".to_string(), "Rust".to_string()] - ); + fn shipped_profile_and_opml_parse() { + let profile = load_profile(Path::new(PROFILE_PATH)).unwrap(); + assert!(profile.body.contains("## Who he is")); + assert!(!profile.body.contains("## Interests")); + assert!(profile.interests.is_empty()); + let interests = parse_interests(Path::new(OPML_PATH)).unwrap(); + assert!(interests.iter().any(|interest| interest == "Rust")); } #[test] - fn profile_document_is_deterministic_and_complete() { - let list = interests(); - let one = build(&list, ""); - let two = build(&list, ""); - assert_eq!(one, two, "profile assembly must be byte-stable"); - assert_eq!(one.as_bytes(), two.as_bytes()); - - assert!(one.contains(STATED_PREFERENCES)); - assert!(one.contains(NO_LEARNED_ADJUSTMENTS)); - assert!(one.contains("Boston")); - assert!(one.contains("Wikipedia-Current-Events")); - assert!(one.contains("Rust")); - assert!(one.contains("ultra-niche") || one.contains("Ultra-niche")); - // A real document, not a stub, but not a novel either. - let words = one.split_whitespace().count(); - assert!((500..3000).contains(&words), "profile is {words} words"); - - let learned = build(&list, "- Rank database internals higher."); - assert!(learned.contains("- Rank database internals higher.")); - assert!(!learned.contains(NO_LEARNED_ADJUSTMENTS)); + fn rebuild_prompt_carries_summary_facets_note_and_diversity_instruction() { + let rating = RatedArticle { + article_id: 1, + issue_date: Some("2026-08-15".parse().unwrap()), + title: "Postgres failover".into(), + feed_title: "Engineering Notes".into(), + summary: Some("A detailed incident report.".into()), + facets: Some(Facets { + format: Some("first_hand_account".into()), + depth: Some("deep".into()), + evidence: Some("first_hand".into()), + technicality: Some("advanced".into()), + topic_group: Some("software_engineering".into()), + ..Facets::default() + }), + note: Some("Great operational detail".into()), + value: 1.0, + label: "loved".into(), + event_at: "2026-08-15T12:00:00Z".parse().unwrap(), + }; + let prompt = build_rebuild_prompt(&[rating]); + assert!(prompt.contains("strong prior, not a rule")); + assert!(prompt.contains("Diversity check:")); + assert!(prompt.contains( + "LOVED | Postgres failover | Engineering Notes | A detailed incident report." + )); + assert!( + prompt.contains( + "facets: first_hand_account/deep/first_hand/advanced/software_engineering" + ) + ); + assert!(prompt.contains("note: Great operational detail")); } - async fn temp_db() -> (tempfile::TempDir, Db) { - let dir = tempfile::tempdir().expect("tempdir"); + #[tokio::test] + async fn per_run_profile_reload_changes_prompt_without_bumping_version() { + let dir = tempfile::tempdir().unwrap(); let db = Db::open_and_migrate(&dir.path().join("profile.db")) .await - .expect("db"); - (dir, db) - } - - #[tokio::test] - async fn load_or_build_persists_and_reuses() { - let (_dir, db) = temp_db().await; - let first = load_or_build(&db, Path::new(OPML_PATH)) - .await - .expect("first build"); - assert_eq!(first.version, 1); - assert!(!is_stale(&db).await.expect("staleness")); - - let second = load_or_build(&db, Path::new(OPML_PATH)) - .await - .expect("second load"); - assert_eq!(first.text, second.text); - assert_eq!(second.version, 1); - assert_eq!( - db.kv_get(KV_TASTE_PROFILE).await.expect("kv").as_deref(), - Some(first.text.as_str()) - ); - } - - #[tokio::test] - async fn missing_version_row_means_stale() { - let (_dir, db) = temp_db().await; - assert!(is_stale(&db).await.expect("staleness")); - db.kv_set(KV_PROFILE_VERSION, "not json") - .await - .expect("kv set"); - assert!(is_stale(&db).await.expect("staleness")); - db.kv_set( - KV_PROFILE_VERSION, - r#"{"version":3,"built_at":"2000-01-01T00:00:00Z"}"#, + .unwrap(); + let opml = dir.path().join("interests.opml"); + let profile_path = dir.path().join("profile.md"); + std::fs::write(&opml, r#""#).unwrap(); + std::fs::write( + &profile_path, + "# Reader profile\n\nOriginal prose.\n\n## Interests\n- Custom Topic\n", ) - .await - .expect("kv set"); - assert!(is_stale(&db).await.expect("staleness")); + .unwrap(); + + let first = load_or_build(&db, &opml, &profile_path, 60).await.unwrap(); + assert_eq!(first.version, 1); + assert!(first.text.contains("Original prose.")); + assert!(first.text.contains("Custom Topic")); + assert!(!first.text.contains("## Interests")); + + std::fs::write( + &profile_path, + "# Reader profile\n\nChanged prose.\n\n## Interests\n- Another Topic\n", + ) + .unwrap(); + let second = load_or_build(&db, &opml, &profile_path, 60).await.unwrap(); + assert_eq!(second.version, first.version); + assert_eq!(second.built_at, first.built_at); + assert!(second.text.contains("Changed prose.")); + assert!(second.text.contains("Another Topic")); + assert!(!second.text.contains("Original prose.")); + + let missing = load_profile(&dir.path().join("missing.md")).unwrap(); + assert!(missing.body.is_empty() && missing.interests.is_empty()); } #[tokio::test] - async fn rebuild_uses_the_model_and_bumps_the_version() { - use super::super::llm::{MockBackend, UsageMeter}; - use crate::config::DeepseekConfig; + async fn weekly_rebuild_uses_mock_backend_and_bumps_profile_version() { use std::sync::Arc; - let (_dir, db) = temp_db().await; - load_or_build(&db, Path::new(OPML_PATH)) + use super::super::llm::{MockBackend, UsageMeter}; + use crate::config::DeepseekConfig; + use crate::types::{RatingEvent, TokenUsage}; + + let dir = tempfile::tempdir().unwrap(); + let db = Db::open_and_migrate(&dir.path().join("profile.db")) .await - .expect("initial"); - seed_rating(&db, 1, "Postgres index internals", Vote::Up).await; - seed_rating(&db, 2, "Series B funding announced", Vote::Down).await; + .unwrap(); + let opml = dir.path().join("interests.opml"); + let profile_path = dir.path().join("profile.md"); + std::fs::write(&opml, r#""#).unwrap(); + std::fs::write(&profile_path, "# Reader profile\n\nLikes depth.\n").unwrap(); + let initial = load_or_build(&db, &opml, &profile_path, 60).await.unwrap(); + assert_eq!(initial.version, 1); + + sqlx::query( + "INSERT INTO articles (id, canonical_url, title, first_seen) VALUES + (1, 'https://example.com/deep', 'A deep report', '2026-08-15T00:00:00Z')", + ) + .execute(db.pool()) + .await + .unwrap(); + db.append_rating_event(&RatingEvent { + id: 0, + article_id: 1, + issue_date: None, + kind: "explicit".into(), + source: "cli".into(), + label: "loved".into(), + value: 1.0, + note: Some("specific evidence".into()), + event_at: Timestamp::now(), + }) + .await + .unwrap(); let backend = Arc::new(MockBackend::new()); backend.push( - r#"{"learned_adjustments": "- Rank database internals deep dives higher.\n- Be sceptical of funding announcements."}"#, - crate::types::TokenUsage::default(), + r#"{"learned_adjustments":"- Rank first-hand reports higher.\n- Diversity check: keep formats balanced."}"#, + TokenUsage::default(), ); let llm = LlmClient::with_backend( "deepseek-v4-flash", - "SYSTEM".into(), + initial.text, UsageMeter::new(&DeepseekConfig::default(), 2.0), backend.clone(), ); - - let rebuilt = rebuild(&db, &llm, Path::new(OPML_PATH)) - .await - .expect("rebuild"); + let rebuilt = rebuild(&db, &llm, &opml, &profile_path, 60).await.unwrap(); assert_eq!(rebuilt.version, 2); + assert!(rebuilt.text.contains("Rank first-hand reports higher.")); assert!( rebuilt .text - .contains("Rank database internals deep dives higher.") + .contains("Diversity check: keep formats balanced.") ); - assert!( - rebuilt - .text - .contains("Be sceptical of funding announcements.") - ); - - // The rating history reached the prompt, with feed + category context. - let prompts = backend.prompts(); - assert_eq!(prompts.len(), 1); - assert!(prompts[0].user.contains("Postgres index internals")); - assert!( - prompts[0] - .user - .contains("DOWN | Series B funding announced") - ); - assert!(prompts[0].user.contains("(1 up, 1 down)")); - - // A later reload sees the new document. - let loaded = load_or_build(&db, Path::new(OPML_PATH)) - .await - .expect("reload"); - assert_eq!(loaded.text, rebuilt.text); - assert_eq!(loaded.version, 2); - } - - #[tokio::test] - async fn rebuild_without_ratings_skips_the_model() { - use super::super::llm::{MockBackend, UsageMeter}; - use crate::config::DeepseekConfig; - use std::sync::Arc; - - let (_dir, db) = temp_db().await; - let backend = Arc::new(MockBackend::new()); - let llm = LlmClient::with_backend( - "deepseek-v4-flash", - "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), 2.0), - backend.clone(), - ); - let profile = rebuild(&db, &llm, Path::new(OPML_PATH)) - .await - .expect("rebuild"); - assert_eq!(backend.calls(), 0, "no ratings ⇒ no LLM call"); - assert!(profile.text.contains(NO_LEARNED_ADJUSTMENTS)); - assert!( - weekly_rebuild_if_due(&db, &llm, Path::new(OPML_PATH)) - .await - .expect("weekly") - .is_none() - ); - } - - #[tokio::test] - async fn feed_priors_are_recomputed_from_ratings() { - let (_dir, db) = temp_db().await; - seed_rating(&db, 1, "Good one", Vote::Up).await; - seed_rating(&db, 2, "Bad one", Vote::Down).await; - let feeds = rebuild_feed_priors(&db).await.expect("priors"); - assert_eq!(feeds, 1); - let priors = db.feed_priors().await.expect("load"); - assert_eq!(priors.len(), 1); - assert_eq!(priors[0].upvotes, 1); - assert_eq!(priors[0].downvotes, 1); - assert!((priors[0].rate() - 0.5).abs() < 1e-12); - } - - /// Insert an entry + article + rating triple that the joins can see. - async fn seed_rating(db: &Db, id: i64, title: &str, vote: Vote) { - use crate::types::{Entry, ExtractMethod, SourceRef}; - let ts: Timestamp = "2026-08-15T05:30:00Z".parse().expect("ts"); - db.upsert_entry(&Entry { - id, - feed_id: 7, - feed_title: Some("A Feed".into()), - category: Some("Tech".into()), - title: title.into(), - url: format!("https://example.com/{id}"), - canonical_url: Some(format!("https://example.com/{id}")), - author: None, - published_at: Some(ts), - comments_url: None, - raw_content: String::new(), - fetched_at: ts, - }) - .await - .expect("entry"); - let article_id = db - .upsert_article(&crate::types::Article { - id: 0, - canonical_url: format!("https://example.com/{id}"), - title: title.into(), - best_entry_id: id, - content_html: String::new(), - word_count: 900, - excerpt_only: false, - image_count: 0, - sources: Vec::::new(), - first_seen: ts, - url: format!("https://example.com/{id}"), - author: None, - feed_id: 7, - feed_title: "A Feed".into(), - category: Some("Tech".into()), - published_at: Some(ts), - comments_url: None, - image_urls: vec![], - social: vec![], - extract_method: ExtractMethod::Miniflux, - }) - .await - .expect("article"); - db.upsert_rating(&Rating { - issue_date: Timestamp::now().to_zoned(jiff::tz::TimeZone::UTC).date(), - article_id, - vote, - rated_at: Timestamp::now(), - }) - .await - .expect("rating"); + assert_eq!(backend.calls(), 1); + assert!(backend.prompts()[0].user.contains("LOVED | A deep report")); } } diff --git a/src/curate/score.rs b/src/curate/score.rs index 49be464..a427f6f 100644 --- a/src/curate/score.rs +++ b/src/curate/score.rs @@ -425,7 +425,6 @@ mod tests { article: article(id, title, words), prefilter_score: 50.0, social_score: 0.0, - feed_prior: 0.5, llm: None, auto_include: false, } diff --git a/src/curate/select.rs b/src/curate/select.rs index 75d3db3..5f7fa06 100644 --- a/src/curate/select.rs +++ b/src/curate/select.rs @@ -144,9 +144,8 @@ fn render_candidate(candidate: &ScoredArticle) -> String { } let _ = writeln!( block, - "signals: social {:.2}; feed prior {:.2}; via {}{}", + "signals: social {:.2}; via {}{}", candidate.social_score, - candidate.feed_prior, source_kinds(candidate), if candidate.auto_include { "; ALWAYS-INCLUDE" @@ -716,7 +715,6 @@ mod tests { article: article(id, title, words), prefilter_score: 40.0 + score, social_score: 1.0, - feed_prior: 0.5, llm: Some(LlmScore { score, category: "Tech & Engineering".into(), diff --git a/src/db.rs b/src/db.rs index f6b342b..70eb550 100644 --- a/src/db.rs +++ b/src/db.rs @@ -2,8 +2,8 @@ //! //! Runtime queries only — no `sqlx::query!` macros (implementation notes §1). //! Timestamps are stored as RFC3339 UTC strings and dates as `YYYY-MM-DD` -//! (implementation notes §2). Every write is an idempotent upsert so that -//! `generate --date X` can be re-run safely (implementation notes §12). +//! (implementation notes §2). Pipeline writes are idempotent upserts so that +//! `generate --date X` can be re-run safely; feedback events are append-only. use std::path::Path; use std::str::FromStr; @@ -15,8 +15,8 @@ use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions, S use sqlx::{Row, SqlitePool}; use crate::types::{ - Article, ArticleId, Entry, EntryId, FeedId, FeedPrior, LlmScore, Pick, Rating, SocialRef, - SocialSource, SourceRef, Vote, + Article, ArticleId, Entry, EntryId, Facets, LlmScore, Pick, RatedArticle, RatingEvent, + SocialRef, SocialSource, SourceRef, }; /// Embedded migrations from `./migrations` (implementation notes §1). @@ -535,113 +535,104 @@ impl Db { } // ----------------------------------------------------------------- - // ratings + feed priors (§3.9) + // append-only rating events (§6.2) // ----------------------------------------------------------------- - /// Idempotent upsert of a reader vote (§3.9). Returns true if it changed anything. - pub async fn upsert_rating(&self, rating: &Rating) -> Result { - let res = sqlx::query( - "INSERT INTO ratings (issue_date, article_id, vote, rated_at) - VALUES (?, ?, ?, ?) - ON CONFLICT(issue_date, article_id) DO UPDATE SET - vote = excluded.vote, - rated_at = excluded.rated_at - WHERE ratings.vote != excluded.vote", + /// Append one feedback event and return its database id. + pub async fn append_rating_event(&self, event: &RatingEvent) -> Result { + let row = sqlx::query( + "INSERT INTO rating_events + (article_id, issue_date, kind, source, label, value, note, event_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?) + RETURNING id", ) - .bind(rating.issue_date.to_string()) - .bind(rating.article_id) - .bind(rating.vote.as_i64()) - .bind(fmt_ts(rating.rated_at)) - .execute(&self.pool) + .bind(event.article_id) + .bind(event.issue_date.map(|date| date.to_string())) + .bind(&event.kind) + .bind(&event.source) + .bind(&event.label) + .bind(event.value) + .bind(event.note.as_deref()) + .bind(fmt_ts(event.event_at)) + .fetch_one(&self.pool) .await?; - Ok(res.rows_affected() > 0) + Ok(row.get("id")) } - /// Every rating joined to the feed that carried the article (§3.9 priors). - pub async fn ratings_with_feed(&self) -> Result> { + /// Latest issue containing an article, used to attach CLI feedback when possible. + pub async fn latest_issue_date_for_article( + &self, + article_id: ArticleId, + ) -> Result> { + let row = sqlx::query( + "SELECT issue_date FROM issue_articles + WHERE article_id = ? ORDER BY issue_date DESC LIMIT 1", + ) + .bind(article_id) + .fetch_optional(&self.pool) + .await?; + row.map(|row| { + parse_date( + "issue_articles.issue_date", + &row.get::("issue_date"), + ) + }) + .transpose() + } + + /// Current explicit verdicts, newest first. A latest `cleared` event removes + /// its article from this learned set (§6.2). + pub async fn current_ratings(&self, lookback_days: i64) -> Result> { + self.latest_explicit_ratings(lookback_days, false).await + } + + /// Current explicit events including `cleared`, for the ratings CLI. + pub async fn current_ratings_including_cleared( + &self, + lookback_days: i64, + ) -> Result> { + self.latest_explicit_ratings(lookback_days, true).await + } + + async fn latest_explicit_ratings( + &self, + lookback_days: i64, + include_cleared: bool, + ) -> Result> { + let since = Timestamp::now() + .checked_sub(jiff::Span::new().hours(lookback_days.max(0).saturating_mul(24))) + .unwrap_or(Timestamp::UNIX_EPOCH); let rows = sqlx::query( - "SELECT e.feed_id AS feed_id, r.vote AS vote - FROM ratings r + "WITH ranked AS ( + SELECT re.*, + ROW_NUMBER() OVER ( + PARTITION BY re.article_id + ORDER BY re.event_at DESC, re.id DESC + ) AS event_rank + FROM rating_events re + WHERE re.kind = 'explicit' AND re.event_at >= ? + ) + SELECT r.article_id, r.issue_date, r.label, r.value, r.note, r.event_at, + COALESCE(a.title, '') AS title, + COALESCE(e.feed_title, '') AS feed_title, + (SELECT ia.summary FROM issue_articles ia + WHERE ia.article_id = r.article_id + ORDER BY ia.issue_date DESC LIMIT 1) AS summary, + aa.facets_json AS facets_json + FROM ranked r JOIN articles a ON a.id = r.article_id - JOIN entries e ON e.id = a.best_entry_id", + LEFT JOIN entries e ON e.id = a.best_entry_id + LEFT JOIN article_assessments aa + ON aa.article_id = r.article_id AND aa.stage = 'deep' + WHERE r.event_rank = 1 AND (? OR r.label != 'cleared') + ORDER BY r.event_at DESC, r.id DESC", ) + .bind(fmt_ts(since)) + .bind(include_cleared) .fetch_all(&self.pool) .await?; - Ok(rows - .iter() - .map(|r| { - let vote = if r.get::("vote") >= 0 { - Vote::Up - } else { - Vote::Down - }; - (r.get::("feed_id"), vote) - }) - .collect()) - } - /// Recent ratings with article titles, for the weekly profile rewrite (§3.6). - pub async fn recent_ratings_detailed(&self, since: Date) -> Result> { - let rows = sqlx::query( - "SELECT r.issue_date AS issue_date, r.article_id AS article_id, r.vote AS vote, - r.rated_at AS rated_at, a.title AS title - FROM ratings r JOIN articles a ON a.id = r.article_id - WHERE r.issue_date >= ? ORDER BY r.rated_at DESC", - ) - .bind(since.to_string()) - .fetch_all(&self.pool) - .await?; - rows.iter() - .map(|r| { - let rating = Rating { - issue_date: parse_date( - "ratings.issue_date", - &r.get::("issue_date"), - )?, - article_id: r.get::("article_id"), - vote: if r.get::("vote") >= 0 { - Vote::Up - } else { - Vote::Down - }, - rated_at: parse_ts("ratings.rated_at", &r.get::("rated_at"))?, - }; - Ok((rating, r.get::("title"))) - }) - .collect() - } - - pub async fn upsert_feed_prior(&self, prior: &FeedPrior) -> Result<()> { - sqlx::query( - "INSERT INTO feed_priors (feed_id, upvotes, downvotes, included) - VALUES (?, ?, ?, ?) - ON CONFLICT(feed_id) DO UPDATE SET - upvotes = excluded.upvotes, - downvotes = excluded.downvotes, - included = excluded.included", - ) - .bind(prior.feed_id) - .bind(prior.upvotes) - .bind(prior.downvotes) - .bind(prior.included) - .execute(&self.pool) - .await?; - Ok(()) - } - - pub async fn feed_priors(&self) -> Result> { - let rows = sqlx::query("SELECT feed_id, upvotes, downvotes, included FROM feed_priors") - .fetch_all(&self.pool) - .await?; - Ok(rows - .iter() - .map(|r| FeedPrior { - feed_id: r.get::("feed_id"), - upvotes: r.get::("upvotes"), - downvotes: r.get::("downvotes"), - included: r.get::("included"), - }) - .collect()) + rows.iter().map(rated_article_from_row).collect() } // ----------------------------------------------------------------- @@ -787,6 +778,34 @@ fn article_from_row(row: &sqlx::sqlite::SqliteRow) -> Result
{ }) } +fn rated_article_from_row(row: &sqlx::sqlite::SqliteRow) -> Result { + let issue_date = row + .get::, _>("issue_date") + .map(|raw| parse_date("rating_events.issue_date", &raw)) + .transpose()?; + let facets = row.get::, _>("facets_json").and_then(|raw| { + match serde_json::from_str::(&raw) { + Ok(facets) => Some(facets), + Err(error) => { + tracing::warn!(%error, "ignoring malformed assessment facets"); + None + } + } + }); + Ok(RatedArticle { + article_id: row.get("article_id"), + issue_date, + title: row.get("title"), + feed_title: row.get("feed_title"), + summary: row.get("summary"), + facets, + note: row.get("note"), + value: row.get("value"), + label: row.get("label"), + event_at: parse_ts("rating_events.event_at", &row.get::("event_at"))?, + }) +} + fn social_from_row(row: &sqlx::sqlite::SqliteRow) -> Result { let raw: String = row.get("source"); let source = SocialSource::parse(&raw).ok_or(DbError::Decode { @@ -991,20 +1010,187 @@ mod tests { } #[tokio::test] - async fn ratings_upsert_is_idempotent() { + async fn latest_explicit_rating_wins_and_clear_removes_it() { let (_dir, db) = temp_db().await; - let rating = Rating { - issue_date: "2026-08-15".parse().unwrap(), - article_id: 1, - vote: Vote::Up, - rated_at: ts("2026-08-15T12:00:00Z"), + db.upsert_entry(&sample_entry(1)).await.unwrap(); + let article = Article { + id: 0, + canonical_url: "https://example.com/1".into(), + title: "Story 1".into(), + best_entry_id: 1, + content_html: "

body

".into(), + word_count: 900, + excerpt_only: false, + image_count: 0, + sources: vec![], + first_seen: ts("2026-08-15T05:30:00Z"), + url: "https://example.com/1".into(), + author: None, + feed_id: 7, + feed_title: "Hacker News".into(), + category: None, + published_at: None, + comments_url: None, + image_urls: vec![], + social: vec![], + extract_method: ExtractMethod::Miniflux, }; - assert!(db.upsert_rating(&rating).await.unwrap()); - assert!(!db.upsert_rating(&rating).await.unwrap()); - let flipped = Rating { - vote: Vote::Down, - ..rating.clone() + let article_id = db.upsert_article(&article).await.unwrap(); + for (date, number, summary) in [ + ("2026-08-14", 1, "Older summary"), + ("2026-08-15", 2, "Newest summary"), + ] { + db.upsert_issue( + date.parse().unwrap(), + number, + ts("2026-08-15T05:30:00Z"), + None, + None, + None, + None, + None, + ) + .await + .unwrap(); + sqlx::query( + "INSERT INTO issue_articles + (issue_date, article_id, section, position, is_lead, summary) + VALUES (?, ?, 'Top Stories', 1, 0, ?)", + ) + .bind(date) + .bind(article_id) + .bind(summary) + .execute(db.pool()) + .await + .unwrap(); + } + sqlx::query( + "INSERT INTO article_assessments + (article_id, stage, model, prompt_version, facets_json, assessed_at) + VALUES (?, 'deep', 'mock', 1, ?, '2026-08-15T11:00:00Z')", + ) + .bind(article_id) + .bind(r#"{"format":"analysis_essay","depth":"deep","evidence":null,"commerciality":null,"topic_group":"software_engineering","technicality":"advanced","locality":null,"specific_topics":null}"#) + .execute(db.pool()) + .await + .unwrap(); + let event = |label: &str, value: f64, at: &str| RatingEvent { + id: 0, + article_id, + issue_date: Some("2026-08-15".parse().unwrap()), + kind: "explicit".into(), + source: "cli".into(), + label: label.into(), + value, + note: None, + event_at: ts(at), }; - assert!(db.upsert_rating(&flipped).await.unwrap()); + db.append_rating_event(&event("loved", 1.0, "2026-08-15T12:00:00Z")) + .await + .unwrap(); + db.append_rating_event(&event("good", 0.35, "2026-08-15T13:00:00Z")) + .await + .unwrap(); + let mut implicit = event("read_fully", 0.5, "2026-08-15T13:30:00Z"); + implicit.kind = "implicit".into(); + implicit.source = "bookorbit".into(); + db.append_rating_event(&implicit).await.unwrap(); + let ratings = db.current_ratings(36500).await.unwrap(); + assert_eq!(ratings.len(), 1); + assert_eq!(ratings[0].label, "good"); + assert_eq!(ratings[0].feed_title, "Hacker News"); + assert_eq!(ratings[0].summary.as_deref(), Some("Newest summary")); + assert_eq!( + ratings[0] + .facets + .as_ref() + .and_then(|facets| facets.format.as_deref()), + Some("analysis_essay") + ); + + db.append_rating_event(&event("cleared", 0.0, "2026-08-15T14:00:00Z")) + .await + .unwrap(); + assert!(db.current_ratings(36500).await.unwrap().is_empty()); + let events = db.current_ratings_including_cleared(36500).await.unwrap(); + assert_eq!(events.len(), 1); + assert_eq!(events[0].label, "cleared"); + let sources: Vec = + sqlx::query_scalar("SELECT source FROM rating_events ORDER BY id") + .fetch_all(db.pool()) + .await + .unwrap(); + assert_eq!(sources, ["cli", "cli", "bookorbit", "cli"]); + } + + #[tokio::test] + async fn curation_v2_migration_copies_ratings_and_drops_old_tables() { + let pool = SqlitePoolOptions::new() + .max_connections(1) + .connect("sqlite::memory:") + .await + .unwrap(); + sqlx::raw_sql(include_str!("../migrations/0001_init.sql")) + .execute(&pool) + .await + .unwrap(); + sqlx::query( + "INSERT INTO articles (id, canonical_url, title, first_seen) VALUES + (1, 'https://example.com/loved', 'Loved', '2026-08-15T00:00:00Z'), + (2, 'https://example.com/down', 'Down', '2026-08-15T00:00:00Z')", + ) + .execute(&pool) + .await + .unwrap(); + sqlx::query( + "INSERT INTO ratings (issue_date, article_id, vote, rated_at) VALUES + ('2026-08-15', 1, 1, '2026-08-15T12:00:00Z'), + ('2026-08-15', 2, -1, '2026-08-15T13:00:00Z')", + ) + .execute(&pool) + .await + .unwrap(); + + sqlx::raw_sql(include_str!("../migrations/0002_curation_v2.sql")) + .execute(&pool) + .await + .unwrap(); + + let rows = sqlx::query( + "SELECT article_id, issue_date, kind, source, label, value, event_at + FROM rating_events ORDER BY article_id", + ) + .fetch_all(&pool) + .await + .unwrap(); + assert_eq!(rows.len(), 2); + assert_eq!(rows[0].get::("label"), "loved"); + assert_eq!(rows[0].get::("value"), 1.0); + assert_eq!(rows[1].get::("label"), "not_for_me"); + assert_eq!(rows[1].get::("value"), -1.0); + for row in &rows { + assert_eq!(row.get::("kind"), "explicit"); + assert_eq!(row.get::("source"), "migration"); + assert_eq!(row.get::("issue_date"), "2026-08-15"); + } + assert_eq!(rows[0].get::("event_at"), "2026-08-15T12:00:00Z"); + + let tables: Vec = + sqlx::query_scalar("SELECT name FROM sqlite_master WHERE type = 'table' ORDER BY name") + .fetch_all(&pool) + .await + .unwrap(); + assert!(!tables.iter().any(|table| table == "ratings")); + assert!(!tables.iter().any(|table| table == "feed_priors")); + assert!(tables.iter().any(|table| table == "scores")); + for expected in [ + "rating_events", + "article_embeddings", + "interest_embeddings", + "article_assessments", + "candidate_runs", + ] { + assert!(tables.iter().any(|table| table == expected), "{expected}"); + } } } diff --git a/src/epub/chapters.rs b/src/epub/chapters.rs index db0ad3e..1328ac8 100644 --- a/src/epub/chapters.rs +++ b/src/epub/chapters.rs @@ -63,8 +63,9 @@ struct SectionPage { } struct RatingLinks { - up_url: String, - down_url: String, + loved_url: String, + good_url: String, + not_for_me_url: String, } #[derive(Template)] @@ -344,8 +345,15 @@ pub fn render_article( // The X4 has no browser, so rating links are pointless there (§7). let rating = match (hmac_secret, edition) { (Some(secret), Edition::Standard) if !secret.is_empty() => Some(RatingLinks { - up_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Up), - down_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Down), + loved_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Loved), + good_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Good), + not_for_me_url: rating_url( + public_url, + secret, + issue.meta.date, + article.id, + Vote::NotForMe, + ), }), _ => None, }; @@ -466,16 +474,19 @@ mod tests { #[test] fn rating_token_matches_the_spec_vector() { let date: Date = "2026-08-15".parse().unwrap(); - assert_eq!(rating_message(date, 1234, Vote::Up), "2026-08-15/1234/up"); - // hex(hmac_sha256("test-secret", "2026-08-15/1234/up"))[..16] - let token = rating_token("test-secret", date, 1234, Vote::Up); + assert_eq!( + rating_message(date, 1234, Vote::Loved), + "2026-08-15/1234/loved" + ); + // hex(hmac_sha256("test-secret", "2026-08-15/1234/loved"))[..16] + let token = rating_token("test-secret", date, 1234, Vote::Loved); assert_eq!(token.len(), TOKEN_LEN); assert!(token.chars().all(|c| c.is_ascii_hexdigit())); // Independently computed reference value. use hmac::Mac; let mut mac = Hmac::::new_from_slice(b"test-secret").unwrap(); - mac.update(b"2026-08-15/1234/up"); + mac.update(b"2026-08-15/1234/loved"); let expected: String = hex::encode(mac.finalize().into_bytes()) .chars() .take(16) @@ -483,9 +494,12 @@ mod tests { assert_eq!(token, expected); // Different vote, article and secret all change the token. - assert_ne!(token, rating_token("test-secret", date, 1234, Vote::Down)); - assert_ne!(token, rating_token("test-secret", date, 1235, Vote::Up)); - assert_ne!(token, rating_token("other-secret", date, 1234, Vote::Up)); + assert_ne!( + token, + rating_token("test-secret", date, 1234, Vote::NotForMe) + ); + assert_ne!(token, rating_token("test-secret", date, 1235, Vote::Loved)); + assert_ne!(token, rating_token("other-secret", date, 1234, Vote::Loved)); } /// The EPUB signs the links and `server.rs` verifies them: one formula, or no @@ -494,17 +508,28 @@ mod tests { #[test] fn epub_and_server_share_one_token_vector() { let date: Date = "2026-08-15".parse().unwrap(); - assert_eq!( - rating_token("test-secret", date, 42, Vote::Up), - "3b314cf7e6d8f50f" - ); + let token = rating_token("test-secret", date, 42, Vote::Loved); + assert_eq!(token, "cece96767d6c5f8a"); + assert!(crate::server::verify_token( + "test-secret", + date, + 42, + Vote::Loved, + &token + )); } #[test] fn rating_url_has_the_spec_shape() { let date: Date = "2026-08-15".parse().unwrap(); - let url = rating_url("https://daily.hallada.net/", "s3cret", date, 99, Vote::Down); - let token = rating_token("s3cret", date, 99, Vote::Down); + let url = rating_url( + "https://daily.hallada.net/", + "s3cret", + date, + 99, + Vote::NotForMe, + ); + let token = rating_token("s3cret", date, 99, Vote::NotForMe); assert_eq!( url, format!("https://daily.hallada.net/r/2026-08-15/99/down?t={token}") @@ -587,7 +612,8 @@ mod tests { assert!(chapter.xhtml.contains("Example Feed")); assert!(chapter.xhtml.contains("6 min read")); assert!(chapter.xhtml.contains("\u{25b2} 342 on HN")); - assert!(chapter.xhtml.contains("/r/2026-08-15/1/up?t=")); + assert!(chapter.xhtml.contains("/r/2026-08-15/1/loved?t=")); + assert!(chapter.xhtml.contains("/r/2026-08-15/1/good?t=")); assert!(chapter.xhtml.contains("/r/2026-08-15/1/down?t=")); assert!(chapter.xhtml.contains("Read online")); assert!(chapter.xhtml.contains("href=\"disc-1001.xhtml\"")); diff --git a/src/epub/templates/chapter.xhtml b/src/epub/templates/chapter.xhtml index 18d4b04..29632ca 100644 --- a/src/epub/templates/chapter.xhtml +++ b/src/epub/templates/chapter.xhtml @@ -24,9 +24,10 @@
{% if let Some(links) = rating %} -

Was this a good pick? [ 👍 Yes ] · [ 👎 No ]

-{% endif %} +

Was this a good pick?   [ Loved it ]   [ Good ]   [ Not for me ]     Read online ↗

+{% else %}

Read online ↗

+{% endif %} {% if let Some(href) = discussion_href %}

💬 Read the discussion

{% endif %} diff --git a/src/epub/templates/style.css b/src/epub/templates/style.css index 44d7789..4aef901 100644 --- a/src/epub/templates/style.css +++ b/src/epub/templates/style.css @@ -224,6 +224,10 @@ hr.rule { font-size: 0.9em; } +.rating { + font-size: 0.8em; +} + .rating a { text-decoration: none; } diff --git a/src/main.rs b/src/main.rs index b806347..cd9cdcf 100644 --- a/src/main.rs +++ b/src/main.rs @@ -6,13 +6,14 @@ use std::path::PathBuf; use anyhow::{Context, Result}; -use clap::{Parser, Subcommand}; +use clap::{Parser, Subcommand, ValueEnum}; use tracing_subscriber::EnvFilter; use daily_epub::config::Config; use daily_epub::db::Db; use daily_epub::pipeline::{self, GenerateOptions, GenerateOutcome}; use daily_epub::report::RunReport; +use daily_epub::types::{ArticleId, RatingEvent, Vote}; use daily_epub::{curate, http, server, social}; /// A personalized daily newspaper, delivered as an EPUB. @@ -36,6 +37,9 @@ enum Command { /// Taste-profile maintenance. #[command(subcommand)] Profile(ProfileCommand), + /// Inspect and edit explicit article verdicts. + #[command(subcommand)] + Ratings(RatingsCommand), /// Re-poll social scores for recent entries. BackfillSocial(BackfillSocialArgs), /// Database maintenance. @@ -80,6 +84,80 @@ enum ProfileCommand { Rebuild, } +#[derive(Debug, Subcommand)] +enum RatingsCommand { + /// List current explicit ratings, newest first. + List(RatingsListArgs), + /// Set or correct an article's explicit rating. + Set(RatingsSetArgs), + /// Clear an article from the learned rating set. + Clear(RatingsClearArgs), +} + +#[derive(Debug, Clone, Copy, ValueEnum)] +enum RatingListLabel { + Loved, + Good, + Down, + Cleared, +} + +impl RatingListLabel { + fn event_label(self) -> &'static str { + match self { + Self::Loved => "loved", + Self::Good => "good", + Self::Down => "not_for_me", + Self::Cleared => "cleared", + } + } +} + +#[derive(Debug, Clone, Copy, ValueEnum)] +enum RatingSetLabel { + Loved, + Good, + Down, +} + +impl RatingSetLabel { + fn vote(self) -> Vote { + match self { + Self::Loved => Vote::Loved, + Self::Good => Vote::Good, + Self::Down => Vote::NotForMe, + } + } +} + +#[derive(Debug, clap::Args)] +struct RatingsListArgs { + #[arg(long, default_value_t = 90)] + days: i64, + #[arg(long, value_enum)] + label: Option, +} + +#[derive(Debug, clap::Args)] +struct RatingsSetArgs { + #[arg(long, required_unless_present = "url", conflicts_with = "url")] + article: Option, + #[arg(long, required_unless_present = "article", conflicts_with = "article")] + url: Option, + #[arg(long, value_enum)] + label: RatingSetLabel, + #[arg(long)] + note: Option, +} + +#[derive(Debug, clap::Args)] +struct RatingsClearArgs { + #[arg(long, required_unless_present = "url", conflicts_with = "url")] + article: Option, + #[arg(long, required_unless_present = "article", conflicts_with = "article")] + url: Option, +} + #[derive(Debug, clap::Args)] struct BackfillSocialArgs { /// How many days back to re-poll. @@ -114,6 +192,10 @@ async fn main() -> Result<()> { let db = Db::open_and_migrate(&config.database_path).await?; cmd_profile_rebuild(&config, &db).await?; } + Command::Ratings(command) => { + let db = Db::open_and_migrate(&config.database_path).await?; + cmd_ratings(&config, &db, command).await?; + } Command::BackfillSocial(args) => { let db = Db::open_and_migrate(&config.database_path).await?; cmd_backfill_social(&db, args.days).await?; @@ -236,18 +318,130 @@ fn print_lineup(issue: &daily_epub::types::Issue) { async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> { let meter = curate::llm::UsageMeter::new(&config.deepseek, config.max_daily_usd); - let profile = curate::profile::load_or_build(db, &config.interests_opml).await?; + let profile = curate::profile::load_or_build( + db, + &config.interests_opml, + &config.profile_path, + config.curation.feedback.verdicts_in_prompt, + ) + .await?; let llm = curate::llm::LlmClient::new(&config.deepseek, profile.text, meter)?; - let rebuilt = curate::profile::rebuild(db, &llm, &config.interests_opml).await?; - let feeds = curate::profile::rebuild_feed_priors(db).await?; + let rebuilt = curate::profile::rebuild( + db, + &llm, + &config.interests_opml, + &config.profile_path, + config.curation.feedback.verdicts_in_prompt, + ) + .await?; println!( - "taste profile rebuilt (version {}, {} chars); {feeds} feed priors refreshed", + "taste profile rebuilt (version {}, {} chars)", rebuilt.version, rebuilt.text.len() ); Ok(()) } +async fn resolve_rating_article( + db: &Db, + article: Option, + url: Option<&str>, +) -> Result { + let article_id = match (article, url) { + (Some(article_id), None) => article_id, + (None, Some(url)) => { + let canonical = daily_epub::dedupe::canonical_url(url) + .with_context(|| format!("invalid article URL {url:?}"))?; + db.article_id_for_url(&canonical) + .await? + .with_context(|| format!("no article found for {canonical}"))? + } + _ => anyhow::bail!("provide exactly one of --article or --url"), + }; + if db.get_article(article_id).await?.is_none() { + anyhow::bail!("article {article_id} was not found"); + } + Ok(article_id) +} + +async fn append_cli_event( + config: &Config, + db: &Db, + article_id: ArticleId, + vote: Option, + note: Option, +) -> Result { + let (label, value) = match vote { + Some(Vote::Loved) => ("loved", Vote::Loved.value(&config.curation.feedback)), + Some(Vote::Good) => ("good", Vote::Good.value(&config.curation.feedback)), + Some(Vote::NotForMe) => ( + "not_for_me", + Vote::NotForMe.value(&config.curation.feedback), + ), + None => ("cleared", 0.0), + }; + let event = RatingEvent { + id: 0, + article_id, + issue_date: db.latest_issue_date_for_article(article_id).await?, + kind: "explicit".into(), + source: "cli".into(), + label: label.into(), + value, + note, + event_at: jiff::Timestamp::now(), + }; + Ok(db.append_rating_event(&event).await?) +} + +async fn cmd_ratings(config: &Config, db: &Db, command: RatingsCommand) -> Result<()> { + match command { + RatingsCommand::List(args) => { + let ratings = db.current_ratings_including_cleared(args.days).await?; + let mut shown = 0usize; + for rating in ratings { + if args + .label + .is_some_and(|label| rating.label != label.event_label()) + { + continue; + } + let note = rating + .note + .as_deref() + .map(|note| format!(" · note: {note}")) + .unwrap_or_default(); + println!( + "{} · article {} · {} · {} — {}{}", + rating.event_at, + rating.article_id, + rating.label, + rating.title, + rating.feed_title, + note + ); + shown += 1; + } + println!("{shown} current rating(s)"); + } + RatingsCommand::Set(args) => { + let article_id = resolve_rating_article(db, args.article, args.url.as_deref()).await?; + let vote = args.label.vote(); + let event_id = append_cli_event(config, db, article_id, Some(vote), args.note).await?; + println!( + "recorded {} for article {article_id} (event {event_id})", + vote.as_str() + ); + } + RatingsCommand::Clear(args) => { + let article_id = resolve_rating_article(db, args.article, args.url.as_deref()).await?; + let event_id = append_cli_event(config, db, article_id, None, None).await?; + println!("cleared article {article_id} (event {event_id})"); + } + } + Ok(()) +} + async fn cmd_backfill_social(db: &Db, days: u32) -> Result<()> { let http = http::build_client(http::DEFAULT_TIMEOUT)?; let enricher = social::SocialEnricher::new(http, db.clone()); @@ -309,6 +503,33 @@ mod tests { .command, Command::Profile(ProfileCommand::Rebuild) )); + assert!(matches!( + Cli::try_parse_from([ + "daily-epub", + "ratings", + "set", + "--article", + "42", + "--label", + "good" + ]) + .unwrap() + .command, + Command::Ratings(RatingsCommand::Set(_)) + )); + assert!(matches!( + Cli::try_parse_from([ + "daily-epub", + "ratings", + "clear", + "--url", + "https://example.com" + ]) + .unwrap() + .command, + Command::Ratings(RatingsCommand::Clear(_)) + )); + assert!(matches!( Cli::try_parse_from(["daily-epub", "backfill-social", "--days", "14"]) .unwrap() @@ -325,4 +546,68 @@ mod tests { let cli = Cli::try_parse_from(["daily-epub", "--config", "/tmp/x.toml", "serve"]).unwrap(); assert_eq!(cli.config, Some(PathBuf::from("/tmp/x.toml"))); } + + #[tokio::test] + async fn cli_set_and_clear_append_cli_events_with_latest_issue_date() { + use sqlx::Row as _; + + let dir = tempfile::tempdir().unwrap(); + let db = Db::open_and_migrate(&dir.path().join("ratings.db")) + .await + .unwrap(); + sqlx::query( + "INSERT INTO articles (id, canonical_url, title, first_seen) VALUES + (42, 'https://example.com/article', 'Article', '2026-08-15T00:00:00Z')", + ) + .execute(db.pool()) + .await + .unwrap(); + sqlx::query( + "INSERT INTO issues (date, issue_number, generated_at) VALUES + ('2026-08-14', 1, '2026-08-14T12:00:00Z'), + ('2026-08-15', 2, '2026-08-15T12:00:00Z')", + ) + .execute(db.pool()) + .await + .unwrap(); + sqlx::query( + "INSERT INTO issue_articles (issue_date, article_id, section) VALUES + ('2026-08-14', 42, 'Top Stories'), + ('2026-08-15', 42, 'Top Stories')", + ) + .execute(db.pool()) + .await + .unwrap(); + + let config = Config::default(); + append_cli_event( + &config, + &db, + 42, + Some(Vote::Good), + Some("useful note".into()), + ) + .await + .unwrap(); + append_cli_event(&config, &db, 42, None, None) + .await + .unwrap(); + + let rows = sqlx::query( + "SELECT issue_date, source, label, value, note FROM rating_events ORDER BY id", + ) + .fetch_all(db.pool()) + .await + .unwrap(); + assert_eq!(rows.len(), 2); + assert_eq!(rows[0].get::("source"), "cli"); + assert_eq!(rows[0].get::("issue_date"), "2026-08-15"); + assert_eq!(rows[0].get::("label"), "good"); + assert_eq!(rows[0].get::("value"), 0.35); + assert_eq!(rows[0].get::("note"), "useful note"); + assert_eq!(rows[1].get::("source"), "cli"); + assert_eq!(rows[1].get::("label"), "cleared"); + assert_eq!(rows[1].get::("value"), 0.0); + assert!(db.current_ratings(36500).await.unwrap().is_empty()); + } } diff --git a/src/pipeline.rs b/src/pipeline.rs index 06b4d36..5155cc6 100644 --- a/src/pipeline.rs +++ b/src/pipeline.rs @@ -367,12 +367,8 @@ async fn run_stages( report.counts.social_hits = enricher.enrich_all(&mut articles).await as i64; report.timings.record("social", elapsed_ms(stage)); - // --- Stage 6: feed priors, then the heuristic pre-filter (§3.5, §3.9) --- + // --- Stage 6: heuristic pre-filter (§3.5) --- let stage = Timestamp::now(); - if let Err(e) = profile::rebuild_feed_priors(db).await { - report.warn(format!("could not rebuild feed priors: {e:#}")); - } - let meter = UsageMeter::new(&config.deepseek, config.max_daily_usd); // `max_daily_usd` is a ceiling for the *day*, not for one invocation, so a // re-run inherits what earlier runs for this date already spent (§3.6). @@ -591,11 +587,14 @@ async fn build_llm( meter: &UsageMeter, report: &mut RunReport, ) -> Option { - if ctx.skip_llm { - tracing::info!("--skip-llm: no DeepSeek call will be made"); - return None; - } - let profile = match profile::load_or_build(ctx.db, &ctx.config.interests_opml).await { + let profile = match profile::load_or_build( + ctx.db, + &ctx.config.interests_opml, + &ctx.config.profile_path, + ctx.config.curation.feedback.verdicts_in_prompt, + ) + .await + { Ok(profile) => profile, Err(e) => { report.warn(format!( @@ -604,6 +603,10 @@ async fn build_llm( return None; } }; + if ctx.skip_llm { + tracing::info!("--skip-llm: profile rebuilt; no DeepSeek call will be made"); + return None; + } let client = match LlmClient::new(&ctx.config.deepseek, profile.text, meter.clone()) { Ok(client) => client, Err(e) => { @@ -616,7 +619,15 @@ async fn build_llm( // Weekly rewrite of the "learned adjustments" section (§3.6c). It changes the // system prompt, so the client is rebuilt around the new profile. - match profile::weekly_rebuild_if_due(ctx.db, &client, &ctx.config.interests_opml).await { + match profile::weekly_rebuild_if_due( + ctx.db, + &client, + &ctx.config.interests_opml, + &ctx.config.profile_path, + ctx.config.curation.feedback.verdicts_in_prompt, + ) + .await + { Ok(Some(rebuilt)) => { tracing::info!(version = rebuilt.version, "taste profile rebuilt"); match LlmClient::new(&ctx.config.deepseek, rebuilt.text, meter.clone()) { diff --git a/src/server.rs b/src/server.rs index a2707f0..69f51c3 100644 --- a/src/server.rs +++ b/src/server.rs @@ -6,7 +6,7 @@ //! Routes (§3.12): //! | route | behaviour | //! |---|---| -//! | `GET /r/{date}/{article_id}/{vote}?t=` | verify HMAC, upsert rating, rebuild feed priors | +//! | `GET /r/{date}/{article_id}/{vote}?t=` | verify HMAC and append a rating event | //! | `GET /opds/daily.xml` (also `/opds`, `/opds/`) | OPDS 1.2 acquisition feed over `publish.epub_dir` | //! | `GET /files/epub/{name}` | EPUB download — what the feed's acquisition links point at | //! | `GET /files/xtc/{name}` | XTC artifact download, unlisted (no path traversal) | @@ -15,7 +15,7 @@ //! //! `/opds/*` and `/files/*` sit behind optional Basic auth (`server.basic_auth_*`). //! -//! The EPUB article footer (§3.10) mints its 👍/👎 links with the very same +//! The EPUB article footer (§3.10) mints its three verdict links with the very same //! [`rating_url`] this module verifies with — both re-export [`crate::auth`], //! which pins the shared test vector (`secret = "test-secret"`, `2026-08-15`, //! article `42`, `up` → `3b314cf7e6d8f50f`). An issue generated while @@ -38,7 +38,7 @@ use tower_http::trace::TraceLayer; use crate::config::Config; use crate::db::Db; -use crate::types::{ArticleId, Rating, Vote}; +use crate::types::{ArticleId, RatingEvent, Vote}; /// Characters of the hex HMAC kept in rating links (§3.9). pub const TOKEN_LEN: usize = crate::auth::TOKEN_LEN; @@ -244,46 +244,47 @@ async fn handle_rating( } }; - let rating = Rating { - issue_date: date, + let label = match vote { + Vote::Loved => "loved", + Vote::Good => "good", + Vote::NotForMe => "not_for_me", + }; + let event = RatingEvent { + id: 0, + issue_date: Some(date), article_id, - vote, - rated_at: Timestamp::now(), + kind: "explicit".into(), + source: "epub".into(), + label: label.into(), + value: vote.value(&state.config.curation.feedback), + note: None, + event_at: Timestamp::now(), }; - let changed = match state.db.upsert_rating(&rating).await { - Ok(changed) => changed, - Err(e) => { - tracing::error!(error = %e, article_id, "recording the rating failed"); - return page(StatusCode::INTERNAL_SERVER_ERROR, "Database error.", None); - } - }; - if changed && let Err(e) = crate::curate::profile::rebuild_feed_priors(&state.db).await { - // The vote is stored; a stale prior only affects the next run's ranking. - tracing::error!(error = %e, "refreshing feed priors failed"); + if let Err(error) = state.db.append_rating_event(&event).await { + tracing::error!(%error, article_id, "recording the rating failed"); + return page(StatusCode::INTERNAL_SERVER_ERROR, "Database error.", None); } tracing::info!( %date, article_id, feed_id = article.feed_id, vote = vote.as_str(), - changed, title = %article.title, - "recorded rating" + "recorded rating event" ); - let glyph = match vote { - Vote::Up => "👍", - Vote::Down => "👎", + let message = match vote { + Vote::Loved => "Recorded: Loved it — thanks.", + Vote::Good => "Recorded: Good — thanks.", + Vote::NotForMe => "Recorded: Not for me — thanks.", }; - let message = if changed { - format!("Recorded {glyph} — thanks!") - } else { - format!("Already recorded {glyph} — thanks!") - }; - page( + confirmation_page( StatusCode::OK, - &message, - Some(&format!("{date} · article {article_id}")), + message, + &state.config, + date, + article_id, + vote, ) } @@ -457,7 +458,7 @@ fn check_basic_auth(config: &Config, headers: &HeaderMap) -> Option { /// A self-contained response page — no external CSS, well under 1 KB, legible on /// a 6" e-ink browser (§3.9). fn page(status: StatusCode, message: &str, note: Option<&str>) -> Response { - let body = page_html(message, note); + let body = page_html(message, note, None); ( status, [ @@ -469,8 +470,52 @@ fn page(status: StatusCode, message: &str, note: Option<&str>) -> Response { .into_response() } -/// The page markup itself: no stylesheet, no script, no images (§3.9). -fn page_html(message: &str, note: Option<&str>) -> String { +fn confirmation_page( + status: StatusCode, + message: &str, + config: &Config, + date: Date, + article_id: ArticleId, + selected: Vote, +) -> Response { + let Some(secret) = config.server.hmac_secret.as_deref() else { + return page(status, message, None); + }; + let choices = [ + (Vote::Loved, "Loved it"), + (Vote::Good, "Good"), + (Vote::NotForMe, "Not for me"), + ] + .into_iter() + .filter(|(vote, _)| *vote != selected) + .map(|(vote, label)| { + let url = rating_url(&config.server.public_url, secret, date, article_id, vote); + format!( + "[ {} ]", + escape_attr(&url), + escape(label) + ) + }) + .collect::>() + .join("   "); + let body = page_html( + message, + None, + Some(&format!("

Change it: {choices}

")), + ); + ( + status, + [ + (header::CONTENT_TYPE, "text/html; charset=utf-8"), + (header::CACHE_CONTROL, "no-store"), + ], + body, + ) + .into_response() +} + +/// The page markup itself: no external stylesheet, script, or images (§6.1). +fn page_html(message: &str, note: Option<&str>, extra_html: Option<&str>) -> String { let note = note .map(|n| format!("

{}

", escape(n))) .unwrap_or_default(); @@ -478,11 +523,12 @@ fn page_html(message: &str, note: Option<&str>) -> String { "\ \ The Daily EPUB\ -\ -

{}

{}", +

{}

{}{}", escape(message), - note + note, + extra_html.unwrap_or_default() ) } @@ -492,6 +538,10 @@ fn escape(s: &str) -> String { .replace('>', ">") } +fn escape_attr(s: &str) -> String { + escape(s).replace('\"', """) +} + #[cfg(test)] mod tests { use super::*; @@ -512,29 +562,33 @@ mod tests { #[test] fn token_matches_the_shared_test_vector() { - assert_eq!( - rating_token(VECTOR_SECRET, date(), 42, Vote::Up), - VECTOR_TOKEN_UP - ); - assert_eq!(rating_token(VECTOR_SECRET, date(), 42, Vote::Up).len(), 16); + let loved = rating_token(VECTOR_SECRET, date(), 42, Vote::Loved); + assert_eq!(loved, "cece96767d6c5f8a"); + assert_eq!(loved.len(), 16); // Down differs from up, and both verify. - let down = rating_token(VECTOR_SECRET, date(), 42, Vote::Down); + let down = rating_token(VECTOR_SECRET, date(), 42, Vote::NotForMe); assert_ne!(down, VECTOR_TOKEN_UP); assert!(verify_token( VECTOR_SECRET, date(), 42, - Vote::Up, + Vote::Loved, VECTOR_TOKEN_UP )); - assert!(verify_token(VECTOR_SECRET, date(), 42, Vote::Down, &down)); + assert!(verify_token( + VECTOR_SECRET, + date(), + 42, + Vote::NotForMe, + &down + )); } /// The links the EPUB footer embeds must verify here — this is the whole /// feedback loop in one assertion (§3.9). #[test] fn epub_footer_links_verify_against_this_server() { - for (id, vote) in [(42, Vote::Up), (1234, Vote::Down)] { + for (id, vote) in [(42, Vote::Loved), (1234, Vote::NotForMe)] { let from_epub = crate::epub::build::rating_url( "https://daily.hallada.net", VECTOR_SECRET, @@ -556,23 +610,23 @@ mod tests { #[test] fn token_verification_rejects_tampering() { - let t = rating_token(VECTOR_SECRET, date(), 42, Vote::Up); - assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::Down, &t)); - assert!(!verify_token(VECTOR_SECRET, date(), 43, Vote::Up, &t)); - assert!(!verify_token("other-secret", date(), 42, Vote::Up, &t)); + let t = rating_token(VECTOR_SECRET, date(), 42, Vote::Loved); + assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::NotForMe, &t)); + assert!(!verify_token(VECTOR_SECRET, date(), 43, Vote::Loved, &t)); + assert!(!verify_token("other-secret", date(), 42, Vote::Loved, &t)); assert!(!verify_token( VECTOR_SECRET, "2026-08-16".parse().unwrap(), 42, - Vote::Up, + Vote::Loved, &t )); - assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::Up, "")); + assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::Loved, "")); assert!(!verify_token( VECTOR_SECRET, date(), 42, - Vote::Up, + Vote::Loved, &format!("{t}00") )); } @@ -585,9 +639,12 @@ mod tests { VECTOR_SECRET, date(), 42, - Vote::Up + Vote::Loved ), - format!("https://daily.hallada.net/r/2026-08-15/42/up?t={VECTOR_TOKEN_UP}") + format!( + "https://daily.hallada.net/r/2026-08-15/42/loved?t={}", + rating_token(VECTOR_SECRET, date(), 42, Vote::Loved) + ) ); } @@ -626,16 +683,20 @@ mod tests { #[test] fn the_confirmation_page_is_tiny_and_self_contained() { - let html = page_html("Recorded 👍 — thanks!", Some("2026-08-15 · article 42")); + let html = page_html( + "Recorded: Loved it — thanks.", + Some("2026-08-15 · article 42"), + None, + ); assert!(html.len() < 1024, "page is {} bytes", html.len()); assert!(!html.contains("x", None).contains("<b>")); + assert!(page_html("x", None, None).contains("<b>")); } // ----------------------------------------------------------------- @@ -810,47 +871,43 @@ mod tests { } #[tokio::test] - async fn rating_happy_path_is_idempotent_and_updates_priors() { + async fn rating_taps_append_events_and_latest_correction_wins() { let server = TestServer::start(false).await; let id = server.seed_article().await; - let url = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::Up); + let loved = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::Loved); - let res = client().get(&url).send().await.unwrap(); - assert_eq!(res.status(), 200); - let body = res.text().await.unwrap(); - assert!(body.contains("Recorded"), "{body}"); - assert!(!body.contains("Already"), "{body}"); + let response = client().get(&loved).send().await.unwrap(); + assert_eq!(response.status(), 200); + let body = response.text().await.unwrap(); + assert!(body.contains("Recorded: Loved it — thanks."), "{body}"); + assert!(body.contains("[ Good ]"), "{body}"); + assert!(body.contains("[ Not for me ]"), "{body}"); assert!( - body.len() < 1024, + body.len() < 2048, "confirmation page is {} bytes", body.len() ); - // Same tap again: still 200, but reported as already recorded. - let body = client() - .get(&url) - .send() - .await - .unwrap() - .text() + // Every tap is history, even a repeated one. + assert_eq!(client().get(&loved).send().await.unwrap().status(), 200); + let count: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM rating_events") + .fetch_one(server.db.pool()) .await .unwrap(); - assert!(body.contains("Already recorded"), "{body}"); + assert_eq!(count, 2); - let ratings = server.db.ratings_with_feed().await.unwrap(); - assert_eq!(ratings, vec![(7, Vote::Up)]); - let priors = server.db.feed_priors().await.unwrap(); - assert_eq!(priors.len(), 1); - assert_eq!( - (priors[0].feed_id, priors[0].upvotes, priors[0].downvotes), - (7, 1, 0) - ); - - // Flipping the vote rewrites the prior rather than double-counting. - let down = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::Down); + // A correction appends and becomes the current verdict. + let down = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::NotForMe); assert_eq!(client().get(&down).send().await.unwrap().status(), 200); - let priors = server.db.feed_priors().await.unwrap(); - assert_eq!((priors[0].upvotes, priors[0].downvotes), (0, 1)); + let current = server.db.current_ratings(36500).await.unwrap(); + assert_eq!(current.len(), 1); + assert_eq!(current[0].label, "not_for_me"); + let sources: Vec = + sqlx::query_scalar("SELECT source FROM rating_events ORDER BY id") + .fetch_all(server.db.pool()) + .await + .unwrap(); + assert_eq!(sources, ["epub", "epub", "epub"]); } #[tokio::test] @@ -864,17 +921,21 @@ mod tests { assert_eq!(client().get(&missing).send().await.unwrap().status(), 403); // A valid token for an article that does not exist. - let unknown = rating_url(&server.base, VECTOR_SECRET, date(), 9999, Vote::Up); + let unknown = rating_url(&server.base, VECTOR_SECRET, date(), 9999, Vote::Loved); assert_eq!(client().get(&unknown).send().await.unwrap().status(), 404); // Malformed date / vote. - let token = rating_token(VECTOR_SECRET, date(), id, Vote::Up); + let token = rating_token(VECTOR_SECRET, date(), id, Vote::Loved); let bad_date = format!("{}/r/not-a-date/{id}/up?t={token}", server.base); assert_eq!(client().get(&bad_date).send().await.unwrap().status(), 400); let bad_vote = format!("{}/r/2026-08-15/{id}/sideways?t={token}", server.base); assert_eq!(client().get(&bad_vote).send().await.unwrap().status(), 400); - assert!(server.db.ratings_with_feed().await.unwrap().is_empty()); + let count: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM rating_events") + .fetch_one(server.db.pool()) + .await + .unwrap(); + assert_eq!(count, 0); } #[tokio::test] diff --git a/src/types.rs b/src/types.rs index 14f1e77..32a6949 100644 --- a/src/types.rs +++ b/src/types.rs @@ -269,8 +269,6 @@ pub struct ScoredArticle { pub prefilter_score: f64, /// Cached [`composite_social_score`] for the article. pub social_score: f64, - /// Beta-smoothed per-feed upvote rate applied by the pre-filter (§3.9). - pub feed_prior: f64, /// `None` until stage A has run (or when `--skip-llm`). pub llm: Option, /// From `curation.always_include_feeds`: may be scored but never dropped (§3.5). @@ -278,10 +276,10 @@ pub struct ScoredArticle { } impl ScoredArticle { - /// Ranking key for stage B: LLM score weighted with social proof and priors (§3.6). + /// Ranking key for stage B: LLM score weighted with social proof (§3.6). pub fn combined_score(&self) -> f64 { let llm = self.llm.as_ref().map(|l| l.score).unwrap_or(0.0); - llm * 10.0 + self.social_score * 4.0 + self.feed_prior * 10.0 + self.prefilter_score * 0.1 + llm * 10.0 + self.social_score * 4.0 + self.prefilter_score * 0.1 } } @@ -565,62 +563,85 @@ pub struct Artifact { // Feedback (§3.9) // --------------------------------------------------------------------------- -/// 👍 / 👎 stored as `+1` / `-1` in `ratings.vote` (§3.9). +/// Explicit reader verdict embedded in rating-link URLs (§6.1). #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] pub enum Vote { - Up, - Down, + #[serde(rename = "loved", alias = "up")] + Loved, + #[serde(rename = "good")] + Good, + #[serde(rename = "down")] + NotForMe, } impl Vote { - pub fn as_i64(self) -> i64 { - match self { - Vote::Up => 1, - Vote::Down => -1, - } - } - - /// Path segment used in rating links: `up` / `down` (§3.9). + /// Stable path segment used in rating links (§6.1). pub fn as_str(self) -> &'static str { match self { - Vote::Up => "up", - Vote::Down => "down", + Vote::Loved => "loved", + Vote::Good => "good", + Vote::NotForMe => "down", } } pub fn parse(s: &str) -> Option { match s { - "up" => Some(Vote::Up), - "down" => Some(Vote::Down), + "loved" | "up" => Some(Vote::Loved), + "good" => Some(Vote::Good), + "down" => Some(Vote::NotForMe), _ => None, } } + + pub fn value(self, cfg: &crate::config::FeedbackConfig) -> f64 { + match self { + Vote::Loved => cfg.loved_value, + Vote::Good => cfg.good_value, + Vote::NotForMe => cfg.not_for_me_value, + } + } } -/// A recorded reader vote (`ratings` table, §3.9). +/// One append-only feedback event (`rating_events`, §6.2). #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -pub struct Rating { - pub issue_date: Date, +pub struct RatingEvent { + pub id: i64, pub article_id: ArticleId, - pub vote: Vote, - pub rated_at: Timestamp, + pub issue_date: Option, + pub kind: String, + pub source: String, + pub label: String, + pub value: f64, + pub note: Option, + pub event_at: Timestamp, } -/// Beta-smoothed per-feed upvote rate used by the pre-filter (`feed_priors`, §3.9). -#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize, Deserialize)] -pub struct FeedPrior { - pub feed_id: FeedId, - pub upvotes: i64, - pub downvotes: i64, - pub included: i64, +/// Descriptive deep-assessment facets (§12.1), populated beginning in step 5. +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct Facets { + pub format: Option, + pub depth: Option, + pub evidence: Option, + pub commerciality: Option, + pub topic_group: Option, + pub technicality: Option, + pub locality: Option, + pub specific_topics: Option>, } -impl FeedPrior { - /// `(up + 1) / (up + down + 2)` — 0.5 with no evidence (§3.9). - pub fn rate(&self) -> f64 { - (self.upvotes + 1) as f64 / (self.upvotes + self.downvotes + 2) as f64 - } +/// The current explicit verdict for an article, enriched for prompts (§6.2). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct RatedArticle { + pub article_id: ArticleId, + pub issue_date: Option, + pub title: String, + pub feed_title: String, + pub summary: Option, + pub facets: Option, + pub note: Option, + pub value: f64, + pub label: String, + pub event_at: Timestamp, } // --------------------------------------------------------------------------- @@ -684,18 +705,6 @@ mod tests { assert_eq!(composite_social_score(&[]), 0.0); } - #[test] - fn feed_prior_is_beta_smoothed() { - assert_eq!(FeedPrior::default().rate(), 0.5); - let p = FeedPrior { - feed_id: 1, - upvotes: 3, - downvotes: 1, - included: 4, - }; - assert!((p.rate() - 4.0 / 6.0).abs() < 1e-12); - } - #[test] fn stats_line_and_reading_time() { assert_eq!(reading_minutes(0), 1); @@ -730,8 +739,18 @@ mod tests { #[test] fn vote_and_social_source_round_trip() { - assert_eq!(Vote::parse("up"), Some(Vote::Up)); - assert_eq!(Vote::Down.as_i64(), -1); + let feedback = crate::config::FeedbackConfig::default(); + assert_eq!(Vote::parse("loved"), Some(Vote::Loved)); + assert_eq!(Vote::parse("up"), Some(Vote::Loved)); + assert_eq!(Vote::parse("good"), Some(Vote::Good)); + assert_eq!(Vote::parse("down"), Some(Vote::NotForMe)); + assert_eq!(Vote::Loved.as_str(), "loved"); + assert_eq!(Vote::Good.as_str(), "good"); + assert_eq!(Vote::NotForMe.as_str(), "down"); + assert_eq!(Vote::Loved.value(&feedback), 1.0); + assert_eq!(Vote::Good.value(&feedback), 0.35); + assert_eq!(Vote::NotForMe.value(&feedback), -1.0); + assert_eq!(serde_json::to_string(&Vote::NotForMe).unwrap(), "\"down\""); assert_eq!( SocialSource::parse("lobsters"), Some(SocialSource::Lobsters) diff --git a/tests/e2e_pipeline.rs b/tests/e2e_pipeline.rs index de17e67..81c07a1 100644 --- a/tests/e2e_pipeline.rs +++ b/tests/e2e_pipeline.rs @@ -283,7 +283,7 @@ async fn assemble_build_publish( ) .expect("render chapters"); let first = issue.lineup.picks[0].article.id; - let expected = auth::rating_url(&cfg.server.public_url, SECRET, date(), first, Vote::Up); + let expected = auth::rating_url(&cfg.server.public_url, SECRET, date(), first, Vote::Loved); let chapter = chapters .iter() .find(|c| c.id == format!("art-{}", issue.lineup.picks[0].article.best_entry_id)) @@ -298,8 +298,8 @@ async fn assemble_build_publish( SECRET, date(), first, - Vote::Up, - &auth::rating_token(SECRET, date(), first, Vote::Up) + Vote::Loved, + &auth::rating_token(SECRET, date(), first, Vote::Loved) ), "the server must accept the token the EPUB minted" ); diff --git a/tests/m3_curation.rs b/tests/m3_curation.rs index 1d700ea..da0d886 100644 --- a/tests/m3_curation.rs +++ b/tests/m3_curation.rs @@ -190,7 +190,18 @@ fn scour_opml_still_yields_the_interest_list() { // The assembled profile is what actually reaches DeepSeek as the system // prompt; it must mention the stated preferences and the interests (§3.6). - let document = profile::build(&interests, profile::NO_LEARNED_ADJUSTMENTS); + let profile_file = profile::load_profile(Path::new(concat!( + env!("CARGO_MANIFEST_DIR"), + "/data/profile.md" + ))) + .expect("profile file"); + let document = profile::build( + &profile_file.body, + &interests, + profile::NO_LEARNED_ADJUSTMENTS, + &[], + 60, + ); assert!(document.contains("Rust")); assert!(document.len() > 1000, "the profile is suspiciously short"); } diff --git a/tests/m4_epub.rs b/tests/m4_epub.rs index fbcde63..3879267 100644 --- a/tests/m4_epub.rs +++ b/tests/m4_epub.rs @@ -270,16 +270,20 @@ fn rating_links_carry_the_spec_token() { .expect("article chapter"); let date = issue.meta.date; - let up = build::rating_token("integration-secret", date, 1, Vote::Up); - let down = build::rating_token("integration-secret", date, 1, Vote::Down); - assert_eq!(up.len(), 16); - assert_ne!(up, down); - assert!(chapter.xhtml.contains(&format!( - "https://daily.hallada.net/r/2026-08-15/1/up?t={up}" - ))); - assert!(chapter.xhtml.contains(&format!( - "https://daily.hallada.net/r/2026-08-15/1/down?t={down}" - ))); + let loved = build::rating_token("integration-secret", date, 1, Vote::Loved); + let good = build::rating_token("integration-secret", date, 1, Vote::Good); + let down = build::rating_token("integration-secret", date, 1, Vote::NotForMe); + assert_eq!(loved.len(), 16); + assert_ne!(loved, good); + assert_ne!(good, down); + for (segment, token) in [("loved", loved), ("good", good), ("down", down)] { + assert!(chapter.xhtml.contains(&format!( + "https://daily.hallada.net/r/2026-08-15/1/{segment}?t={token}" + ))); + } + assert!(chapter.xhtml.contains("[ Loved it ]")); + assert!(chapter.xhtml.contains("[ Good ]")); + assert!(chapter.xhtml.contains("[ Not for me ]")); } #[test]