Curation v2 step 1: three-way feedback, rating events, reader profile
- Migration 0002: rating_events, article_embeddings, interest_embeddings, article_assessments, candidate_runs, runs.config_json/provider_costs_json, issue_articles.why; copies ratings into rating_events and drops ratings and feed_priors (scores stays until step 4). - Vote is Loved | Good | NotForMe; legacy `up` links still verify as Loved. - Footer offers Loved it / Good / Not for me; the confirmation page offers the other two so a mis-tap can be corrected. handle_rating appends one event. - db::current_ratings implements the latest-explicit-event rule with summaries and facets; `ratings list|set|clear` CLI appends source='cli' events. - data/profile.md replaces the hard-coded reader prose; the system prompt is rebuilt every run in the §8.4 order with a recent-verdicts block. - Weekly rebuild reads summaries, facets and notes; feed priors removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
This commit is contained in:
Claude Fable 5.1
@@ -10,7 +10,7 @@ select and introduce 15–25 of them. It assembles two EPUB editions (a standard
|
||||
and one tuned for the Xteink X4 e-ink reader), converts the X4 edition to XTC, and
|
||||
publishes the lot over its own OPDS catalog — which doubles as a
|
||||
[BookOrbit](https://github.com/thallada/bookorbit) watched folder if you run one.
|
||||
Each article chapter ends with 👍/👎 links that feed back into tomorrow's curation.
|
||||
Each article chapter ends with Loved it / Good / Not for me links that feed back into tomorrow's curation.
|
||||
|
||||
Steady-state cost is roughly **$0.05–0.30/day** in DeepSeek tokens, hard-capped by
|
||||
`max_daily_usd`.
|
||||
@@ -48,13 +48,13 @@ selects, feed excerpts stand in for summaries) instead of losing the day's issue
|
||||
| Rust (2024 edition toolchain) | building | `cargo build --release` |
|
||||
| **Miniflux** with an API key | the only content source | Settings → API Keys. The client is read-only and never mutates read state. |
|
||||
| **DeepSeek API key** | curation + editorial | <https://platform.deepseek.com>. Optional: `--skip-llm` runs the whole pipeline without it. |
|
||||
| A 32+ byte random secret | signs the 👍/👎 rating links | `openssl rand -hex 32` |
|
||||
| A 32+ byte random secret | signs the article rating links | `openssl rand -hex 32` |
|
||||
| **BookOrbit** library + watched folder | *optional* — a richer library UI on top of the same folder | Delivery does not need it: `daily-epub serve` has its own OPDS catalog over `publish.epub_dir`. If you do run it, create a dedicated "The Daily EPUB" library, enable *Watch folders*, and point `publish.epub_dir` at it. |
|
||||
| **Node.js 18+** and a clone of [`epub-to-xtc-converter`](https://github.com/bigbag/epub-to-xtc-converter) | XTC/XTCH output for the Xteink X4 | Optional (`xtc.enabled = false` turns it off). Needs `npm install` **inside `cli/`**, and a settings JSON naming a real TTF/OTF — see below. It has **no global npm bin** — it is invoked as `node <repo>/cli/index.js convert …`, which is why `xtc.command`/`xtc.args` are fully general. |
|
||||
| A reverse proxy for `daily.hallada.net` → `127.0.0.1:3499` | rating links must be reachable from e-readers on the internet | TLS via your existing setup. |
|
||||
|
||||
`data/scour-interests.opml` (the ~220 Scour interests the taste profile is seeded
|
||||
from) must be readable at the path in `interests_opml`.
|
||||
`data/profile.md` is the hand-maintained reader profile; its optional interests are merged
|
||||
with `data/scour-interests.opml`. Both paths are configurable.
|
||||
|
||||
---
|
||||
|
||||
@@ -72,7 +72,10 @@ sudo install -m0755 target/release/daily-epub /usr/local/bin/
|
||||
```
|
||||
daily-epub generate [--date YYYY-MM-DD] [--dry-run] [--out DIR] [--max-articles N] [--skip-llm]
|
||||
daily-epub serve # rating endpoints + OPDS catalog + downloads
|
||||
daily-epub profile rebuild # regenerate the taste profile from ratings (weekly inside generate)
|
||||
daily-epub profile rebuild # regenerate learned profile adjustments
|
||||
daily-epub ratings list --days 90
|
||||
daily-epub ratings set --article 42 --label loved --note "excellent"
|
||||
daily-epub ratings clear --url https://example.com/article
|
||||
daily-epub backfill-social # re-poll social scores for recent articles
|
||||
daily-epub db migrate # run migrations (also automatic on every start)
|
||||
```
|
||||
@@ -114,7 +117,8 @@ Secrets belong in the environment file, never in the TOML.
|
||||
| `world_briefing` | `true` | Include the Wikipedia Current Events section. |
|
||||
| `database_path` | `/var/lib/daily-epub/daily-epub.db` | SQLite file; parent dirs are created. |
|
||||
| `out_dir` | `/var/lib/daily-epub/out` | Where `generate` writes artifacts before publishing. |
|
||||
| `interests_opml` | `data/scour-interests.opml` | Scour interest export used to seed the taste profile. |
|
||||
| `profile_path` | `data/profile.md` | Hand-maintained reader profile, loaded every run. |
|
||||
| `interests_opml` | `data/scour-interests.opml` | Scour interests merged with the profile interests. |
|
||||
| `miniflux.base_url` | `http://127.0.0.1:8082` | Miniflux root (no `/v1`). |
|
||||
| `miniflux.api_key` | — | **`DAILY_EPUB_MINIFLUX__API_KEY`**. Required. |
|
||||
| `miniflux.page_limit` | `250` | Entries per page; Miniflux caps this at 250. |
|
||||
@@ -131,6 +135,10 @@ Secrets belong in the environment file, never in the TOML.
|
||||
| `curation.blocked_domains` | `[]` | Hosts excluded outright. |
|
||||
| `curation.paywall_domains` | `[]` | Extra paywalled hosts, merged with the built-in list (nytimes, wsj, ft, economist, …). |
|
||||
| `curation.sections` | 8 sections | The **only** section names the model may use. `World Briefing` is reserved and never offered. |
|
||||
| `curation.feedback.loved_value` | `1.0` | Weight for a Loved it verdict. |
|
||||
| `curation.feedback.good_value` | `0.35` | Weight for a Good verdict. |
|
||||
| `curation.feedback.not_for_me_value` | `-1.0` | Weight for a Not for me verdict. |
|
||||
| `curation.feedback.verdicts_in_prompt` | `60` | Recent explicit verdicts included in the system prompt. |
|
||||
| `publish.epub_dir` | `/srv/bookorbit/libraries/daily-epub` | Both EPUB editions land here by atomic copy, and this is the directory the OPDS feed lists. The editions are distinguished by a `(X4)` tag in **both** the filename and `dc:title` — libraries and OPDS clients list books by title, so the filename alone would make them look identical. Point a BookOrbit watched folder at it if you want its UI too. **Renamed from `bookorbit_dir`**; the old key is a hard config error. |
|
||||
| `publish.xtc_dir` | `/var/lib/daily-epub/xtc` | XTC artifacts. **Not** listed in the OPDS feed — CrossPoint cannot acquire them — but downloadable at `/files/xtc/<name>` for sideloading. |
|
||||
| `xtc.enabled` | `true` | Set `false` to skip the converter entirely. |
|
||||
@@ -356,9 +364,8 @@ curl -s https://daily.hallada.net/healthz
|
||||
curl -s https://daily.hallada.net/opds/daily.xml | head
|
||||
curl -s https://daily.hallada.net/issues.json | jq '.[0]'
|
||||
|
||||
# 7. Feedback loop: tap 👍 in KOReader, then
|
||||
sqlite3 /var/lib/daily-epub/daily-epub.db 'select * from ratings;'
|
||||
sqlite3 /var/lib/daily-epub/daily-epub.db 'select * from feed_priors;'
|
||||
# 7. Feedback loop: tap Loved it / Good / Not for me in KOReader, then
|
||||
sqlite3 /var/lib/daily-epub/daily-epub.db 'select * from rating_events order by event_at desc;'
|
||||
|
||||
# 8. Watch cost and quality for a week
|
||||
sqlite3 /var/lib/daily-epub/daily-epub.db \
|
||||
|
||||
+8
-1
@@ -22,7 +22,8 @@ database_path = "/var/lib/daily-epub/daily-epub.db"
|
||||
# Default output directory for generated artifacts (overridden by `--out`).
|
||||
out_dir = "/var/lib/daily-epub/out"
|
||||
|
||||
# Path to the Scour interests OPML used to seed the taste profile (§3.6).
|
||||
# Hand-maintained reader profile and Scour interests merged into the system prompt.
|
||||
profile_path = "data/profile.md"
|
||||
interests_opml = "data/scour-interests.opml"
|
||||
|
||||
[miniflux]
|
||||
@@ -59,6 +60,12 @@ sections = [
|
||||
"From the Blogroll",
|
||||
]
|
||||
|
||||
[curation.feedback]
|
||||
loved_value = 1.0
|
||||
good_value = 0.35
|
||||
not_for_me_value = -1.0
|
||||
verdicts_in_prompt = 60
|
||||
|
||||
[publish]
|
||||
# Where both EPUB editions land, and what the OPDS feed lists. BookOrbit is
|
||||
# optional — it just watches this folder if you run it.
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# Reader profile
|
||||
|
||||
## Who he is
|
||||
A software engineer in the Boston area who reads on e-ink in the morning. He would
|
||||
rather read six excellent long pieces than thirty adequate short ones. He reads across
|
||||
an unusually wide range of subjects and does not need a topic to be professionally
|
||||
useful to enjoy it.
|
||||
|
||||
## What he wants
|
||||
- Long-form and high-effort above all: essays, deep dives, post-mortems, field notes,
|
||||
annotated experiments, thorough explainers, personal narratives with real specificity.
|
||||
Length is a proxy, not the goal.
|
||||
- Any topic, if the writing is excellent.
|
||||
- Social proof is evidence a critical audience read it, not a verdict.
|
||||
- Boston and New England local news: city government, transit, universities, civic stories.
|
||||
- Ultra-niche community news: small scenes with their own vocabulary.
|
||||
- World and US news kept light and neutral (the World Briefing covers it separately).
|
||||
|
||||
## What he does not want
|
||||
Press releases and funding announcements dressed as news; SEO listicles; link roundups;
|
||||
changelogs without analysis; sponsored content; crypto and engagement bait; rewrites of a
|
||||
story he can read at the source; culture-war outrage; one paragraph stretched to five.
|
||||
|
||||
## How to judge
|
||||
Would he still be glad he read this an hour later? Reward specificity, first-hand
|
||||
experience, honest uncertainty, and prose with a human behind it. Penalize padding,
|
||||
unsourced confidence, and summaries of other people's work.
|
||||
|
||||
## Interests
|
||||
(optional: one per line; merged with data/scour-interests.opml)
|
||||
@@ -0,0 +1,96 @@
|
||||
-- Personalized curation v2 schema (plan 2026-09-02 §7).
|
||||
-- `scores` intentionally remains until migration 0003 (step 4).
|
||||
|
||||
CREATE TABLE rating_events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
article_id INTEGER NOT NULL REFERENCES articles(id) ON DELETE CASCADE,
|
||||
issue_date TEXT,
|
||||
kind TEXT NOT NULL CHECK (kind IN ('explicit', 'implicit')),
|
||||
source TEXT NOT NULL,
|
||||
label TEXT NOT NULL,
|
||||
value REAL NOT NULL,
|
||||
note TEXT,
|
||||
event_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX idx_rating_events_article ON rating_events(article_id, event_at);
|
||||
CREATE INDEX idx_rating_events_at ON rating_events(event_at);
|
||||
|
||||
-- Filled in by step 3.
|
||||
CREATE TABLE article_embeddings (
|
||||
article_id INTEGER PRIMARY KEY REFERENCES articles(id) ON DELETE CASCADE,
|
||||
model TEXT NOT NULL,
|
||||
dimension INTEGER NOT NULL,
|
||||
input_hash TEXT NOT NULL,
|
||||
embedding BLOB NOT NULL,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
-- Filled in by step 3.
|
||||
CREATE TABLE interest_embeddings (
|
||||
interest TEXT PRIMARY KEY,
|
||||
model TEXT NOT NULL,
|
||||
dimension INTEGER NOT NULL,
|
||||
embedding BLOB NOT NULL,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
-- Triage rows are filled in by step 4; deep rows by step 5.
|
||||
CREATE TABLE article_assessments (
|
||||
article_id INTEGER NOT NULL REFERENCES articles(id) ON DELETE CASCADE,
|
||||
stage TEXT NOT NULL CHECK (stage IN ('triage', 'deep')),
|
||||
model TEXT NOT NULL,
|
||||
prompt_version INTEGER NOT NULL,
|
||||
profile_version INTEGER,
|
||||
score REAL,
|
||||
fit REAL,
|
||||
kind TEXT,
|
||||
facets_json TEXT,
|
||||
rationale TEXT,
|
||||
category TEXT,
|
||||
paywalled_guess INTEGER NOT NULL DEFAULT 0,
|
||||
assessed_at TEXT NOT NULL,
|
||||
PRIMARY KEY (article_id, stage)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_article_assessments_at ON article_assessments(assessed_at);
|
||||
|
||||
-- Filled in by step 3.
|
||||
CREATE TABLE candidate_runs (
|
||||
run_id INTEGER NOT NULL REFERENCES runs(id) ON DELETE CASCADE,
|
||||
article_id INTEGER NOT NULL REFERENCES articles(id) ON DELETE CASCADE,
|
||||
stage TEXT NOT NULL,
|
||||
excluded_reason TEXT,
|
||||
admitted_by TEXT,
|
||||
signals_json TEXT NOT NULL,
|
||||
utility REAL,
|
||||
rank_utility INTEGER,
|
||||
cluster_id INTEGER,
|
||||
cluster_rank INTEGER,
|
||||
editor_why TEXT,
|
||||
PRIMARY KEY (run_id, article_id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_candidate_runs_article ON candidate_runs(article_id);
|
||||
CREATE INDEX idx_candidate_runs_run_stage ON candidate_runs(run_id, stage);
|
||||
|
||||
-- Filled in by later telemetry/provider steps.
|
||||
ALTER TABLE runs ADD COLUMN config_json TEXT;
|
||||
ALTER TABLE runs ADD COLUMN provider_costs_json TEXT;
|
||||
-- Filled in by step 2.
|
||||
ALTER TABLE issue_articles ADD COLUMN why TEXT;
|
||||
|
||||
INSERT INTO rating_events
|
||||
(article_id, issue_date, kind, source, label, value, note, event_at)
|
||||
SELECT article_id,
|
||||
issue_date,
|
||||
'explicit',
|
||||
'migration',
|
||||
CASE vote WHEN 1 THEN 'loved' ELSE 'not_for_me' END,
|
||||
CASE vote WHEN 1 THEN 1.0 ELSE -1.0 END,
|
||||
NULL,
|
||||
rated_at
|
||||
FROM ratings;
|
||||
|
||||
DROP TABLE ratings;
|
||||
DROP TABLE feed_priors;
|
||||
+53
-33
@@ -5,14 +5,14 @@
|
||||
//! both sides. It lives here and nowhere else:
|
||||
//!
|
||||
//! ```text
|
||||
//! message = "{issue_date}/{article_id}/{up|down}"
|
||||
//! message = "{issue_date}/{article_id}/{loved|good|down}"
|
||||
//! token = hex(hmac_sha256(secret, message))[..16]
|
||||
//! link = {public_url}/r/{issue_date}/{article_id}/{vote}?t={token}
|
||||
//! ```
|
||||
//!
|
||||
//! Pinned test vector, asserted from three places (here, `epub::build`,
|
||||
//! `tests/m7_server.rs`): `secret = "test-secret"`, date `2026-08-15`,
|
||||
//! article `42`, `up` → `3b314cf7e6d8f50f`.
|
||||
//! article `42`, `loved` (with legacy `up` verification).
|
||||
|
||||
use hmac::{Hmac, KeyInit, Mac};
|
||||
use jiff::civil::Date;
|
||||
@@ -23,17 +23,26 @@ use crate::types::{ArticleId, Vote};
|
||||
/// Characters of the hex HMAC kept in rating links (§3.9).
|
||||
pub const TOKEN_LEN: usize = 16;
|
||||
|
||||
/// The exact signed string: `{issue_date}/{article_id}/{up|down}` (§3.9).
|
||||
/// The exact signed string: `{issue_date}/{article_id}/{loved|good|down}` (§3.9).
|
||||
pub fn rating_message(issue_date: Date, article_id: ArticleId, vote: Vote) -> String {
|
||||
format!("{issue_date}/{article_id}/{}", vote.as_str())
|
||||
}
|
||||
|
||||
/// `hex(hmac_sha256(secret, "{issue_date}/{article_id}/{vote}"))[..16]` (§3.9).
|
||||
pub fn rating_token(secret: &str, issue_date: Date, article_id: ArticleId, vote: Vote) -> String {
|
||||
rating_token_for_segment(secret, issue_date, article_id, vote.as_str())
|
||||
}
|
||||
|
||||
fn rating_token_for_segment(
|
||||
secret: &str,
|
||||
issue_date: Date,
|
||||
article_id: ArticleId,
|
||||
segment: &str,
|
||||
) -> String {
|
||||
// `Hmac` derives a fixed-size key from any input length, so this never fails.
|
||||
let mut mac = <Hmac<Sha256> as KeyInit>::new_from_slice(secret.as_bytes())
|
||||
.expect("HMAC accepts keys of any length");
|
||||
mac.update(rating_message(issue_date, article_id, vote).as_bytes());
|
||||
mac.update(format!("{issue_date}/{article_id}/{segment}").as_bytes());
|
||||
let digest = hex::encode(mac.finalize().into_bytes());
|
||||
digest[..TOKEN_LEN].to_string()
|
||||
}
|
||||
@@ -46,8 +55,14 @@ pub fn verify_token(
|
||||
vote: Vote,
|
||||
token: &str,
|
||||
) -> bool {
|
||||
constant_time_eq(
|
||||
rating_token(secret, issue_date, article_id, vote).as_bytes(),
|
||||
let current = rating_token(secret, issue_date, article_id, vote);
|
||||
if constant_time_eq(current.as_bytes(), token.as_bytes()) {
|
||||
return true;
|
||||
}
|
||||
// Already-published `up` links were signed over the literal legacy segment.
|
||||
vote == Vote::Loved
|
||||
&& constant_time_eq(
|
||||
rating_token_for_segment(secret, issue_date, article_id, "up").as_bytes(),
|
||||
token.as_bytes(),
|
||||
)
|
||||
}
|
||||
@@ -67,7 +82,7 @@ pub fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
|
||||
}
|
||||
|
||||
/// Full rating URL embedded in an article footer:
|
||||
/// `{public_url}/r/{date}/{article_id}/{up|down}?t={token}` (§3.9).
|
||||
/// `{public_url}/r/{date}/{article_id}/{loved|good|down}?t={token}` (§3.9).
|
||||
pub fn rating_url(
|
||||
public_url: &str,
|
||||
secret: &str,
|
||||
@@ -92,50 +107,55 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn token_matches_the_shared_vector() {
|
||||
assert_eq!(rating_message(date(), 42, Vote::Up), "2026-08-15/42/up");
|
||||
assert_eq!(
|
||||
rating_token("test-secret", date(), 42, Vote::Up),
|
||||
"3b314cf7e6d8f50f"
|
||||
);
|
||||
assert_eq!(rating_token("test-secret", date(), 42, Vote::Up).len(), 16);
|
||||
fn all_three_tokens_verify_and_are_distinct() {
|
||||
let votes = [Vote::Loved, Vote::Good, Vote::NotForMe];
|
||||
let tokens: Vec<String> = votes
|
||||
.iter()
|
||||
.map(|vote| rating_token("test-secret", date(), 42, *vote))
|
||||
.collect();
|
||||
assert_eq!(tokens.len(), 3);
|
||||
assert!(tokens.iter().all(|token| token.len() == TOKEN_LEN));
|
||||
assert_ne!(tokens[0], tokens[1]);
|
||||
assert_ne!(tokens[1], tokens[2]);
|
||||
for (vote, token) in votes.into_iter().zip(tokens) {
|
||||
assert!(verify_token("test-secret", date(), 42, vote, &token));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tokens_are_per_article_and_per_vote() {
|
||||
let up = rating_token("s", date(), 42, Vote::Up);
|
||||
assert_ne!(up, rating_token("s", date(), 42, Vote::Down));
|
||||
assert_ne!(up, rating_token("s", date(), 43, Vote::Up));
|
||||
assert_ne!(up, rating_token("other", date(), 42, Vote::Up));
|
||||
let tomorrow: Date = "2026-08-16".parse().unwrap();
|
||||
assert_ne!(up, rating_token("s", tomorrow, 42, Vote::Up));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verification_is_exact() {
|
||||
fn legacy_up_token_still_verifies_as_loved() {
|
||||
let legacy = rating_token_for_segment("test-secret", date(), 42, "up");
|
||||
assert_eq!(legacy, "3b314cf7e6d8f50f");
|
||||
assert!(verify_token(
|
||||
"s",
|
||||
"test-secret",
|
||||
date(),
|
||||
42,
|
||||
Vote::Up,
|
||||
&rating_token("s", date(), 42, Vote::Up)
|
||||
Vote::Loved,
|
||||
&legacy
|
||||
));
|
||||
assert!(!verify_token("s", date(), 42, Vote::Up, "deadbeefdeadbeef"));
|
||||
assert!(!verify_token("s", date(), 42, Vote::Up, ""));
|
||||
assert!(!verify_token("s", date(), 42, Vote::Up, "short"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verification_rejects_tampering() {
|
||||
let token = rating_token("s", date(), 42, Vote::Loved);
|
||||
assert!(!verify_token("s", date(), 42, Vote::Good, &token));
|
||||
assert!(!verify_token("s", date(), 43, Vote::Loved, &token));
|
||||
assert!(!verify_token("other", date(), 42, Vote::Loved, &token));
|
||||
assert!(!verify_token("s", date(), 42, Vote::Loved, "short"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_shape_matches_the_spec() {
|
||||
let token = rating_token("test-secret", date(), 42, Vote::Good);
|
||||
assert_eq!(
|
||||
rating_url(
|
||||
"https://daily.hallada.net/",
|
||||
"test-secret",
|
||||
date(),
|
||||
42,
|
||||
Vote::Up
|
||||
Vote::Good
|
||||
),
|
||||
"https://daily.hallada.net/r/2026-08-15/42/up?t=3b314cf7e6d8f50f"
|
||||
format!("https://daily.hallada.net/r/2026-08-15/42/good?t={token}")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -69,6 +69,8 @@ pub struct Config {
|
||||
pub out_dir: PathBuf,
|
||||
/// Scour interests OPML used to seed the taste profile (§3.6).
|
||||
pub interests_opml: PathBuf,
|
||||
/// Hand-maintained reader profile loaded for every curation run (§8.2).
|
||||
pub profile_path: PathBuf,
|
||||
|
||||
pub miniflux: MinifluxConfig,
|
||||
pub deepseek: DeepseekConfig,
|
||||
@@ -92,6 +94,7 @@ impl Default for Config {
|
||||
database_path: PathBuf::from("/var/lib/daily-epub/daily-epub.db"),
|
||||
out_dir: PathBuf::from("/var/lib/daily-epub/out"),
|
||||
interests_opml: PathBuf::from("data/scour-interests.opml"),
|
||||
profile_path: PathBuf::from("data/profile.md"),
|
||||
miniflux: MinifluxConfig::default(),
|
||||
deepseek: DeepseekConfig::default(),
|
||||
curation: CurationConfig::default(),
|
||||
@@ -172,6 +175,7 @@ pub struct CurationConfig {
|
||||
pub paywall_domains: Vec<String>,
|
||||
/// The only section names the LLM may use (§3.6 stage B).
|
||||
pub sections: Vec<String>,
|
||||
pub feedback: FeedbackConfig,
|
||||
}
|
||||
|
||||
impl Default for CurationConfig {
|
||||
@@ -193,6 +197,28 @@ impl Default for CurationConfig {
|
||||
.iter()
|
||||
.map(|s| s.to_string())
|
||||
.collect(),
|
||||
feedback: FeedbackConfig::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `[curation.feedback]` — explicit verdict weights and prompt history (§6, §8.4).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields, default)]
|
||||
pub struct FeedbackConfig {
|
||||
pub loved_value: f64,
|
||||
pub good_value: f64,
|
||||
pub not_for_me_value: f64,
|
||||
pub verdicts_in_prompt: usize,
|
||||
}
|
||||
|
||||
impl Default for FeedbackConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
loved_value: 1.0,
|
||||
good_value: 0.35,
|
||||
not_for_me_value: -1.0,
|
||||
verdicts_in_prompt: 60,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -382,6 +408,9 @@ mod tests {
|
||||
assert_eq!(c.max_daily_usd, 2.0);
|
||||
assert!(c.world_briefing);
|
||||
assert_eq!(c.deepseek.model, "deepseek-v4-flash");
|
||||
assert_eq!(c.profile_path, PathBuf::from("data/profile.md"));
|
||||
assert_eq!(c.curation.feedback.good_value, 0.35);
|
||||
assert_eq!(c.curation.feedback.verdicts_in_prompt, 60);
|
||||
assert_eq!(c.xtc.format, XtcFormat::Xtch);
|
||||
assert_eq!(c.curation.sections.len(), 8);
|
||||
c.validate().unwrap();
|
||||
|
||||
+5
-79
@@ -12,15 +12,14 @@
|
||||
//! | came via Scour | +8 | §3.5 (already matched a stated interest) |
|
||||
//! | came via HN frontpage | +8 | §3.5 |
|
||||
//! | carried by several feeds | 0 … +8 | §3.2 (multi-source *is* social proof) |
|
||||
//! | feed prior | −12 … +12 | §3.9 beta-smoothed upvote rate, neutral at 0.5 |
|
||||
//! | excerpt only | −20 | §3.5 (penalized, never banned — §7) |
|
||||
//! | roundup/release-notes title | −15 | §3.5 |
|
||||
//! | blocked domain | excluded | §3.5 |
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::collections::HashSet;
|
||||
|
||||
use crate::config::{Config, CurationConfig};
|
||||
use crate::types::{Article, ArticleId, FeedId, FeedPrior, ScoredArticle, SourceKind};
|
||||
use crate::types::{Article, ArticleId, FeedId, ScoredArticle, SourceKind};
|
||||
|
||||
/// Title patterns that mark low-effort posts: link roundups, release notes,
|
||||
/// sponsor posts (§3.5).
|
||||
@@ -64,7 +63,6 @@ pub const MAX_SOCIAL_POINTS: f64 = 25.0;
|
||||
pub const SCOUR_BONUS: f64 = 8.0;
|
||||
pub const HN_FRONTPAGE_BONUS: f64 = 8.0;
|
||||
pub const MAX_MULTI_SOURCE_POINTS: f64 = 8.0;
|
||||
pub const MAX_FEED_PRIOR_POINTS: f64 = 12.0;
|
||||
pub const EXCERPT_ONLY_PENALTY: f64 = 20.0;
|
||||
pub const ROUNDUP_TITLE_PENALTY: f64 = 15.0;
|
||||
|
||||
@@ -75,8 +73,6 @@ const SOCIAL_SATURATION: f64 = 6.0;
|
||||
/// Everything the pre-filter needs beyond the articles themselves (§3.5, §3.9).
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct PrefilterContext {
|
||||
/// Per-feed Bayesian upvote rate from ratings history (§3.9).
|
||||
pub feed_priors: HashMap<FeedId, FeedPrior>,
|
||||
/// Article ids already published in a previous issue (§3.5).
|
||||
pub already_published: Vec<ArticleId>,
|
||||
/// Article ids the LLM scored < [`STALE_LOW_SCORE`] recently (§3.5).
|
||||
@@ -94,43 +90,18 @@ impl PrefilterContext {
|
||||
let since = today
|
||||
.checked_sub(jiff::Span::new().days(STALE_LOOKBACK_DAYS))
|
||||
.unwrap_or(today);
|
||||
let feed_priors = db
|
||||
.feed_priors()
|
||||
.await?
|
||||
.into_iter()
|
||||
.map(|p| (p.feed_id, p))
|
||||
.collect();
|
||||
let already_published = db.previously_published_ids().await?;
|
||||
let recently_rejected = db.recently_low_scored_ids(STALE_LOW_SCORE, since).await?;
|
||||
tracing::debug!(
|
||||
priors = ?feed_priors_len(&feed_priors),
|
||||
published = already_published.len(),
|
||||
rejected = recently_rejected.len(),
|
||||
"loaded prefilter context"
|
||||
);
|
||||
Ok(Self {
|
||||
feed_priors,
|
||||
already_published,
|
||||
recently_rejected,
|
||||
})
|
||||
}
|
||||
|
||||
fn prior_for(&self, article: &Article) -> f64 {
|
||||
// The cluster's feeds are all candidates; take the most favourable one,
|
||||
// since a story carried by a well-rated feed is a better bet.
|
||||
let mut best = self.feed_priors.get(&article.feed_id).map(FeedPrior::rate);
|
||||
for source in &article.sources {
|
||||
if let Some(p) = self.feed_priors.get(&source.feed_id) {
|
||||
let rate = p.rate();
|
||||
best = Some(best.map_or(rate, |b: f64| b.max(rate)));
|
||||
}
|
||||
}
|
||||
best.unwrap_or(0.5)
|
||||
}
|
||||
}
|
||||
|
||||
fn feed_priors_len(m: &HashMap<FeedId, FeedPrior>) -> usize {
|
||||
m.len()
|
||||
}
|
||||
|
||||
/// True when the article's feed is in `curation.always_include_feeds` (§3.5).
|
||||
@@ -226,9 +197,9 @@ pub fn social_points(social_score: f64) -> f64 {
|
||||
MAX_SOCIAL_POINTS * (social_score / SOCIAL_SATURATION).min(1.0).sqrt()
|
||||
}
|
||||
|
||||
/// Score one article 0–100 from word count, social proof, source signals, feed
|
||||
/// prior, and the excerpt/roundup/blocklist penalties (§3.5).
|
||||
pub fn score_article(article: &Article, ctx: &PrefilterContext, cfg: &Config) -> f64 {
|
||||
/// Score one article 0–100 from word count, social proof, source signals,
|
||||
/// and the excerpt/roundup/blocklist penalties (§3.5).
|
||||
pub fn score_article(article: &Article, _ctx: &PrefilterContext, cfg: &Config) -> f64 {
|
||||
if is_blocked(article, &cfg.curation) {
|
||||
return 0.0;
|
||||
}
|
||||
@@ -245,9 +216,6 @@ pub fn score_article(article: &Article, ctx: &PrefilterContext, cfg: &Config) ->
|
||||
let extra_feeds = article.sources.len().saturating_sub(1) as f64;
|
||||
score += (extra_feeds * 4.0).min(MAX_MULTI_SOURCE_POINTS);
|
||||
|
||||
// Beta-smoothed upvote rate, neutral (0.5) contributing nothing (§3.9).
|
||||
score += (ctx.prior_for(article) - 0.5) * 2.0 * MAX_FEED_PRIOR_POINTS;
|
||||
|
||||
if article.excerpt_only {
|
||||
score -= EXCERPT_ONLY_PENALTY;
|
||||
}
|
||||
@@ -288,12 +256,10 @@ pub fn run(articles: Vec<Article>, ctx: &PrefilterContext, cfg: &Config) -> Vec<
|
||||
|
||||
let prefilter_score = score_article(&article, ctx, cfg);
|
||||
let social_score = article.social_score();
|
||||
let feed_prior = ctx.prior_for(&article);
|
||||
scored.push(ScoredArticle {
|
||||
article,
|
||||
prefilter_score,
|
||||
social_score,
|
||||
feed_prior,
|
||||
llm: None,
|
||||
auto_include,
|
||||
});
|
||||
@@ -488,35 +454,6 @@ pub(crate) mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn feed_prior_moves_the_score_both_ways() {
|
||||
let cfg = cfg();
|
||||
let mut liked = PrefilterContext::default();
|
||||
liked.feed_priors.insert(
|
||||
7,
|
||||
FeedPrior {
|
||||
feed_id: 7,
|
||||
upvotes: 18,
|
||||
downvotes: 0,
|
||||
included: 18,
|
||||
},
|
||||
);
|
||||
let mut disliked = PrefilterContext::default();
|
||||
disliked.feed_priors.insert(
|
||||
7,
|
||||
FeedPrior {
|
||||
feed_id: 7,
|
||||
upvotes: 0,
|
||||
downvotes: 18,
|
||||
included: 18,
|
||||
},
|
||||
);
|
||||
let a = article(1, "Deep dive", 1200);
|
||||
let neutral = score_article(&a, &PrefilterContext::default(), &cfg);
|
||||
assert!(score_article(&a, &liked, &cfg) > neutral);
|
||||
assert!(score_article(&a, &disliked, &cfg) < neutral);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blocked_domains_and_auto_includes_match_urls_and_ids() {
|
||||
let mut cfg = cfg();
|
||||
@@ -563,7 +500,6 @@ pub(crate) mod tests {
|
||||
let ctx = PrefilterContext {
|
||||
already_published: vec![4],
|
||||
recently_rejected: vec![6],
|
||||
..PrefilterContext::default()
|
||||
};
|
||||
|
||||
let kept = run(articles, &ctx, &cfg);
|
||||
@@ -598,7 +534,6 @@ pub(crate) mod tests {
|
||||
let ctx = PrefilterContext {
|
||||
recently_rejected: vec![1],
|
||||
already_published: vec![2],
|
||||
..PrefilterContext::default()
|
||||
};
|
||||
let kept = run(vec![a, b], &ctx, &cfg);
|
||||
let ids: Vec<ArticleId> = kept.iter().map(|s| s.article.id).collect();
|
||||
@@ -613,14 +548,6 @@ pub(crate) mod tests {
|
||||
.expect("db");
|
||||
let date: jiff::civil::Date = "2026-08-15".parse().expect("date");
|
||||
|
||||
db.upsert_feed_prior(&FeedPrior {
|
||||
feed_id: 7,
|
||||
upvotes: 4,
|
||||
downvotes: 1,
|
||||
included: 5,
|
||||
})
|
||||
.await
|
||||
.expect("prior");
|
||||
sqlx::query(
|
||||
"INSERT INTO articles (id, canonical_url, title, first_seen) VALUES
|
||||
(42, 'https://example.com/42', 'Printed', '2026-08-14T00:00:00Z'),
|
||||
@@ -660,6 +587,5 @@ pub(crate) mod tests {
|
||||
let ctx = PrefilterContext::load(&db, date).await.expect("context");
|
||||
assert_eq!(ctx.already_published, vec![42]);
|
||||
assert_eq!(ctx.recently_rejected, vec![43], "old rejects age out");
|
||||
assert!((ctx.feed_priors[&7].rate() - 5.0 / 7.0).abs() < 1e-12);
|
||||
}
|
||||
}
|
||||
|
||||
+434
-566
File diff suppressed because it is too large
Load Diff
@@ -425,7 +425,6 @@ mod tests {
|
||||
article: article(id, title, words),
|
||||
prefilter_score: 50.0,
|
||||
social_score: 0.0,
|
||||
feed_prior: 0.5,
|
||||
llm: None,
|
||||
auto_include: false,
|
||||
}
|
||||
|
||||
@@ -144,9 +144,8 @@ fn render_candidate(candidate: &ScoredArticle) -> String {
|
||||
}
|
||||
let _ = writeln!(
|
||||
block,
|
||||
"signals: social {:.2}; feed prior {:.2}; via {}{}",
|
||||
"signals: social {:.2}; via {}{}",
|
||||
candidate.social_score,
|
||||
candidate.feed_prior,
|
||||
source_kinds(candidate),
|
||||
if candidate.auto_include {
|
||||
"; ALWAYS-INCLUDE"
|
||||
@@ -716,7 +715,6 @@ mod tests {
|
||||
article: article(id, title, words),
|
||||
prefilter_score: 40.0 + score,
|
||||
social_score: 1.0,
|
||||
feed_prior: 0.5,
|
||||
llm: Some(LlmScore {
|
||||
score,
|
||||
category: "Tech & Engineering".into(),
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
//!
|
||||
//! Runtime queries only — no `sqlx::query!` macros (implementation notes §1).
|
||||
//! Timestamps are stored as RFC3339 UTC strings and dates as `YYYY-MM-DD`
|
||||
//! (implementation notes §2). Every write is an idempotent upsert so that
|
||||
//! `generate --date X` can be re-run safely (implementation notes §12).
|
||||
//! (implementation notes §2). Pipeline writes are idempotent upserts so that
|
||||
//! `generate --date X` can be re-run safely; feedback events are append-only.
|
||||
|
||||
use std::path::Path;
|
||||
use std::str::FromStr;
|
||||
@@ -15,8 +15,8 @@ use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions, S
|
||||
use sqlx::{Row, SqlitePool};
|
||||
|
||||
use crate::types::{
|
||||
Article, ArticleId, Entry, EntryId, FeedId, FeedPrior, LlmScore, Pick, Rating, SocialRef,
|
||||
SocialSource, SourceRef, Vote,
|
||||
Article, ArticleId, Entry, EntryId, Facets, LlmScore, Pick, RatedArticle, RatingEvent,
|
||||
SocialRef, SocialSource, SourceRef,
|
||||
};
|
||||
|
||||
/// Embedded migrations from `./migrations` (implementation notes §1).
|
||||
@@ -535,113 +535,104 @@ impl Db {
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------
|
||||
// ratings + feed priors (§3.9)
|
||||
// append-only rating events (§6.2)
|
||||
// -----------------------------------------------------------------
|
||||
|
||||
/// Idempotent upsert of a reader vote (§3.9). Returns true if it changed anything.
|
||||
pub async fn upsert_rating(&self, rating: &Rating) -> Result<bool> {
|
||||
let res = sqlx::query(
|
||||
"INSERT INTO ratings (issue_date, article_id, vote, rated_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(issue_date, article_id) DO UPDATE SET
|
||||
vote = excluded.vote,
|
||||
rated_at = excluded.rated_at
|
||||
WHERE ratings.vote != excluded.vote",
|
||||
/// Append one feedback event and return its database id.
|
||||
pub async fn append_rating_event(&self, event: &RatingEvent) -> Result<i64> {
|
||||
let row = sqlx::query(
|
||||
"INSERT INTO rating_events
|
||||
(article_id, issue_date, kind, source, label, value, note, event_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(rating.issue_date.to_string())
|
||||
.bind(rating.article_id)
|
||||
.bind(rating.vote.as_i64())
|
||||
.bind(fmt_ts(rating.rated_at))
|
||||
.execute(&self.pool)
|
||||
.bind(event.article_id)
|
||||
.bind(event.issue_date.map(|date| date.to_string()))
|
||||
.bind(&event.kind)
|
||||
.bind(&event.source)
|
||||
.bind(&event.label)
|
||||
.bind(event.value)
|
||||
.bind(event.note.as_deref())
|
||||
.bind(fmt_ts(event.event_at))
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(res.rows_affected() > 0)
|
||||
Ok(row.get("id"))
|
||||
}
|
||||
|
||||
/// Every rating joined to the feed that carried the article (§3.9 priors).
|
||||
pub async fn ratings_with_feed(&self) -> Result<Vec<(FeedId, Vote)>> {
|
||||
/// Latest issue containing an article, used to attach CLI feedback when possible.
|
||||
pub async fn latest_issue_date_for_article(
|
||||
&self,
|
||||
article_id: ArticleId,
|
||||
) -> Result<Option<Date>> {
|
||||
let row = sqlx::query(
|
||||
"SELECT issue_date FROM issue_articles
|
||||
WHERE article_id = ? ORDER BY issue_date DESC LIMIT 1",
|
||||
)
|
||||
.bind(article_id)
|
||||
.fetch_optional(&self.pool)
|
||||
.await?;
|
||||
row.map(|row| {
|
||||
parse_date(
|
||||
"issue_articles.issue_date",
|
||||
&row.get::<String, _>("issue_date"),
|
||||
)
|
||||
})
|
||||
.transpose()
|
||||
}
|
||||
|
||||
/// Current explicit verdicts, newest first. A latest `cleared` event removes
|
||||
/// its article from this learned set (§6.2).
|
||||
pub async fn current_ratings(&self, lookback_days: i64) -> Result<Vec<RatedArticle>> {
|
||||
self.latest_explicit_ratings(lookback_days, false).await
|
||||
}
|
||||
|
||||
/// Current explicit events including `cleared`, for the ratings CLI.
|
||||
pub async fn current_ratings_including_cleared(
|
||||
&self,
|
||||
lookback_days: i64,
|
||||
) -> Result<Vec<RatedArticle>> {
|
||||
self.latest_explicit_ratings(lookback_days, true).await
|
||||
}
|
||||
|
||||
async fn latest_explicit_ratings(
|
||||
&self,
|
||||
lookback_days: i64,
|
||||
include_cleared: bool,
|
||||
) -> Result<Vec<RatedArticle>> {
|
||||
let since = Timestamp::now()
|
||||
.checked_sub(jiff::Span::new().hours(lookback_days.max(0).saturating_mul(24)))
|
||||
.unwrap_or(Timestamp::UNIX_EPOCH);
|
||||
let rows = sqlx::query(
|
||||
"SELECT e.feed_id AS feed_id, r.vote AS vote
|
||||
FROM ratings r
|
||||
"WITH ranked AS (
|
||||
SELECT re.*,
|
||||
ROW_NUMBER() OVER (
|
||||
PARTITION BY re.article_id
|
||||
ORDER BY re.event_at DESC, re.id DESC
|
||||
) AS event_rank
|
||||
FROM rating_events re
|
||||
WHERE re.kind = 'explicit' AND re.event_at >= ?
|
||||
)
|
||||
SELECT r.article_id, r.issue_date, r.label, r.value, r.note, r.event_at,
|
||||
COALESCE(a.title, '') AS title,
|
||||
COALESCE(e.feed_title, '') AS feed_title,
|
||||
(SELECT ia.summary FROM issue_articles ia
|
||||
WHERE ia.article_id = r.article_id
|
||||
ORDER BY ia.issue_date DESC LIMIT 1) AS summary,
|
||||
aa.facets_json AS facets_json
|
||||
FROM ranked r
|
||||
JOIN articles a ON a.id = r.article_id
|
||||
JOIN entries e ON e.id = a.best_entry_id",
|
||||
LEFT JOIN entries e ON e.id = a.best_entry_id
|
||||
LEFT JOIN article_assessments aa
|
||||
ON aa.article_id = r.article_id AND aa.stage = 'deep'
|
||||
WHERE r.event_rank = 1 AND (? OR r.label != 'cleared')
|
||||
ORDER BY r.event_at DESC, r.id DESC",
|
||||
)
|
||||
.bind(fmt_ts(since))
|
||||
.bind(include_cleared)
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
Ok(rows
|
||||
.iter()
|
||||
.map(|r| {
|
||||
let vote = if r.get::<i64, _>("vote") >= 0 {
|
||||
Vote::Up
|
||||
} else {
|
||||
Vote::Down
|
||||
};
|
||||
(r.get::<i64, _>("feed_id"), vote)
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Recent ratings with article titles, for the weekly profile rewrite (§3.6).
|
||||
pub async fn recent_ratings_detailed(&self, since: Date) -> Result<Vec<(Rating, String)>> {
|
||||
let rows = sqlx::query(
|
||||
"SELECT r.issue_date AS issue_date, r.article_id AS article_id, r.vote AS vote,
|
||||
r.rated_at AS rated_at, a.title AS title
|
||||
FROM ratings r JOIN articles a ON a.id = r.article_id
|
||||
WHERE r.issue_date >= ? ORDER BY r.rated_at DESC",
|
||||
)
|
||||
.bind(since.to_string())
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
rows.iter()
|
||||
.map(|r| {
|
||||
let rating = Rating {
|
||||
issue_date: parse_date(
|
||||
"ratings.issue_date",
|
||||
&r.get::<String, _>("issue_date"),
|
||||
)?,
|
||||
article_id: r.get::<i64, _>("article_id"),
|
||||
vote: if r.get::<i64, _>("vote") >= 0 {
|
||||
Vote::Up
|
||||
} else {
|
||||
Vote::Down
|
||||
},
|
||||
rated_at: parse_ts("ratings.rated_at", &r.get::<String, _>("rated_at"))?,
|
||||
};
|
||||
Ok((rating, r.get::<String, _>("title")))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub async fn upsert_feed_prior(&self, prior: &FeedPrior) -> Result<()> {
|
||||
sqlx::query(
|
||||
"INSERT INTO feed_priors (feed_id, upvotes, downvotes, included)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(feed_id) DO UPDATE SET
|
||||
upvotes = excluded.upvotes,
|
||||
downvotes = excluded.downvotes,
|
||||
included = excluded.included",
|
||||
)
|
||||
.bind(prior.feed_id)
|
||||
.bind(prior.upvotes)
|
||||
.bind(prior.downvotes)
|
||||
.bind(prior.included)
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn feed_priors(&self) -> Result<Vec<FeedPrior>> {
|
||||
let rows = sqlx::query("SELECT feed_id, upvotes, downvotes, included FROM feed_priors")
|
||||
.fetch_all(&self.pool)
|
||||
.await?;
|
||||
Ok(rows
|
||||
.iter()
|
||||
.map(|r| FeedPrior {
|
||||
feed_id: r.get::<i64, _>("feed_id"),
|
||||
upvotes: r.get::<i64, _>("upvotes"),
|
||||
downvotes: r.get::<i64, _>("downvotes"),
|
||||
included: r.get::<i64, _>("included"),
|
||||
})
|
||||
.collect())
|
||||
rows.iter().map(rated_article_from_row).collect()
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------
|
||||
@@ -787,6 +778,34 @@ fn article_from_row(row: &sqlx::sqlite::SqliteRow) -> Result<Article> {
|
||||
})
|
||||
}
|
||||
|
||||
fn rated_article_from_row(row: &sqlx::sqlite::SqliteRow) -> Result<RatedArticle> {
|
||||
let issue_date = row
|
||||
.get::<Option<String>, _>("issue_date")
|
||||
.map(|raw| parse_date("rating_events.issue_date", &raw))
|
||||
.transpose()?;
|
||||
let facets = row.get::<Option<String>, _>("facets_json").and_then(|raw| {
|
||||
match serde_json::from_str::<Facets>(&raw) {
|
||||
Ok(facets) => Some(facets),
|
||||
Err(error) => {
|
||||
tracing::warn!(%error, "ignoring malformed assessment facets");
|
||||
None
|
||||
}
|
||||
}
|
||||
});
|
||||
Ok(RatedArticle {
|
||||
article_id: row.get("article_id"),
|
||||
issue_date,
|
||||
title: row.get("title"),
|
||||
feed_title: row.get("feed_title"),
|
||||
summary: row.get("summary"),
|
||||
facets,
|
||||
note: row.get("note"),
|
||||
value: row.get("value"),
|
||||
label: row.get("label"),
|
||||
event_at: parse_ts("rating_events.event_at", &row.get::<String, _>("event_at"))?,
|
||||
})
|
||||
}
|
||||
|
||||
fn social_from_row(row: &sqlx::sqlite::SqliteRow) -> Result<SocialRef> {
|
||||
let raw: String = row.get("source");
|
||||
let source = SocialSource::parse(&raw).ok_or(DbError::Decode {
|
||||
@@ -991,20 +1010,187 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn ratings_upsert_is_idempotent() {
|
||||
async fn latest_explicit_rating_wins_and_clear_removes_it() {
|
||||
let (_dir, db) = temp_db().await;
|
||||
let rating = Rating {
|
||||
issue_date: "2026-08-15".parse().unwrap(),
|
||||
article_id: 1,
|
||||
vote: Vote::Up,
|
||||
rated_at: ts("2026-08-15T12:00:00Z"),
|
||||
db.upsert_entry(&sample_entry(1)).await.unwrap();
|
||||
let article = Article {
|
||||
id: 0,
|
||||
canonical_url: "https://example.com/1".into(),
|
||||
title: "Story 1".into(),
|
||||
best_entry_id: 1,
|
||||
content_html: "<p>body</p>".into(),
|
||||
word_count: 900,
|
||||
excerpt_only: false,
|
||||
image_count: 0,
|
||||
sources: vec![],
|
||||
first_seen: ts("2026-08-15T05:30:00Z"),
|
||||
url: "https://example.com/1".into(),
|
||||
author: None,
|
||||
feed_id: 7,
|
||||
feed_title: "Hacker News".into(),
|
||||
category: None,
|
||||
published_at: None,
|
||||
comments_url: None,
|
||||
image_urls: vec![],
|
||||
social: vec![],
|
||||
extract_method: ExtractMethod::Miniflux,
|
||||
};
|
||||
assert!(db.upsert_rating(&rating).await.unwrap());
|
||||
assert!(!db.upsert_rating(&rating).await.unwrap());
|
||||
let flipped = Rating {
|
||||
vote: Vote::Down,
|
||||
..rating.clone()
|
||||
let article_id = db.upsert_article(&article).await.unwrap();
|
||||
for (date, number, summary) in [
|
||||
("2026-08-14", 1, "Older summary"),
|
||||
("2026-08-15", 2, "Newest summary"),
|
||||
] {
|
||||
db.upsert_issue(
|
||||
date.parse().unwrap(),
|
||||
number,
|
||||
ts("2026-08-15T05:30:00Z"),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"INSERT INTO issue_articles
|
||||
(issue_date, article_id, section, position, is_lead, summary)
|
||||
VALUES (?, ?, 'Top Stories', 1, 0, ?)",
|
||||
)
|
||||
.bind(date)
|
||||
.bind(article_id)
|
||||
.bind(summary)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
sqlx::query(
|
||||
"INSERT INTO article_assessments
|
||||
(article_id, stage, model, prompt_version, facets_json, assessed_at)
|
||||
VALUES (?, 'deep', 'mock', 1, ?, '2026-08-15T11:00:00Z')",
|
||||
)
|
||||
.bind(article_id)
|
||||
.bind(r#"{"format":"analysis_essay","depth":"deep","evidence":null,"commerciality":null,"topic_group":"software_engineering","technicality":"advanced","locality":null,"specific_topics":null}"#)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
let event = |label: &str, value: f64, at: &str| RatingEvent {
|
||||
id: 0,
|
||||
article_id,
|
||||
issue_date: Some("2026-08-15".parse().unwrap()),
|
||||
kind: "explicit".into(),
|
||||
source: "cli".into(),
|
||||
label: label.into(),
|
||||
value,
|
||||
note: None,
|
||||
event_at: ts(at),
|
||||
};
|
||||
assert!(db.upsert_rating(&flipped).await.unwrap());
|
||||
db.append_rating_event(&event("loved", 1.0, "2026-08-15T12:00:00Z"))
|
||||
.await
|
||||
.unwrap();
|
||||
db.append_rating_event(&event("good", 0.35, "2026-08-15T13:00:00Z"))
|
||||
.await
|
||||
.unwrap();
|
||||
let mut implicit = event("read_fully", 0.5, "2026-08-15T13:30:00Z");
|
||||
implicit.kind = "implicit".into();
|
||||
implicit.source = "bookorbit".into();
|
||||
db.append_rating_event(&implicit).await.unwrap();
|
||||
let ratings = db.current_ratings(36500).await.unwrap();
|
||||
assert_eq!(ratings.len(), 1);
|
||||
assert_eq!(ratings[0].label, "good");
|
||||
assert_eq!(ratings[0].feed_title, "Hacker News");
|
||||
assert_eq!(ratings[0].summary.as_deref(), Some("Newest summary"));
|
||||
assert_eq!(
|
||||
ratings[0]
|
||||
.facets
|
||||
.as_ref()
|
||||
.and_then(|facets| facets.format.as_deref()),
|
||||
Some("analysis_essay")
|
||||
);
|
||||
|
||||
db.append_rating_event(&event("cleared", 0.0, "2026-08-15T14:00:00Z"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(db.current_ratings(36500).await.unwrap().is_empty());
|
||||
let events = db.current_ratings_including_cleared(36500).await.unwrap();
|
||||
assert_eq!(events.len(), 1);
|
||||
assert_eq!(events[0].label, "cleared");
|
||||
let sources: Vec<String> =
|
||||
sqlx::query_scalar("SELECT source FROM rating_events ORDER BY id")
|
||||
.fetch_all(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(sources, ["cli", "cli", "bookorbit", "cli"]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn curation_v2_migration_copies_ratings_and_drops_old_tables() {
|
||||
let pool = SqlitePoolOptions::new()
|
||||
.max_connections(1)
|
||||
.connect("sqlite::memory:")
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::raw_sql(include_str!("../migrations/0001_init.sql"))
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"INSERT INTO articles (id, canonical_url, title, first_seen) VALUES
|
||||
(1, 'https://example.com/loved', 'Loved', '2026-08-15T00:00:00Z'),
|
||||
(2, 'https://example.com/down', 'Down', '2026-08-15T00:00:00Z')",
|
||||
)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"INSERT INTO ratings (issue_date, article_id, vote, rated_at) VALUES
|
||||
('2026-08-15', 1, 1, '2026-08-15T12:00:00Z'),
|
||||
('2026-08-15', 2, -1, '2026-08-15T13:00:00Z')",
|
||||
)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
sqlx::raw_sql(include_str!("../migrations/0002_curation_v2.sql"))
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let rows = sqlx::query(
|
||||
"SELECT article_id, issue_date, kind, source, label, value, event_at
|
||||
FROM rating_events ORDER BY article_id",
|
||||
)
|
||||
.fetch_all(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(rows.len(), 2);
|
||||
assert_eq!(rows[0].get::<String, _>("label"), "loved");
|
||||
assert_eq!(rows[0].get::<f64, _>("value"), 1.0);
|
||||
assert_eq!(rows[1].get::<String, _>("label"), "not_for_me");
|
||||
assert_eq!(rows[1].get::<f64, _>("value"), -1.0);
|
||||
for row in &rows {
|
||||
assert_eq!(row.get::<String, _>("kind"), "explicit");
|
||||
assert_eq!(row.get::<String, _>("source"), "migration");
|
||||
assert_eq!(row.get::<String, _>("issue_date"), "2026-08-15");
|
||||
}
|
||||
assert_eq!(rows[0].get::<String, _>("event_at"), "2026-08-15T12:00:00Z");
|
||||
|
||||
let tables: Vec<String> =
|
||||
sqlx::query_scalar("SELECT name FROM sqlite_master WHERE type = 'table' ORDER BY name")
|
||||
.fetch_all(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!tables.iter().any(|table| table == "ratings"));
|
||||
assert!(!tables.iter().any(|table| table == "feed_priors"));
|
||||
assert!(tables.iter().any(|table| table == "scores"));
|
||||
for expected in [
|
||||
"rating_events",
|
||||
"article_embeddings",
|
||||
"interest_embeddings",
|
||||
"article_assessments",
|
||||
"candidate_runs",
|
||||
] {
|
||||
assert!(tables.iter().any(|table| table == expected), "{expected}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+44
-18
@@ -63,8 +63,9 @@ struct SectionPage {
|
||||
}
|
||||
|
||||
struct RatingLinks {
|
||||
up_url: String,
|
||||
down_url: String,
|
||||
loved_url: String,
|
||||
good_url: String,
|
||||
not_for_me_url: String,
|
||||
}
|
||||
|
||||
#[derive(Template)]
|
||||
@@ -344,8 +345,15 @@ pub fn render_article(
|
||||
// The X4 has no browser, so rating links are pointless there (§7).
|
||||
let rating = match (hmac_secret, edition) {
|
||||
(Some(secret), Edition::Standard) if !secret.is_empty() => Some(RatingLinks {
|
||||
up_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Up),
|
||||
down_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Down),
|
||||
loved_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Loved),
|
||||
good_url: rating_url(public_url, secret, issue.meta.date, article.id, Vote::Good),
|
||||
not_for_me_url: rating_url(
|
||||
public_url,
|
||||
secret,
|
||||
issue.meta.date,
|
||||
article.id,
|
||||
Vote::NotForMe,
|
||||
),
|
||||
}),
|
||||
_ => None,
|
||||
};
|
||||
@@ -466,16 +474,19 @@ mod tests {
|
||||
#[test]
|
||||
fn rating_token_matches_the_spec_vector() {
|
||||
let date: Date = "2026-08-15".parse().unwrap();
|
||||
assert_eq!(rating_message(date, 1234, Vote::Up), "2026-08-15/1234/up");
|
||||
// hex(hmac_sha256("test-secret", "2026-08-15/1234/up"))[..16]
|
||||
let token = rating_token("test-secret", date, 1234, Vote::Up);
|
||||
assert_eq!(
|
||||
rating_message(date, 1234, Vote::Loved),
|
||||
"2026-08-15/1234/loved"
|
||||
);
|
||||
// hex(hmac_sha256("test-secret", "2026-08-15/1234/loved"))[..16]
|
||||
let token = rating_token("test-secret", date, 1234, Vote::Loved);
|
||||
assert_eq!(token.len(), TOKEN_LEN);
|
||||
assert!(token.chars().all(|c| c.is_ascii_hexdigit()));
|
||||
|
||||
// Independently computed reference value.
|
||||
use hmac::Mac;
|
||||
let mut mac = Hmac::<Sha256>::new_from_slice(b"test-secret").unwrap();
|
||||
mac.update(b"2026-08-15/1234/up");
|
||||
mac.update(b"2026-08-15/1234/loved");
|
||||
let expected: String = hex::encode(mac.finalize().into_bytes())
|
||||
.chars()
|
||||
.take(16)
|
||||
@@ -483,9 +494,12 @@ mod tests {
|
||||
assert_eq!(token, expected);
|
||||
|
||||
// Different vote, article and secret all change the token.
|
||||
assert_ne!(token, rating_token("test-secret", date, 1234, Vote::Down));
|
||||
assert_ne!(token, rating_token("test-secret", date, 1235, Vote::Up));
|
||||
assert_ne!(token, rating_token("other-secret", date, 1234, Vote::Up));
|
||||
assert_ne!(
|
||||
token,
|
||||
rating_token("test-secret", date, 1234, Vote::NotForMe)
|
||||
);
|
||||
assert_ne!(token, rating_token("test-secret", date, 1235, Vote::Loved));
|
||||
assert_ne!(token, rating_token("other-secret", date, 1234, Vote::Loved));
|
||||
}
|
||||
|
||||
/// The EPUB signs the links and `server.rs` verifies them: one formula, or no
|
||||
@@ -494,17 +508,28 @@ mod tests {
|
||||
#[test]
|
||||
fn epub_and_server_share_one_token_vector() {
|
||||
let date: Date = "2026-08-15".parse().unwrap();
|
||||
assert_eq!(
|
||||
rating_token("test-secret", date, 42, Vote::Up),
|
||||
"3b314cf7e6d8f50f"
|
||||
);
|
||||
let token = rating_token("test-secret", date, 42, Vote::Loved);
|
||||
assert_eq!(token, "cece96767d6c5f8a");
|
||||
assert!(crate::server::verify_token(
|
||||
"test-secret",
|
||||
date,
|
||||
42,
|
||||
Vote::Loved,
|
||||
&token
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rating_url_has_the_spec_shape() {
|
||||
let date: Date = "2026-08-15".parse().unwrap();
|
||||
let url = rating_url("https://daily.hallada.net/", "s3cret", date, 99, Vote::Down);
|
||||
let token = rating_token("s3cret", date, 99, Vote::Down);
|
||||
let url = rating_url(
|
||||
"https://daily.hallada.net/",
|
||||
"s3cret",
|
||||
date,
|
||||
99,
|
||||
Vote::NotForMe,
|
||||
);
|
||||
let token = rating_token("s3cret", date, 99, Vote::NotForMe);
|
||||
assert_eq!(
|
||||
url,
|
||||
format!("https://daily.hallada.net/r/2026-08-15/99/down?t={token}")
|
||||
@@ -587,7 +612,8 @@ mod tests {
|
||||
assert!(chapter.xhtml.contains("Example Feed"));
|
||||
assert!(chapter.xhtml.contains("6 min read"));
|
||||
assert!(chapter.xhtml.contains("\u{25b2} 342 on HN"));
|
||||
assert!(chapter.xhtml.contains("/r/2026-08-15/1/up?t="));
|
||||
assert!(chapter.xhtml.contains("/r/2026-08-15/1/loved?t="));
|
||||
assert!(chapter.xhtml.contains("/r/2026-08-15/1/good?t="));
|
||||
assert!(chapter.xhtml.contains("/r/2026-08-15/1/down?t="));
|
||||
assert!(chapter.xhtml.contains("Read online"));
|
||||
assert!(chapter.xhtml.contains("href=\"disc-1001.xhtml\""));
|
||||
|
||||
@@ -24,9 +24,10 @@
|
||||
<hr class="rule"/>
|
||||
<div class="article-footer">
|
||||
{% if let Some(links) = rating %}
|
||||
<p class="rating">Was this a good pick? <a href="{{ links.up_url }}">[ 👍 Yes ]</a> · <a href="{{ links.down_url }}">[ 👎 No ]</a></p>
|
||||
{% endif %}
|
||||
<p class="rating">Was this a good pick?   <a href="{{ links.loved_url }}">[ Loved it ]</a>   <a href="{{ links.good_url }}">[ Good ]</a>   <a href="{{ links.not_for_me_url }}">[ Not for me ]</a>     <a href="{{ read_online_url }}">Read online ↗</a></p>
|
||||
{% else %}
|
||||
<p class="read-online"><a href="{{ read_online_url }}">Read online ↗</a></p>
|
||||
{% endif %}
|
||||
{% if let Some(href) = discussion_href %}
|
||||
<p class="see-discussion"><a href="{{ href }}">💬 Read the discussion</a></p>
|
||||
{% endif %}
|
||||
|
||||
@@ -224,6 +224,10 @@ hr.rule {
|
||||
font-size: 0.9em;
|
||||
}
|
||||
|
||||
.rating {
|
||||
font-size: 0.8em;
|
||||
}
|
||||
|
||||
.rating a {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
+290
-5
@@ -6,13 +6,14 @@
|
||||
use std::path::PathBuf;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use clap::{Parser, Subcommand};
|
||||
use clap::{Parser, Subcommand, ValueEnum};
|
||||
use tracing_subscriber::EnvFilter;
|
||||
|
||||
use daily_epub::config::Config;
|
||||
use daily_epub::db::Db;
|
||||
use daily_epub::pipeline::{self, GenerateOptions, GenerateOutcome};
|
||||
use daily_epub::report::RunReport;
|
||||
use daily_epub::types::{ArticleId, RatingEvent, Vote};
|
||||
use daily_epub::{curate, http, server, social};
|
||||
|
||||
/// A personalized daily newspaper, delivered as an EPUB.
|
||||
@@ -36,6 +37,9 @@ enum Command {
|
||||
/// Taste-profile maintenance.
|
||||
#[command(subcommand)]
|
||||
Profile(ProfileCommand),
|
||||
/// Inspect and edit explicit article verdicts.
|
||||
#[command(subcommand)]
|
||||
Ratings(RatingsCommand),
|
||||
/// Re-poll social scores for recent entries.
|
||||
BackfillSocial(BackfillSocialArgs),
|
||||
/// Database maintenance.
|
||||
@@ -80,6 +84,80 @@ enum ProfileCommand {
|
||||
Rebuild,
|
||||
}
|
||||
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum RatingsCommand {
|
||||
/// List current explicit ratings, newest first.
|
||||
List(RatingsListArgs),
|
||||
/// Set or correct an article's explicit rating.
|
||||
Set(RatingsSetArgs),
|
||||
/// Clear an article from the learned rating set.
|
||||
Clear(RatingsClearArgs),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, ValueEnum)]
|
||||
enum RatingListLabel {
|
||||
Loved,
|
||||
Good,
|
||||
Down,
|
||||
Cleared,
|
||||
}
|
||||
|
||||
impl RatingListLabel {
|
||||
fn event_label(self) -> &'static str {
|
||||
match self {
|
||||
Self::Loved => "loved",
|
||||
Self::Good => "good",
|
||||
Self::Down => "not_for_me",
|
||||
Self::Cleared => "cleared",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, ValueEnum)]
|
||||
enum RatingSetLabel {
|
||||
Loved,
|
||||
Good,
|
||||
Down,
|
||||
}
|
||||
|
||||
impl RatingSetLabel {
|
||||
fn vote(self) -> Vote {
|
||||
match self {
|
||||
Self::Loved => Vote::Loved,
|
||||
Self::Good => Vote::Good,
|
||||
Self::Down => Vote::NotForMe,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, clap::Args)]
|
||||
struct RatingsListArgs {
|
||||
#[arg(long, default_value_t = 90)]
|
||||
days: i64,
|
||||
#[arg(long, value_enum)]
|
||||
label: Option<RatingListLabel>,
|
||||
}
|
||||
|
||||
#[derive(Debug, clap::Args)]
|
||||
struct RatingsSetArgs {
|
||||
#[arg(long, required_unless_present = "url", conflicts_with = "url")]
|
||||
article: Option<ArticleId>,
|
||||
#[arg(long, required_unless_present = "article", conflicts_with = "article")]
|
||||
url: Option<String>,
|
||||
#[arg(long, value_enum)]
|
||||
label: RatingSetLabel,
|
||||
#[arg(long)]
|
||||
note: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, clap::Args)]
|
||||
struct RatingsClearArgs {
|
||||
#[arg(long, required_unless_present = "url", conflicts_with = "url")]
|
||||
article: Option<ArticleId>,
|
||||
#[arg(long, required_unless_present = "article", conflicts_with = "article")]
|
||||
url: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, clap::Args)]
|
||||
struct BackfillSocialArgs {
|
||||
/// How many days back to re-poll.
|
||||
@@ -114,6 +192,10 @@ async fn main() -> Result<()> {
|
||||
let db = Db::open_and_migrate(&config.database_path).await?;
|
||||
cmd_profile_rebuild(&config, &db).await?;
|
||||
}
|
||||
Command::Ratings(command) => {
|
||||
let db = Db::open_and_migrate(&config.database_path).await?;
|
||||
cmd_ratings(&config, &db, command).await?;
|
||||
}
|
||||
Command::BackfillSocial(args) => {
|
||||
let db = Db::open_and_migrate(&config.database_path).await?;
|
||||
cmd_backfill_social(&db, args.days).await?;
|
||||
@@ -236,18 +318,130 @@ fn print_lineup(issue: &daily_epub::types::Issue) {
|
||||
|
||||
async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> {
|
||||
let meter = curate::llm::UsageMeter::new(&config.deepseek, config.max_daily_usd);
|
||||
let profile = curate::profile::load_or_build(db, &config.interests_opml).await?;
|
||||
let profile = curate::profile::load_or_build(
|
||||
db,
|
||||
&config.interests_opml,
|
||||
&config.profile_path,
|
||||
config.curation.feedback.verdicts_in_prompt,
|
||||
)
|
||||
.await?;
|
||||
let llm = curate::llm::LlmClient::new(&config.deepseek, profile.text, meter)?;
|
||||
let rebuilt = curate::profile::rebuild(db, &llm, &config.interests_opml).await?;
|
||||
let feeds = curate::profile::rebuild_feed_priors(db).await?;
|
||||
let rebuilt = curate::profile::rebuild(
|
||||
db,
|
||||
&llm,
|
||||
&config.interests_opml,
|
||||
&config.profile_path,
|
||||
config.curation.feedback.verdicts_in_prompt,
|
||||
)
|
||||
.await?;
|
||||
println!(
|
||||
"taste profile rebuilt (version {}, {} chars); {feeds} feed priors refreshed",
|
||||
"taste profile rebuilt (version {}, {} chars)",
|
||||
rebuilt.version,
|
||||
rebuilt.text.len()
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn resolve_rating_article(
|
||||
db: &Db,
|
||||
article: Option<ArticleId>,
|
||||
url: Option<&str>,
|
||||
) -> Result<ArticleId> {
|
||||
let article_id = match (article, url) {
|
||||
(Some(article_id), None) => article_id,
|
||||
(None, Some(url)) => {
|
||||
let canonical = daily_epub::dedupe::canonical_url(url)
|
||||
.with_context(|| format!("invalid article URL {url:?}"))?;
|
||||
db.article_id_for_url(&canonical)
|
||||
.await?
|
||||
.with_context(|| format!("no article found for {canonical}"))?
|
||||
}
|
||||
_ => anyhow::bail!("provide exactly one of --article or --url"),
|
||||
};
|
||||
if db.get_article(article_id).await?.is_none() {
|
||||
anyhow::bail!("article {article_id} was not found");
|
||||
}
|
||||
Ok(article_id)
|
||||
}
|
||||
|
||||
async fn append_cli_event(
|
||||
config: &Config,
|
||||
db: &Db,
|
||||
article_id: ArticleId,
|
||||
vote: Option<Vote>,
|
||||
note: Option<String>,
|
||||
) -> Result<i64> {
|
||||
let (label, value) = match vote {
|
||||
Some(Vote::Loved) => ("loved", Vote::Loved.value(&config.curation.feedback)),
|
||||
Some(Vote::Good) => ("good", Vote::Good.value(&config.curation.feedback)),
|
||||
Some(Vote::NotForMe) => (
|
||||
"not_for_me",
|
||||
Vote::NotForMe.value(&config.curation.feedback),
|
||||
),
|
||||
None => ("cleared", 0.0),
|
||||
};
|
||||
let event = RatingEvent {
|
||||
id: 0,
|
||||
article_id,
|
||||
issue_date: db.latest_issue_date_for_article(article_id).await?,
|
||||
kind: "explicit".into(),
|
||||
source: "cli".into(),
|
||||
label: label.into(),
|
||||
value,
|
||||
note,
|
||||
event_at: jiff::Timestamp::now(),
|
||||
};
|
||||
Ok(db.append_rating_event(&event).await?)
|
||||
}
|
||||
|
||||
async fn cmd_ratings(config: &Config, db: &Db, command: RatingsCommand) -> Result<()> {
|
||||
match command {
|
||||
RatingsCommand::List(args) => {
|
||||
let ratings = db.current_ratings_including_cleared(args.days).await?;
|
||||
let mut shown = 0usize;
|
||||
for rating in ratings {
|
||||
if args
|
||||
.label
|
||||
.is_some_and(|label| rating.label != label.event_label())
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let note = rating
|
||||
.note
|
||||
.as_deref()
|
||||
.map(|note| format!(" · note: {note}"))
|
||||
.unwrap_or_default();
|
||||
println!(
|
||||
"{} · article {} · {} · {} — {}{}",
|
||||
rating.event_at,
|
||||
rating.article_id,
|
||||
rating.label,
|
||||
rating.title,
|
||||
rating.feed_title,
|
||||
note
|
||||
);
|
||||
shown += 1;
|
||||
}
|
||||
println!("{shown} current rating(s)");
|
||||
}
|
||||
RatingsCommand::Set(args) => {
|
||||
let article_id = resolve_rating_article(db, args.article, args.url.as_deref()).await?;
|
||||
let vote = args.label.vote();
|
||||
let event_id = append_cli_event(config, db, article_id, Some(vote), args.note).await?;
|
||||
println!(
|
||||
"recorded {} for article {article_id} (event {event_id})",
|
||||
vote.as_str()
|
||||
);
|
||||
}
|
||||
RatingsCommand::Clear(args) => {
|
||||
let article_id = resolve_rating_article(db, args.article, args.url.as_deref()).await?;
|
||||
let event_id = append_cli_event(config, db, article_id, None, None).await?;
|
||||
println!("cleared article {article_id} (event {event_id})");
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn cmd_backfill_social(db: &Db, days: u32) -> Result<()> {
|
||||
let http = http::build_client(http::DEFAULT_TIMEOUT)?;
|
||||
let enricher = social::SocialEnricher::new(http, db.clone());
|
||||
@@ -309,6 +503,33 @@ mod tests {
|
||||
.command,
|
||||
Command::Profile(ProfileCommand::Rebuild)
|
||||
));
|
||||
assert!(matches!(
|
||||
Cli::try_parse_from([
|
||||
"daily-epub",
|
||||
"ratings",
|
||||
"set",
|
||||
"--article",
|
||||
"42",
|
||||
"--label",
|
||||
"good"
|
||||
])
|
||||
.unwrap()
|
||||
.command,
|
||||
Command::Ratings(RatingsCommand::Set(_))
|
||||
));
|
||||
assert!(matches!(
|
||||
Cli::try_parse_from([
|
||||
"daily-epub",
|
||||
"ratings",
|
||||
"clear",
|
||||
"--url",
|
||||
"https://example.com"
|
||||
])
|
||||
.unwrap()
|
||||
.command,
|
||||
Command::Ratings(RatingsCommand::Clear(_))
|
||||
));
|
||||
|
||||
assert!(matches!(
|
||||
Cli::try_parse_from(["daily-epub", "backfill-social", "--days", "14"])
|
||||
.unwrap()
|
||||
@@ -325,4 +546,68 @@ mod tests {
|
||||
let cli = Cli::try_parse_from(["daily-epub", "--config", "/tmp/x.toml", "serve"]).unwrap();
|
||||
assert_eq!(cli.config, Some(PathBuf::from("/tmp/x.toml")));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cli_set_and_clear_append_cli_events_with_latest_issue_date() {
|
||||
use sqlx::Row as _;
|
||||
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let db = Db::open_and_migrate(&dir.path().join("ratings.db"))
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"INSERT INTO articles (id, canonical_url, title, first_seen) VALUES
|
||||
(42, 'https://example.com/article', 'Article', '2026-08-15T00:00:00Z')",
|
||||
)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"INSERT INTO issues (date, issue_number, generated_at) VALUES
|
||||
('2026-08-14', 1, '2026-08-14T12:00:00Z'),
|
||||
('2026-08-15', 2, '2026-08-15T12:00:00Z')",
|
||||
)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"INSERT INTO issue_articles (issue_date, article_id, section) VALUES
|
||||
('2026-08-14', 42, 'Top Stories'),
|
||||
('2026-08-15', 42, 'Top Stories')",
|
||||
)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let config = Config::default();
|
||||
append_cli_event(
|
||||
&config,
|
||||
&db,
|
||||
42,
|
||||
Some(Vote::Good),
|
||||
Some("useful note".into()),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
append_cli_event(&config, &db, 42, None, None)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let rows = sqlx::query(
|
||||
"SELECT issue_date, source, label, value, note FROM rating_events ORDER BY id",
|
||||
)
|
||||
.fetch_all(db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(rows.len(), 2);
|
||||
assert_eq!(rows[0].get::<String, _>("source"), "cli");
|
||||
assert_eq!(rows[0].get::<String, _>("issue_date"), "2026-08-15");
|
||||
assert_eq!(rows[0].get::<String, _>("label"), "good");
|
||||
assert_eq!(rows[0].get::<f64, _>("value"), 0.35);
|
||||
assert_eq!(rows[0].get::<String, _>("note"), "useful note");
|
||||
assert_eq!(rows[1].get::<String, _>("source"), "cli");
|
||||
assert_eq!(rows[1].get::<String, _>("label"), "cleared");
|
||||
assert_eq!(rows[1].get::<f64, _>("value"), 0.0);
|
||||
assert!(db.current_ratings(36500).await.unwrap().is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
+22
-11
@@ -367,12 +367,8 @@ async fn run_stages(
|
||||
report.counts.social_hits = enricher.enrich_all(&mut articles).await as i64;
|
||||
report.timings.record("social", elapsed_ms(stage));
|
||||
|
||||
// --- Stage 6: feed priors, then the heuristic pre-filter (§3.5, §3.9) ---
|
||||
// --- Stage 6: heuristic pre-filter (§3.5) ---
|
||||
let stage = Timestamp::now();
|
||||
if let Err(e) = profile::rebuild_feed_priors(db).await {
|
||||
report.warn(format!("could not rebuild feed priors: {e:#}"));
|
||||
}
|
||||
|
||||
let meter = UsageMeter::new(&config.deepseek, config.max_daily_usd);
|
||||
// `max_daily_usd` is a ceiling for the *day*, not for one invocation, so a
|
||||
// re-run inherits what earlier runs for this date already spent (§3.6).
|
||||
@@ -591,11 +587,14 @@ async fn build_llm(
|
||||
meter: &UsageMeter,
|
||||
report: &mut RunReport,
|
||||
) -> Option<LlmClient> {
|
||||
if ctx.skip_llm {
|
||||
tracing::info!("--skip-llm: no DeepSeek call will be made");
|
||||
return None;
|
||||
}
|
||||
let profile = match profile::load_or_build(ctx.db, &ctx.config.interests_opml).await {
|
||||
let profile = match profile::load_or_build(
|
||||
ctx.db,
|
||||
&ctx.config.interests_opml,
|
||||
&ctx.config.profile_path,
|
||||
ctx.config.curation.feedback.verdicts_in_prompt,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(profile) => profile,
|
||||
Err(e) => {
|
||||
report.warn(format!(
|
||||
@@ -604,6 +603,10 @@ async fn build_llm(
|
||||
return None;
|
||||
}
|
||||
};
|
||||
if ctx.skip_llm {
|
||||
tracing::info!("--skip-llm: profile rebuilt; no DeepSeek call will be made");
|
||||
return None;
|
||||
}
|
||||
let client = match LlmClient::new(&ctx.config.deepseek, profile.text, meter.clone()) {
|
||||
Ok(client) => client,
|
||||
Err(e) => {
|
||||
@@ -616,7 +619,15 @@ async fn build_llm(
|
||||
|
||||
// Weekly rewrite of the "learned adjustments" section (§3.6c). It changes the
|
||||
// system prompt, so the client is rebuilt around the new profile.
|
||||
match profile::weekly_rebuild_if_due(ctx.db, &client, &ctx.config.interests_opml).await {
|
||||
match profile::weekly_rebuild_if_due(
|
||||
ctx.db,
|
||||
&client,
|
||||
&ctx.config.interests_opml,
|
||||
&ctx.config.profile_path,
|
||||
ctx.config.curation.feedback.verdicts_in_prompt,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(Some(rebuilt)) => {
|
||||
tracing::info!(version = rebuilt.version, "taste profile rebuilt");
|
||||
match LlmClient::new(&ctx.config.deepseek, rebuilt.text, meter.clone()) {
|
||||
|
||||
+149
-88
@@ -6,7 +6,7 @@
|
||||
//! Routes (§3.12):
|
||||
//! | route | behaviour |
|
||||
//! |---|---|
|
||||
//! | `GET /r/{date}/{article_id}/{vote}?t=` | verify HMAC, upsert rating, rebuild feed priors |
|
||||
//! | `GET /r/{date}/{article_id}/{vote}?t=` | verify HMAC and append a rating event |
|
||||
//! | `GET /opds/daily.xml` (also `/opds`, `/opds/`) | OPDS 1.2 acquisition feed over `publish.epub_dir` |
|
||||
//! | `GET /files/epub/{name}` | EPUB download — what the feed's acquisition links point at |
|
||||
//! | `GET /files/xtc/{name}` | XTC artifact download, unlisted (no path traversal) |
|
||||
@@ -15,7 +15,7 @@
|
||||
//!
|
||||
//! `/opds/*` and `/files/*` sit behind optional Basic auth (`server.basic_auth_*`).
|
||||
//!
|
||||
//! The EPUB article footer (§3.10) mints its 👍/👎 links with the very same
|
||||
//! The EPUB article footer (§3.10) mints its three verdict links with the very same
|
||||
//! [`rating_url`] this module verifies with — both re-export [`crate::auth`],
|
||||
//! which pins the shared test vector (`secret = "test-secret"`, `2026-08-15`,
|
||||
//! article `42`, `up` → `3b314cf7e6d8f50f`). An issue generated while
|
||||
@@ -38,7 +38,7 @@ use tower_http::trace::TraceLayer;
|
||||
|
||||
use crate::config::Config;
|
||||
use crate::db::Db;
|
||||
use crate::types::{ArticleId, Rating, Vote};
|
||||
use crate::types::{ArticleId, RatingEvent, Vote};
|
||||
|
||||
/// Characters of the hex HMAC kept in rating links (§3.9).
|
||||
pub const TOKEN_LEN: usize = crate::auth::TOKEN_LEN;
|
||||
@@ -244,46 +244,47 @@ async fn handle_rating(
|
||||
}
|
||||
};
|
||||
|
||||
let rating = Rating {
|
||||
issue_date: date,
|
||||
let label = match vote {
|
||||
Vote::Loved => "loved",
|
||||
Vote::Good => "good",
|
||||
Vote::NotForMe => "not_for_me",
|
||||
};
|
||||
let event = RatingEvent {
|
||||
id: 0,
|
||||
issue_date: Some(date),
|
||||
article_id,
|
||||
vote,
|
||||
rated_at: Timestamp::now(),
|
||||
kind: "explicit".into(),
|
||||
source: "epub".into(),
|
||||
label: label.into(),
|
||||
value: vote.value(&state.config.curation.feedback),
|
||||
note: None,
|
||||
event_at: Timestamp::now(),
|
||||
};
|
||||
let changed = match state.db.upsert_rating(&rating).await {
|
||||
Ok(changed) => changed,
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e, article_id, "recording the rating failed");
|
||||
if let Err(error) = state.db.append_rating_event(&event).await {
|
||||
tracing::error!(%error, article_id, "recording the rating failed");
|
||||
return page(StatusCode::INTERNAL_SERVER_ERROR, "Database error.", None);
|
||||
}
|
||||
};
|
||||
if changed && let Err(e) = crate::curate::profile::rebuild_feed_priors(&state.db).await {
|
||||
// The vote is stored; a stale prior only affects the next run's ranking.
|
||||
tracing::error!(error = %e, "refreshing feed priors failed");
|
||||
}
|
||||
tracing::info!(
|
||||
%date,
|
||||
article_id,
|
||||
feed_id = article.feed_id,
|
||||
vote = vote.as_str(),
|
||||
changed,
|
||||
title = %article.title,
|
||||
"recorded rating"
|
||||
"recorded rating event"
|
||||
);
|
||||
|
||||
let glyph = match vote {
|
||||
Vote::Up => "👍",
|
||||
Vote::Down => "👎",
|
||||
let message = match vote {
|
||||
Vote::Loved => "Recorded: Loved it — thanks.",
|
||||
Vote::Good => "Recorded: Good — thanks.",
|
||||
Vote::NotForMe => "Recorded: Not for me — thanks.",
|
||||
};
|
||||
let message = if changed {
|
||||
format!("Recorded {glyph} — thanks!")
|
||||
} else {
|
||||
format!("Already recorded {glyph} — thanks!")
|
||||
};
|
||||
page(
|
||||
confirmation_page(
|
||||
StatusCode::OK,
|
||||
&message,
|
||||
Some(&format!("{date} · article {article_id}")),
|
||||
message,
|
||||
&state.config,
|
||||
date,
|
||||
article_id,
|
||||
vote,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -457,7 +458,7 @@ fn check_basic_auth(config: &Config, headers: &HeaderMap) -> Option<Response> {
|
||||
/// A self-contained response page — no external CSS, well under 1 KB, legible on
|
||||
/// a 6" e-ink browser (§3.9).
|
||||
fn page(status: StatusCode, message: &str, note: Option<&str>) -> Response {
|
||||
let body = page_html(message, note);
|
||||
let body = page_html(message, note, None);
|
||||
(
|
||||
status,
|
||||
[
|
||||
@@ -469,8 +470,52 @@ fn page(status: StatusCode, message: &str, note: Option<&str>) -> Response {
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// The page markup itself: no stylesheet, no script, no images (§3.9).
|
||||
fn page_html(message: &str, note: Option<&str>) -> String {
|
||||
fn confirmation_page(
|
||||
status: StatusCode,
|
||||
message: &str,
|
||||
config: &Config,
|
||||
date: Date,
|
||||
article_id: ArticleId,
|
||||
selected: Vote,
|
||||
) -> Response {
|
||||
let Some(secret) = config.server.hmac_secret.as_deref() else {
|
||||
return page(status, message, None);
|
||||
};
|
||||
let choices = [
|
||||
(Vote::Loved, "Loved it"),
|
||||
(Vote::Good, "Good"),
|
||||
(Vote::NotForMe, "Not for me"),
|
||||
]
|
||||
.into_iter()
|
||||
.filter(|(vote, _)| *vote != selected)
|
||||
.map(|(vote, label)| {
|
||||
let url = rating_url(&config.server.public_url, secret, date, article_id, vote);
|
||||
format!(
|
||||
"<a href=\"{}\">[ {} ]</a>",
|
||||
escape_attr(&url),
|
||||
escape(label)
|
||||
)
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join(" ");
|
||||
let body = page_html(
|
||||
message,
|
||||
None,
|
||||
Some(&format!("<p><small>Change it: {choices}</small></p>")),
|
||||
);
|
||||
(
|
||||
status,
|
||||
[
|
||||
(header::CONTENT_TYPE, "text/html; charset=utf-8"),
|
||||
(header::CACHE_CONTROL, "no-store"),
|
||||
],
|
||||
body,
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// The page markup itself: no external stylesheet, script, or images (§6.1).
|
||||
fn page_html(message: &str, note: Option<&str>, extra_html: Option<&str>) -> String {
|
||||
let note = note
|
||||
.map(|n| format!("<p><small>{}</small></p>", escape(n)))
|
||||
.unwrap_or_default();
|
||||
@@ -478,11 +523,12 @@ fn page_html(message: &str, note: Option<&str>) -> String {
|
||||
"<!doctype html><html lang=\"en\"><meta charset=\"utf-8\">\
|
||||
<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\
|
||||
<title>The Daily EPUB</title>\
|
||||
<style>body{{margin:3em auto;max-width:16em;padding:0 1em;text-align:center;\
|
||||
<style>body{{margin:3em auto;max-width:18em;padding:0 1em;text-align:center;\
|
||||
font:1.3em/1.5 Georgia,serif}}small{{font-size:.65em}}</style>\
|
||||
<p>{}</p>{}",
|
||||
<p>{}</p>{}{}",
|
||||
escape(message),
|
||||
note
|
||||
note,
|
||||
extra_html.unwrap_or_default()
|
||||
)
|
||||
}
|
||||
|
||||
@@ -492,6 +538,10 @@ fn escape(s: &str) -> String {
|
||||
.replace('>', ">")
|
||||
}
|
||||
|
||||
fn escape_attr(s: &str) -> String {
|
||||
escape(s).replace('\"', """)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -512,29 +562,33 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn token_matches_the_shared_test_vector() {
|
||||
assert_eq!(
|
||||
rating_token(VECTOR_SECRET, date(), 42, Vote::Up),
|
||||
VECTOR_TOKEN_UP
|
||||
);
|
||||
assert_eq!(rating_token(VECTOR_SECRET, date(), 42, Vote::Up).len(), 16);
|
||||
let loved = rating_token(VECTOR_SECRET, date(), 42, Vote::Loved);
|
||||
assert_eq!(loved, "cece96767d6c5f8a");
|
||||
assert_eq!(loved.len(), 16);
|
||||
// Down differs from up, and both verify.
|
||||
let down = rating_token(VECTOR_SECRET, date(), 42, Vote::Down);
|
||||
let down = rating_token(VECTOR_SECRET, date(), 42, Vote::NotForMe);
|
||||
assert_ne!(down, VECTOR_TOKEN_UP);
|
||||
assert!(verify_token(
|
||||
VECTOR_SECRET,
|
||||
date(),
|
||||
42,
|
||||
Vote::Up,
|
||||
Vote::Loved,
|
||||
VECTOR_TOKEN_UP
|
||||
));
|
||||
assert!(verify_token(VECTOR_SECRET, date(), 42, Vote::Down, &down));
|
||||
assert!(verify_token(
|
||||
VECTOR_SECRET,
|
||||
date(),
|
||||
42,
|
||||
Vote::NotForMe,
|
||||
&down
|
||||
));
|
||||
}
|
||||
|
||||
/// The links the EPUB footer embeds must verify here — this is the whole
|
||||
/// feedback loop in one assertion (§3.9).
|
||||
#[test]
|
||||
fn epub_footer_links_verify_against_this_server() {
|
||||
for (id, vote) in [(42, Vote::Up), (1234, Vote::Down)] {
|
||||
for (id, vote) in [(42, Vote::Loved), (1234, Vote::NotForMe)] {
|
||||
let from_epub = crate::epub::build::rating_url(
|
||||
"https://daily.hallada.net",
|
||||
VECTOR_SECRET,
|
||||
@@ -556,23 +610,23 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn token_verification_rejects_tampering() {
|
||||
let t = rating_token(VECTOR_SECRET, date(), 42, Vote::Up);
|
||||
assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::Down, &t));
|
||||
assert!(!verify_token(VECTOR_SECRET, date(), 43, Vote::Up, &t));
|
||||
assert!(!verify_token("other-secret", date(), 42, Vote::Up, &t));
|
||||
let t = rating_token(VECTOR_SECRET, date(), 42, Vote::Loved);
|
||||
assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::NotForMe, &t));
|
||||
assert!(!verify_token(VECTOR_SECRET, date(), 43, Vote::Loved, &t));
|
||||
assert!(!verify_token("other-secret", date(), 42, Vote::Loved, &t));
|
||||
assert!(!verify_token(
|
||||
VECTOR_SECRET,
|
||||
"2026-08-16".parse().unwrap(),
|
||||
42,
|
||||
Vote::Up,
|
||||
Vote::Loved,
|
||||
&t
|
||||
));
|
||||
assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::Up, ""));
|
||||
assert!(!verify_token(VECTOR_SECRET, date(), 42, Vote::Loved, ""));
|
||||
assert!(!verify_token(
|
||||
VECTOR_SECRET,
|
||||
date(),
|
||||
42,
|
||||
Vote::Up,
|
||||
Vote::Loved,
|
||||
&format!("{t}00")
|
||||
));
|
||||
}
|
||||
@@ -585,9 +639,12 @@ mod tests {
|
||||
VECTOR_SECRET,
|
||||
date(),
|
||||
42,
|
||||
Vote::Up
|
||||
Vote::Loved
|
||||
),
|
||||
format!("https://daily.hallada.net/r/2026-08-15/42/up?t={VECTOR_TOKEN_UP}")
|
||||
format!(
|
||||
"https://daily.hallada.net/r/2026-08-15/42/loved?t={}",
|
||||
rating_token(VECTOR_SECRET, date(), 42, Vote::Loved)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -626,16 +683,20 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn the_confirmation_page_is_tiny_and_self_contained() {
|
||||
let html = page_html("Recorded 👍 — thanks!", Some("2026-08-15 · article 42"));
|
||||
let html = page_html(
|
||||
"Recorded: Loved it — thanks.",
|
||||
Some("2026-08-15 · article 42"),
|
||||
None,
|
||||
);
|
||||
assert!(html.len() < 1024, "page is {} bytes", html.len());
|
||||
assert!(!html.contains("<link"), "no external stylesheet");
|
||||
assert!(!html.contains("<script"), "no script");
|
||||
assert!(html.contains("Recorded 👍"));
|
||||
assert!(html.contains("Recorded: Loved it"));
|
||||
assert_eq!(
|
||||
page(StatusCode::FORBIDDEN, "Invalid link.", None).status(),
|
||||
StatusCode::FORBIDDEN
|
||||
);
|
||||
assert!(page_html("<b>x</b>", None).contains("<b>"));
|
||||
assert!(page_html("<b>x</b>", None, None).contains("<b>"));
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------
|
||||
@@ -810,47 +871,43 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn rating_happy_path_is_idempotent_and_updates_priors() {
|
||||
async fn rating_taps_append_events_and_latest_correction_wins() {
|
||||
let server = TestServer::start(false).await;
|
||||
let id = server.seed_article().await;
|
||||
let url = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::Up);
|
||||
let loved = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::Loved);
|
||||
|
||||
let res = client().get(&url).send().await.unwrap();
|
||||
assert_eq!(res.status(), 200);
|
||||
let body = res.text().await.unwrap();
|
||||
assert!(body.contains("Recorded"), "{body}");
|
||||
assert!(!body.contains("Already"), "{body}");
|
||||
let response = client().get(&loved).send().await.unwrap();
|
||||
assert_eq!(response.status(), 200);
|
||||
let body = response.text().await.unwrap();
|
||||
assert!(body.contains("Recorded: Loved it — thanks."), "{body}");
|
||||
assert!(body.contains("[ Good ]"), "{body}");
|
||||
assert!(body.contains("[ Not for me ]"), "{body}");
|
||||
assert!(
|
||||
body.len() < 1024,
|
||||
body.len() < 2048,
|
||||
"confirmation page is {} bytes",
|
||||
body.len()
|
||||
);
|
||||
|
||||
// Same tap again: still 200, but reported as already recorded.
|
||||
let body = client()
|
||||
.get(&url)
|
||||
.send()
|
||||
.await
|
||||
.unwrap()
|
||||
.text()
|
||||
// Every tap is history, even a repeated one.
|
||||
assert_eq!(client().get(&loved).send().await.unwrap().status(), 200);
|
||||
let count: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM rating_events")
|
||||
.fetch_one(server.db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(body.contains("Already recorded"), "{body}");
|
||||
assert_eq!(count, 2);
|
||||
|
||||
let ratings = server.db.ratings_with_feed().await.unwrap();
|
||||
assert_eq!(ratings, vec![(7, Vote::Up)]);
|
||||
let priors = server.db.feed_priors().await.unwrap();
|
||||
assert_eq!(priors.len(), 1);
|
||||
assert_eq!(
|
||||
(priors[0].feed_id, priors[0].upvotes, priors[0].downvotes),
|
||||
(7, 1, 0)
|
||||
);
|
||||
|
||||
// Flipping the vote rewrites the prior rather than double-counting.
|
||||
let down = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::Down);
|
||||
// A correction appends and becomes the current verdict.
|
||||
let down = rating_url(&server.base, VECTOR_SECRET, date(), id, Vote::NotForMe);
|
||||
assert_eq!(client().get(&down).send().await.unwrap().status(), 200);
|
||||
let priors = server.db.feed_priors().await.unwrap();
|
||||
assert_eq!((priors[0].upvotes, priors[0].downvotes), (0, 1));
|
||||
let current = server.db.current_ratings(36500).await.unwrap();
|
||||
assert_eq!(current.len(), 1);
|
||||
assert_eq!(current[0].label, "not_for_me");
|
||||
let sources: Vec<String> =
|
||||
sqlx::query_scalar("SELECT source FROM rating_events ORDER BY id")
|
||||
.fetch_all(server.db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(sources, ["epub", "epub", "epub"]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -864,17 +921,21 @@ mod tests {
|
||||
assert_eq!(client().get(&missing).send().await.unwrap().status(), 403);
|
||||
|
||||
// A valid token for an article that does not exist.
|
||||
let unknown = rating_url(&server.base, VECTOR_SECRET, date(), 9999, Vote::Up);
|
||||
let unknown = rating_url(&server.base, VECTOR_SECRET, date(), 9999, Vote::Loved);
|
||||
assert_eq!(client().get(&unknown).send().await.unwrap().status(), 404);
|
||||
|
||||
// Malformed date / vote.
|
||||
let token = rating_token(VECTOR_SECRET, date(), id, Vote::Up);
|
||||
let token = rating_token(VECTOR_SECRET, date(), id, Vote::Loved);
|
||||
let bad_date = format!("{}/r/not-a-date/{id}/up?t={token}", server.base);
|
||||
assert_eq!(client().get(&bad_date).send().await.unwrap().status(), 400);
|
||||
let bad_vote = format!("{}/r/2026-08-15/{id}/sideways?t={token}", server.base);
|
||||
assert_eq!(client().get(&bad_vote).send().await.unwrap().status(), 400);
|
||||
|
||||
assert!(server.db.ratings_with_feed().await.unwrap().is_empty());
|
||||
let count: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM rating_events")
|
||||
.fetch_one(server.db.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(count, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
+70
-51
@@ -269,8 +269,6 @@ pub struct ScoredArticle {
|
||||
pub prefilter_score: f64,
|
||||
/// Cached [`composite_social_score`] for the article.
|
||||
pub social_score: f64,
|
||||
/// Beta-smoothed per-feed upvote rate applied by the pre-filter (§3.9).
|
||||
pub feed_prior: f64,
|
||||
/// `None` until stage A has run (or when `--skip-llm`).
|
||||
pub llm: Option<LlmScore>,
|
||||
/// From `curation.always_include_feeds`: may be scored but never dropped (§3.5).
|
||||
@@ -278,10 +276,10 @@ pub struct ScoredArticle {
|
||||
}
|
||||
|
||||
impl ScoredArticle {
|
||||
/// Ranking key for stage B: LLM score weighted with social proof and priors (§3.6).
|
||||
/// Ranking key for stage B: LLM score weighted with social proof (§3.6).
|
||||
pub fn combined_score(&self) -> f64 {
|
||||
let llm = self.llm.as_ref().map(|l| l.score).unwrap_or(0.0);
|
||||
llm * 10.0 + self.social_score * 4.0 + self.feed_prior * 10.0 + self.prefilter_score * 0.1
|
||||
llm * 10.0 + self.social_score * 4.0 + self.prefilter_score * 0.1
|
||||
}
|
||||
}
|
||||
|
||||
@@ -565,62 +563,85 @@ pub struct Artifact {
|
||||
// Feedback (§3.9)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// 👍 / 👎 stored as `+1` / `-1` in `ratings.vote` (§3.9).
|
||||
/// Explicit reader verdict embedded in rating-link URLs (§6.1).
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum Vote {
|
||||
Up,
|
||||
Down,
|
||||
#[serde(rename = "loved", alias = "up")]
|
||||
Loved,
|
||||
#[serde(rename = "good")]
|
||||
Good,
|
||||
#[serde(rename = "down")]
|
||||
NotForMe,
|
||||
}
|
||||
|
||||
impl Vote {
|
||||
pub fn as_i64(self) -> i64 {
|
||||
match self {
|
||||
Vote::Up => 1,
|
||||
Vote::Down => -1,
|
||||
}
|
||||
}
|
||||
|
||||
/// Path segment used in rating links: `up` / `down` (§3.9).
|
||||
/// Stable path segment used in rating links (§6.1).
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Vote::Up => "up",
|
||||
Vote::Down => "down",
|
||||
Vote::Loved => "loved",
|
||||
Vote::Good => "good",
|
||||
Vote::NotForMe => "down",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse(s: &str) -> Option<Self> {
|
||||
match s {
|
||||
"up" => Some(Vote::Up),
|
||||
"down" => Some(Vote::Down),
|
||||
"loved" | "up" => Some(Vote::Loved),
|
||||
"good" => Some(Vote::Good),
|
||||
"down" => Some(Vote::NotForMe),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn value(self, cfg: &crate::config::FeedbackConfig) -> f64 {
|
||||
match self {
|
||||
Vote::Loved => cfg.loved_value,
|
||||
Vote::Good => cfg.good_value,
|
||||
Vote::NotForMe => cfg.not_for_me_value,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A recorded reader vote (`ratings` table, §3.9).
|
||||
/// One append-only feedback event (`rating_events`, §6.2).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Rating {
|
||||
pub issue_date: Date,
|
||||
pub struct RatingEvent {
|
||||
pub id: i64,
|
||||
pub article_id: ArticleId,
|
||||
pub vote: Vote,
|
||||
pub rated_at: Timestamp,
|
||||
pub issue_date: Option<Date>,
|
||||
pub kind: String,
|
||||
pub source: String,
|
||||
pub label: String,
|
||||
pub value: f64,
|
||||
pub note: Option<String>,
|
||||
pub event_at: Timestamp,
|
||||
}
|
||||
|
||||
/// Beta-smoothed per-feed upvote rate used by the pre-filter (`feed_priors`, §3.9).
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct FeedPrior {
|
||||
pub feed_id: FeedId,
|
||||
pub upvotes: i64,
|
||||
pub downvotes: i64,
|
||||
pub included: i64,
|
||||
/// Descriptive deep-assessment facets (§12.1), populated beginning in step 5.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Facets {
|
||||
pub format: Option<String>,
|
||||
pub depth: Option<String>,
|
||||
pub evidence: Option<String>,
|
||||
pub commerciality: Option<String>,
|
||||
pub topic_group: Option<String>,
|
||||
pub technicality: Option<String>,
|
||||
pub locality: Option<String>,
|
||||
pub specific_topics: Option<Vec<String>>,
|
||||
}
|
||||
|
||||
impl FeedPrior {
|
||||
/// `(up + 1) / (up + down + 2)` — 0.5 with no evidence (§3.9).
|
||||
pub fn rate(&self) -> f64 {
|
||||
(self.upvotes + 1) as f64 / (self.upvotes + self.downvotes + 2) as f64
|
||||
}
|
||||
/// The current explicit verdict for an article, enriched for prompts (§6.2).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct RatedArticle {
|
||||
pub article_id: ArticleId,
|
||||
pub issue_date: Option<Date>,
|
||||
pub title: String,
|
||||
pub feed_title: String,
|
||||
pub summary: Option<String>,
|
||||
pub facets: Option<Facets>,
|
||||
pub note: Option<String>,
|
||||
pub value: f64,
|
||||
pub label: String,
|
||||
pub event_at: Timestamp,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -684,18 +705,6 @@ mod tests {
|
||||
assert_eq!(composite_social_score(&[]), 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn feed_prior_is_beta_smoothed() {
|
||||
assert_eq!(FeedPrior::default().rate(), 0.5);
|
||||
let p = FeedPrior {
|
||||
feed_id: 1,
|
||||
upvotes: 3,
|
||||
downvotes: 1,
|
||||
included: 4,
|
||||
};
|
||||
assert!((p.rate() - 4.0 / 6.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stats_line_and_reading_time() {
|
||||
assert_eq!(reading_minutes(0), 1);
|
||||
@@ -730,8 +739,18 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn vote_and_social_source_round_trip() {
|
||||
assert_eq!(Vote::parse("up"), Some(Vote::Up));
|
||||
assert_eq!(Vote::Down.as_i64(), -1);
|
||||
let feedback = crate::config::FeedbackConfig::default();
|
||||
assert_eq!(Vote::parse("loved"), Some(Vote::Loved));
|
||||
assert_eq!(Vote::parse("up"), Some(Vote::Loved));
|
||||
assert_eq!(Vote::parse("good"), Some(Vote::Good));
|
||||
assert_eq!(Vote::parse("down"), Some(Vote::NotForMe));
|
||||
assert_eq!(Vote::Loved.as_str(), "loved");
|
||||
assert_eq!(Vote::Good.as_str(), "good");
|
||||
assert_eq!(Vote::NotForMe.as_str(), "down");
|
||||
assert_eq!(Vote::Loved.value(&feedback), 1.0);
|
||||
assert_eq!(Vote::Good.value(&feedback), 0.35);
|
||||
assert_eq!(Vote::NotForMe.value(&feedback), -1.0);
|
||||
assert_eq!(serde_json::to_string(&Vote::NotForMe).unwrap(), "\"down\"");
|
||||
assert_eq!(
|
||||
SocialSource::parse("lobsters"),
|
||||
Some(SocialSource::Lobsters)
|
||||
|
||||
@@ -283,7 +283,7 @@ async fn assemble_build_publish(
|
||||
)
|
||||
.expect("render chapters");
|
||||
let first = issue.lineup.picks[0].article.id;
|
||||
let expected = auth::rating_url(&cfg.server.public_url, SECRET, date(), first, Vote::Up);
|
||||
let expected = auth::rating_url(&cfg.server.public_url, SECRET, date(), first, Vote::Loved);
|
||||
let chapter = chapters
|
||||
.iter()
|
||||
.find(|c| c.id == format!("art-{}", issue.lineup.picks[0].article.best_entry_id))
|
||||
@@ -298,8 +298,8 @@ async fn assemble_build_publish(
|
||||
SECRET,
|
||||
date(),
|
||||
first,
|
||||
Vote::Up,
|
||||
&auth::rating_token(SECRET, date(), first, Vote::Up)
|
||||
Vote::Loved,
|
||||
&auth::rating_token(SECRET, date(), first, Vote::Loved)
|
||||
),
|
||||
"the server must accept the token the EPUB minted"
|
||||
);
|
||||
|
||||
+12
-1
@@ -190,7 +190,18 @@ fn scour_opml_still_yields_the_interest_list() {
|
||||
|
||||
// The assembled profile is what actually reaches DeepSeek as the system
|
||||
// prompt; it must mention the stated preferences and the interests (§3.6).
|
||||
let document = profile::build(&interests, profile::NO_LEARNED_ADJUSTMENTS);
|
||||
let profile_file = profile::load_profile(Path::new(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/data/profile.md"
|
||||
)))
|
||||
.expect("profile file");
|
||||
let document = profile::build(
|
||||
&profile_file.body,
|
||||
&interests,
|
||||
profile::NO_LEARNED_ADJUSTMENTS,
|
||||
&[],
|
||||
60,
|
||||
);
|
||||
assert!(document.contains("Rust"));
|
||||
assert!(document.len() > 1000, "the profile is suspiciously short");
|
||||
}
|
||||
|
||||
+12
-8
@@ -270,17 +270,21 @@ fn rating_links_carry_the_spec_token() {
|
||||
.expect("article chapter");
|
||||
|
||||
let date = issue.meta.date;
|
||||
let up = build::rating_token("integration-secret", date, 1, Vote::Up);
|
||||
let down = build::rating_token("integration-secret", date, 1, Vote::Down);
|
||||
assert_eq!(up.len(), 16);
|
||||
assert_ne!(up, down);
|
||||
let loved = build::rating_token("integration-secret", date, 1, Vote::Loved);
|
||||
let good = build::rating_token("integration-secret", date, 1, Vote::Good);
|
||||
let down = build::rating_token("integration-secret", date, 1, Vote::NotForMe);
|
||||
assert_eq!(loved.len(), 16);
|
||||
assert_ne!(loved, good);
|
||||
assert_ne!(good, down);
|
||||
for (segment, token) in [("loved", loved), ("good", good), ("down", down)] {
|
||||
assert!(chapter.xhtml.contains(&format!(
|
||||
"https://daily.hallada.net/r/2026-08-15/1/up?t={up}"
|
||||
)));
|
||||
assert!(chapter.xhtml.contains(&format!(
|
||||
"https://daily.hallada.net/r/2026-08-15/1/down?t={down}"
|
||||
"https://daily.hallada.net/r/2026-08-15/1/{segment}?t={token}"
|
||||
)));
|
||||
}
|
||||
assert!(chapter.xhtml.contains("[ Loved it ]"));
|
||||
assert!(chapter.xhtml.contains("[ Good ]"));
|
||||
assert!(chapter.xhtml.contains("[ Not for me ]"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn comment_and_world_fixtures_feed_real_chapters() {
|
||||
|
||||
Reference in New Issue
Block a user