diff --git a/README.md b/README.md index c35b816..554cda0 100644 --- a/README.md +++ b/README.md @@ -5,15 +5,20 @@ A personalized daily newspaper, delivered as an EPUB. Every morning a systemd timer wakes one Rust binary. It pulls the last ~26 hours from a self-hosted [Miniflux](https://miniflux.app), deduplicates and extracts the articles, enriches them with HackerNews/Lobsters/Reddit social proof, filters -300–500 candidates down to ~120 with cheap heuristics, and asks DeepSeek to score, -select and introduce 15–25 of them. It assembles two EPUB editions (a standard one +300–500 candidates down to ~120 with cheap heuristics, asks DeepSeek to score them, +and hands the shortlist to Claude Opus 5 — the editor — which assembles the issue +(no minimum size, a hard ceiling), writes a one-line *why* under every headline, +the summaries and *The Brief*. It assembles two EPUB editions (a standard one and one tuned for the Xteink X4 e-ink reader), converts the X4 edition to XTC, and publishes the lot over its own OPDS catalog — which doubles as a [BookOrbit](https://github.com/thallada/bookorbit) watched folder if you run one. Each article chapter ends with Loved it / Good / Not for me links that feed back into tomorrow's curation. -Steady-state cost is roughly **$0.05–0.30/day** in DeepSeek tokens, hard-capped by -`max_daily_usd`. +Steady-state cost is roughly **$1/day**: $0.05–0.30 in DeepSeek tokens plus +~$0.50–0.80 for the Claude editor, each with its own per-UTC-day ceiling +(`max_daily_usd` and `anthropic.max_daily_usd`). Those ceilings are runaway +guards, not accounting — set hard spend limits in both providers' dashboards as +the real backstop. - Full design: [`docs/plans/2026-08-15-the-daily-epub.md`](docs/plans/2026-08-15-the-daily-epub.md) - Implementation decisions: [`docs/plans/2026-08-15-implementation-notes.md`](docs/plans/2026-08-15-implementation-notes.md) @@ -24,7 +29,7 @@ Steady-state cost is roughly **$0.05–0.30/day** in DeepSeek tokens, hard-cappe ``` Miniflux ingest ─▶ dedupe ─▶ extraction ─▶ persist ─▶ social enrichment - ─▶ pre-filter ─▶ LLM scoring ─▶ selection ─▶ comments ─▶ editorial + ─▶ pre-filter ─▶ scoring (DeepSeek) ─▶ editor (Claude) ─▶ comments ─▶ editorial (Claude) ─▶ world briefing ─▶ EPUB (standard + X4) ─▶ XTC ─▶ publish ─▶ report ``` @@ -35,9 +40,12 @@ Every stage writes to SQLite, so a run is idempotent per date: re-running are fatal — without them there is no issue, and the `runs` row records why. Social lookups, comment fetching, the world briefing, images and the XTC conversion are best-effort: they log, add a warning (run status `degraded`) and -the run continues. Every DeepSeek stage *degrades*: a missing key, a dead API or -a tripped budget turns the run into the `--skip-llm` shape (prefilter order -selects, feed excerpts stand in for summaries) instead of losing the day's issue. +the run continues. Every LLM stage *degrades*: a Claude call that fails, is +refused, or is over its daily ceiling is retried with the same prompt on +DeepSeek; if DeepSeek is missing, dead or over budget too, the run takes the +`--skip-llm` shape (prefilter order selects, feed excerpts stand in for +summaries) instead of losing the day's issue. Anthropic's server-side refusal +fallback (`fallbacks = "default"`) is enabled on every editor request. --- @@ -47,7 +55,8 @@ selects, feed excerpts stand in for summaries) instead of losing the day's issue |---|---|---| | Rust (2024 edition toolchain) | building | `cargo build --release` | | **Miniflux** with an API key | the only content source | Settings → API Keys. The client is read-only and never mutates read state. | -| **DeepSeek API key** | curation + editorial | . Optional: `--skip-llm` runs the whole pipeline without it. | +| **DeepSeek API key** | scoring, and the fallback for every editor call | . Optional: `--skip-llm` runs the whole pipeline without it. | +| **Anthropic API key** | the editor: selection, summaries, The Brief, the weekly profile rebuild | . Optional: without it every editor call runs on DeepSeek. Set a dashboard spend limit; `anthropic.max_daily_usd` is only a runaway guard. | | A 32+ byte random secret | signs the article rating links | `openssl rand -hex 32` | | **BookOrbit** library + watched folder | *optional* — a richer library UI on top of the same folder | Delivery does not need it: `daily-epub serve` has its own OPDS catalog over `publish.epub_dir`. If you do run it, create a dedicated "The Daily EPUB" library, enable *Watch folders*, and point `publish.epub_dir` at it. | | **Node.js 18+** and a clone of [`epub-to-xtc-converter`](https://github.com/bigbag/epub-to-xtc-converter) | XTC/XTCH output for the Xteink X4 | Optional (`xtc.enabled = false` turns it off). Needs `npm install` **inside `cli/`**, and a settings JSON naming a real TTF/OTF — see below. It has **no global npm bin** — it is invoked as `node /cli/index.js convert …`, which is why `xtc.command`/`xtc.args` are fully general. | @@ -109,11 +118,11 @@ Secrets belong in the environment file, never in the TOML. |---|---|---| | `timezone` | `America/New_York` | Day boundaries and `--date` interpretation. | | `lookback_hours` | `26` | Size of the ingest window ending at the issue day's end (clamped to now). | -| `target_article_count` | `20` | Lineup size the selector aims for. `--max-articles` overrides it. | +| `target_article_count` | `20` | Soft target the editor aims for. There is no minimum: a nine-pick issue is published as nine. | | `prefilter_keep` | `120` | Candidates surviving the heuristic pre-filter. Must be ≥ `target_article_count`. | | `retention_days` | `21` | EPUBs older than this are deleted from `publish.epub_dir`. SQLite history is kept forever. | | `xtc_retention_count` | `5` | How many XTC issues to keep in `publish.xtc_dir`. Counted, not dated: each `.xtch` is ~80–100 MB, so the binding constraint is disk, not age. | -| `max_daily_usd` | `2.0` | Hard ceiling on DeepSeek spend **per day**, not per run — a re-run inherits what earlier runs for that date already spent. Tripping it skips remaining LLM work and degrades to excerpts. | +| `max_daily_usd` | `2.0` | Ceiling on DeepSeek spend per **UTC day** of the run's start, not per run — a re-run inherits what earlier runs that day already spent (`runs.provider_costs_json`). Tripping it skips remaining DeepSeek calls; in-flight requests finish and the paper still publishes. | | `world_briefing` | `true` | Include the Wikipedia Current Events section. | | `database_path` | `/var/lib/daily-epub/daily-epub.db` | SQLite file; parent dirs are created. | | `out_dir` | `/var/lib/daily-epub/out` | Where `generate` writes artifacts before publishing. | @@ -126,11 +135,24 @@ Secrets belong in the environment file, never in the TOML. | `deepseek.model` | `deepseek-v4-flash` | Verified 2026-08-15 (DeepSeek-V4-Flash-0731). | | `deepseek.api_key` | — | **`DAILY_EPUB_DEEPSEEK__API_KEY`**. Absent ⇒ the run curates heuristically. | | `deepseek.score_batch_size` | `12` | Articles per stage-A scoring request. | -| `deepseek.score_temperature` | `0.3` | Scoring/selection temperature. | -| `deepseek.editorial_temperature` | `0.8` | Summaries, intros, front page. | +| `deepseek.max_concurrent_requests` | `4` | Stage-A batches in flight at once; the budget is checked before each is spawned. | +| `deepseek.score_temperature` | `0.3` | Scoring temperature. | +| `deepseek.editorial_temperature` | `0.8` | Summaries and The Brief, only when DeepSeek is the fallback editor. | | `deepseek.price_input_per_mtok` | `0.14` | USD per 1M cache-miss input tokens (cost guardrail arithmetic). | | `deepseek.price_cached_input_per_mtok` | `0.0028` | USD per 1M prefix-cache-hit input tokens. | | `deepseek.price_output_per_mtok` | `0.28` | USD per 1M output tokens. | +| `anthropic.enabled` | `true` | `false` runs every editor call on DeepSeek. | +| `anthropic.base_url` | `https://api.anthropic.com` | Messages API root. | +| `anthropic.model` | `claude-opus-5` | The editor. Requests carry `output_config.effort`, a cached system block, and `fallbacks = "default"` with the `server-side-fallback-2026-07-01` beta so a classifier refusal is re-routed server-side. | +| `anthropic.api_key` | — | **`DAILY_EPUB_ANTHROPIC__API_KEY`**. Absent ⇒ editor calls fall back to DeepSeek. | +| `anthropic.effort` | `high` | `low`, `medium`, `high`, `xhigh` or `max`. | +| `anthropic.price_input_per_mtok` | `5.0` | USD per 1M uncached input tokens. | +| `anthropic.price_cache_write_per_mtok` | `6.25` | USD per 1M tokens written to the prompt cache. | +| `anthropic.price_cache_read_per_mtok` | `0.5` | USD per 1M cache-read input tokens. | +| `anthropic.price_output_per_mtok` | `25.0` | USD per 1M output tokens. | +| `anthropic.max_daily_usd` | `3.0` | Claude ceiling per UTC day; tripping it moves the remaining editor work to DeepSeek. | +| `anthropic.max_concurrent_requests` | `4` | Reserved for the parallel editor stages. | +| `curation.max_article_count` | `28` | Hard ceiling on issue size. `--max-articles N` lowers it to `min(28, N)` and drags the soft target down with it. Must be ≥ `target_article_count`. | | `curation.always_include_feeds` | `[]` | Miniflux feed ids or URL substrings that can never be dropped. | | `curation.blocked_domains` | `[]` | Hosts excluded outright. | | `curation.paywall_domains` | `[]` | Extra paywalled hosts, merged with the built-in list (nytimes, wsj, ft, economist, …). | @@ -139,6 +161,8 @@ Secrets belong in the environment file, never in the TOML. | `curation.feedback.good_value` | `0.35` | Weight for a Good verdict. | | `curation.feedback.not_for_me_value` | `-1.0` | Weight for a Not for me verdict. | | `curation.feedback.verdicts_in_prompt` | `60` | Recent explicit verdicts included in the system prompt. | +| `editorial.summary_model` | `editor` | `editor` (Claude) or `bulk` (DeepSeek) for the per-article summaries. | +| `editorial.summary_input_tokens` | `3000` | Article text offered to the summary prompt. | | `publish.epub_dir` | `/srv/bookorbit/libraries/daily-epub` | Both EPUB editions land here by atomic copy, and this is the directory the OPDS feed lists. The editions are distinguished by a `(X4)` tag in **both** the filename and `dc:title` — libraries and OPDS clients list books by title, so the filename alone would make them look identical. Point a BookOrbit watched folder at it if you want its UI too. **Renamed from `bookorbit_dir`**; the old key is a hard config error. | | `publish.xtc_dir` | `/var/lib/daily-epub/xtc` | XTC artifacts. **Not** listed in the OPDS feed — CrossPoint cannot acquire them — but downloadable at `/files/xtc/` for sideloading. | | `xtc.enabled` | `true` | Set `false` to skip the converter entirely. | @@ -174,6 +198,7 @@ sudo -e /etc/daily-epub/config.toml # set publish dirs, xtc args, pub sudo tee /etc/daily-epub/env >/dev/null < Pick { section: target.issue.clone(), position: 0, is_lead: false, + why: None, summary: None, llm: None, discussion: None, @@ -412,7 +413,6 @@ fn write_audit_epub(out_dir: &Path, picks: &[Pick], assets: &[ImageAsset]) { Articles are re-extracted live; editorial, discussions and the \ world briefing are absent by design.

" .into(), - section_intros: Default::default(), summaries: Default::default(), }, world_briefing: None, diff --git a/src/config.rs b/src/config.rs index 3c42a58..2ac63ad 100644 --- a/src/config.rs +++ b/src/config.rs @@ -74,7 +74,9 @@ pub struct Config { pub miniflux: MinifluxConfig, pub deepseek: DeepseekConfig, + pub anthropic: AnthropicConfig, pub curation: CurationConfig, + pub editorial: EditorialConfig, pub publish: PublishConfig, pub xtc: XtcConfig, pub server: ServerConfig, @@ -97,7 +99,9 @@ impl Default for Config { profile_path: PathBuf::from("data/profile.md"), miniflux: MinifluxConfig::default(), deepseek: DeepseekConfig::default(), + anthropic: AnthropicConfig::default(), curation: CurationConfig::default(), + editorial: EditorialConfig::default(), publish: PublishConfig::default(), xtc: XtcConfig::default(), server: ServerConfig::default(), @@ -136,6 +140,7 @@ pub struct DeepseekConfig { pub api_key: Option, /// Articles per stage-A scoring request (§3.6). pub score_batch_size: usize, + pub max_concurrent_requests: usize, pub score_temperature: f32, pub editorial_temperature: f32, /// USD per 1M cache-miss input tokens. @@ -153,6 +158,7 @@ impl Default for DeepseekConfig { model: "deepseek-v4-flash".into(), api_key: None, score_batch_size: 12, + max_concurrent_requests: 4, score_temperature: 0.3, editorial_temperature: 0.8, price_input_per_mtok: 0.14, @@ -162,10 +168,73 @@ impl Default for DeepseekConfig { } } +/// `[anthropic]` — Claude editor, editorial and profile settings (§4.2). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields, default)] +pub struct AnthropicConfig { + pub enabled: bool, + pub base_url: String, + pub model: String, + /// Supply only via `DAILY_EPUB_ANTHROPIC__API_KEY`. + pub api_key: Option, + pub effort: String, + pub price_input_per_mtok: f64, + pub price_cache_write_per_mtok: f64, + pub price_cache_read_per_mtok: f64, + pub price_output_per_mtok: f64, + pub max_daily_usd: f64, + pub max_concurrent_requests: usize, +} + +impl Default for AnthropicConfig { + fn default() -> Self { + Self { + enabled: true, + base_url: "https://api.anthropic.com".into(), + model: "claude-opus-5".into(), + api_key: None, + effort: "high".into(), + price_input_per_mtok: 5.0, + price_cache_write_per_mtok: 6.25, + price_cache_read_per_mtok: 0.5, + price_output_per_mtok: 25.0, + max_daily_usd: 3.0, + max_concurrent_requests: 4, + } + } +} + +/// Which provider writes per-article summaries (§14.1). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum SummaryModel { + Editor, + Bulk, +} + +/// `[editorial]` — summary provider and per-article input budget (§14). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields, default)] +pub struct EditorialConfig { + pub summary_model: SummaryModel, + pub summary_input_tokens: usize, +} + +impl Default for EditorialConfig { + fn default() -> Self { + Self { + summary_model: SummaryModel::Editor, + summary_input_tokens: 3_000, + } + } +} + /// `[curation]` — pre-filter and section palette (§3.5, §3.6). #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[serde(deny_unknown_fields, default)] pub struct CurationConfig { + /// Absolute issue-size ceiling; the editor has no minimum (§13). + pub max_article_count: usize, /// Miniflux feed ids or site URLs that can never be dropped (§3.5). pub always_include_feeds: Vec, /// Hosts excluded outright (§3.5). @@ -181,6 +250,7 @@ pub struct CurationConfig { impl Default for CurationConfig { fn default() -> Self { Self { + max_article_count: 28, always_include_feeds: Vec::new(), blocked_domains: Vec::new(), paywall_domains: Vec::new(), @@ -376,6 +446,39 @@ impl Config { "prefilter_keep must be >= target_article_count".into(), )); } + if self.curation.max_article_count < self.target_article_count { + return Err(ConfigError::Invalid( + "curation.max_article_count must be >= target_article_count".into(), + )); + } + if self.deepseek.score_batch_size == 0 { + return Err(ConfigError::Invalid( + "deepseek.score_batch_size must be >= 1".into(), + )); + } + if self.deepseek.max_concurrent_requests == 0 { + return Err(ConfigError::Invalid( + "deepseek.max_concurrent_requests must be >= 1".into(), + )); + } + if self.anthropic.max_concurrent_requests == 0 { + return Err(ConfigError::Invalid( + "anthropic.max_concurrent_requests must be >= 1".into(), + )); + } + if self.editorial.summary_input_tokens == 0 { + return Err(ConfigError::Invalid( + "editorial.summary_input_tokens must be >= 1".into(), + )); + } + if !matches!( + self.anthropic.effort.as_str(), + "low" | "medium" | "high" | "xhigh" | "max" + ) { + return Err(ConfigError::Invalid( + "anthropic.effort must be one of low, medium, high, xhigh, max".into(), + )); + } if self.curation.sections.is_empty() { return Err(ConfigError::Invalid( "curation.sections must not be empty".into(), @@ -498,6 +601,42 @@ mod tests { assert_eq!(c.xtc.format, XtcFormat::Xtch); assert_eq!(c.server.bind, "127.0.0.1:3499"); assert_eq!(c.deepseek.base_url, "https://api.deepseek.com/v1"); + assert_eq!(c.deepseek.max_concurrent_requests, 4); + assert!(c.anthropic.enabled); + assert_eq!(c.anthropic.model, "claude-opus-5"); + assert_eq!(c.anthropic.effort, "high"); + assert!(c.anthropic.api_key.is_none(), "keys never live in the file"); + assert_eq!(c.anthropic.max_daily_usd, 3.0); + assert_eq!(c.curation.max_article_count, 28); + assert_eq!(c.editorial.summary_model, SummaryModel::Editor); + assert_eq!(c.editorial.summary_input_tokens, 3000); + } + + #[test] + fn provider_validation_rejects_nonsense() { + let mut c = Config::default(); + c.curation.max_article_count = c.target_article_count - 1; + assert!(c.validate().is_err(), "max_article_count below the target"); + let mut c = Config::default(); + c.anthropic.effort = "turbo".into(); + assert!(c.validate().is_err(), "unknown effort"); + for effort in ["low", "medium", "high", "xhigh", "max"] { + let mut c = Config::default(); + c.anthropic.effort = effort.into(); + assert!(c.validate().is_ok(), "{effort} is a valid effort"); + } + let mut c = Config::default(); + c.deepseek.max_concurrent_requests = 0; + assert!(c.validate().is_err()); + let mut c = Config::default(); + c.anthropic.max_concurrent_requests = 0; + assert!(c.validate().is_err()); + let mut c = Config::default(); + c.deepseek.score_batch_size = 0; + assert!(c.validate().is_err()); + let mut c = Config::default(); + c.editorial.summary_input_tokens = 0; + assert!(c.validate().is_err()); } #[test] diff --git a/src/curate/editorial.rs b/src/curate/editorial.rs index 35ce406..842e833 100644 --- a/src/curate/editorial.rs +++ b/src/curate/editorial.rs @@ -1,34 +1,19 @@ -//! Stage C — summaries, section intros and the front page (spec §3.6). -//! -//! Voice: warm, literate, a little playful; never fabricates facts that are not -//! present in the summaries. -//! -//! Everything here is best-effort. If the cost ceiling trips mid-way (§3.6) or a -//! call fails, the affected article silently falls back to its own opening words -//! and the run continues — an issue with plain excerpts is far better than no -//! issue at all. +//! Claude-first summaries and The Brief, with per-call DeepSeek fallback (§14). use std::collections::BTreeMap; use std::fmt::Write as _; -use serde::{Deserialize, Serialize}; +use futures::{StreamExt, stream}; +use serde::Deserialize; -use super::llm::{LlmClient, LlmError}; +use super::llm::{LlmClient, LlmError, Llms}; use super::{escape_html, prompt_text, text_to_paragraphs, truncate_tokens, truncate_words}; +use crate::config::{EditorialConfig, SummaryModel}; use crate::types::{ArticleId, Editorial, Lineup, Pick}; -/// Article text is truncated to roughly this many tokens per summary call (§3.6). -pub const SUMMARY_INPUT_TOKEN_BUDGET: usize = 5000; -/// Target length of the "From the Editor" front page, in words (§3.6). -pub const FRONT_PAGE_WORDS: (usize, usize) = (250, 400); -/// Words of body text used when a summary has to fall back to the excerpt. pub const FALLBACK_SUMMARY_WORDS: usize = 45; +pub const SUMMARY_CONCURRENCY: usize = 4; -// --------------------------------------------------------------------------- -// Prompts (reusable instructions here; per-call material in the user message) -// --------------------------------------------------------------------------- - -/// Per-article summary instructions (§3.6 stage C). pub const SUMMARY_INSTRUCTIONS: &str = "\ TASK: write the newspaper abstract for one article in today's issue. @@ -57,66 +42,35 @@ excerpt. Return JSON exactly: {\"summary\": \"\"}"; -/// Front-page + section-intro instructions (§3.6 stage C). -pub const FRONT_PAGE_INSTRUCTIONS: &str = "\ -TASK: write the front page of today's issue of The Daily EPUB. +/// The Brief instructions (§14.2). +pub const BRIEF_INSTRUCTIONS: &str = r#"TASK: write "The Brief" for today's issue — the note at the top of the paper. -You are given the whole lineup: sections, headlines, sources and the abstract \ -written for each article. Everything you write must come from those abstracts — \ -you have not read the articles themselves, and inventing a fact would be worse \ -than saying less. +120-200 words, one or two paragraphs. It must earn its place: if a reader skipped +it, what would he miss? Name at least three of today's picks by title and say the +specific thing that makes each worth his time (the result, the argument, the scale, +the person). If there is a thread connecting several pieces, say it in one sentence; +if there is not, do not invent one. If the issue is short, say why in one clause. -Produce two things. +Do not: welcome the reader, describe the weather, summarize every section, use +"delve", "dive", "explore", "a mix of", "something for everyone", or any sentence +that could introduce any other issue. No headings. No bullet points. -1. \"from_the_editor\" — 250 to 400 words of prose addressed to the paper's one \ -reader. Find the two or three threads that actually run through today's lineup \ -(a shared question, an argument between two pieces, an accidental theme) and use \ -them to guide the read: what to start with over coffee, what to save for the \ -commute, what rewards patience. Name the lead story and say why it leads. It is \ -fine — good, even — to note when a day is quiet or lopsided. Voice: warm, \ -literate, lightly playful, never breathless; a real editor writing to someone \ -whose taste he knows. No bullet lists, no headings, no emoji, 2–4 paragraphs \ -separated by a blank line. +Return JSON exactly: {"brief": ""}"#; -2. \"section_intros\" — for EACH section name given below, two or three \ -sentences (35–60 words) introducing what is in it today. Concrete, specific to \ -these articles, no filler like \"a variety of interesting stories\". Use the \ -section names exactly as spelled in the lineup. - -Return JSON exactly: -{\"from_the_editor\": \"\", \ -\"section_intros\": {\"
\": \"<2-3 sentences>\"}}"; - -/// The single front-page call's JSON response (§3.6). -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -pub struct FrontPageResponse { - /// "From the Editor", 250–400 words. - pub from_the_editor: String, - /// Section name → 2–3 sentence intro. +#[derive(Debug, Clone, PartialEq, Deserialize)] +pub struct BriefResponse { #[serde(default)] - pub section_intros: BTreeMap, + pub brief: String, } -/// The per-article summary call's JSON response. #[derive(Debug, Clone, Default, Deserialize)] struct SummaryResponse { #[serde(default)] summary: String, } -// --------------------------------------------------------------------------- -// Per-article summaries -// --------------------------------------------------------------------------- - -/// One 2–3 sentence newspaper abstract: what it argues, why it's worth reading (§3.6). -pub async fn summarize_article( - llm: &LlmClient, - title: &str, - body_html: &str, - temperature: f32, -) -> Result { - llm.meter.check_budget()?; - let body = truncate_tokens(&prompt_text(body_html), SUMMARY_INPUT_TOKEN_BUDGET); +fn summary_prompt(title: &str, body_html: &str, input_tokens: usize) -> String { + let body = truncate_tokens(&prompt_text(body_html), input_tokens); let mut prompt = String::with_capacity(body.len() + SUMMARY_INSTRUCTIONS.len() + 256); prompt.push_str(SUMMARY_INSTRUCTIONS); let _ = write!( @@ -129,184 +83,182 @@ pub async fn summarize_article( "" }, if body.is_empty() { - "(no body text was extracted; summarize from the headline alone and say the \ - full text was unavailable)" + "(no body text was extracted; summarize from the headline alone and say the full text was unavailable)" } else { &body } ); + prompt +} + +pub async fn summarize_article( + llm: &LlmClient, + title: &str, + body_html: &str, + input_tokens: usize, + temperature: f32, +) -> Result { + let prompt = summary_prompt(title, body_html, input_tokens); let response: SummaryResponse = llm.complete_json(&prompt, temperature).await?; let summary = response.summary.trim().to_string(); if summary.is_empty() { - return Err(LlmError::EmptyResponse); + return Err(LlmError::EmptyResponse { + provider: llm.provider, + }); } Ok(summary) } -/// Summarize every pick, returning `article_id → summary` (§3.6). -/// -/// Stops early and returns what it has when the cost guardrail trips (§3.6). +/// `(primary, fallback)` for the summaries per `editorial.summary_model` (§14.1). +fn summary_clients(llms: &Llms, model: SummaryModel) -> (Option<&LlmClient>, Option<&LlmClient>) { + match model { + SummaryModel::Bulk => (llms.bulk.as_ref(), None), + SummaryModel::Editor => { + let primary = llms.editor_or_bulk(); + let fallback = primary.and_then(|client| { + llms.bulk + .as_ref() + .filter(|bulk| bulk.provider != client.provider) + }); + (primary, fallback) + } + } +} + +async fn summarize_pick( + pick: &Pick, + primary: Option<&LlmClient>, + fallback: Option<&LlmClient>, + config: &EditorialConfig, + temperature: f32, +) -> Option { + let primary = primary?; + match summarize_article( + primary, + &pick.article.title, + &pick.article.content_html, + config.summary_input_tokens, + temperature, + ) + .await + { + Ok(summary) => Some(summary), + Err(error) => { + let Some(fallback) = fallback else { + tracing::warn!(article_id = pick.article.id, %error, "summary failed; using excerpt"); + return None; + }; + tracing::warn!(article_id = pick.article.id, %error, "editor summary failed; retrying on bulk"); + summarize_article( + fallback, + &pick.article.title, + &pick.article.content_html, + config.summary_input_tokens, + temperature, + ) + .await + .map_err(|fallback_error| { + tracing::warn!(article_id = pick.article.id, %fallback_error, "bulk summary failed; using excerpt"); + }) + .ok() + } + } +} + pub async fn summarize_all( - llm: &LlmClient, + llms: &Llms, lineup: &Lineup, + config: &EditorialConfig, temperature: f32, ) -> BTreeMap { - let mut out = BTreeMap::new(); - for (n, pick) in lineup.picks.iter().enumerate() { - if llm.meter.budget_exceeded() { - tracing::error!( - summarized = out.len(), - remaining = lineup.picks.len() - n, - spent_usd = llm.meter.cost_usd(), - "COST CEILING HIT during stage C — the remaining articles fall back to \ - feed excerpts as summaries" - ); - break; - } - match summarize_article( - llm, - &pick.article.title, - &pick.article.content_html, - temperature, - ) + let (primary, fallback) = summary_clients(llms, config.summary_model); + stream::iter(lineup.picks.iter()) + .map(|pick| async move { + let summary = summarize_pick(pick, primary, fallback, config, temperature).await; + (pick.article.id, summary) + }) + .buffer_unordered(SUMMARY_CONCURRENCY) + .filter_map(|(id, summary)| async move { summary.map(|summary| (id, summary)) }) + .collect() .await - { - Ok(summary) => { - out.insert(pick.article.id, summary); - } - Err(LlmError::BudgetExceeded { spent, limit }) => { - tracing::error!(spent, limit, "COST CEILING HIT during stage C"); - break; - } - Err(e) => { - tracing::warn!( - article_id = pick.article.id, - title = %pick.article.title, - error = %e, - "summary failed; falling back to the article's own opening" - ); - } - } - } - tracing::info!( - summarized = out.len(), - picks = lineup.picks.len(), - "stage C summaries complete" - ); - out } -// --------------------------------------------------------------------------- -// Front page -// --------------------------------------------------------------------------- - -/// The single front-page + section-intro call (§3.6). -pub async fn front_page( - llm: &LlmClient, - lineup: &Lineup, - summaries: &BTreeMap, - temperature: f32, -) -> Result { - llm.meter.check_budget()?; - let prompt = build_front_page_prompt(lineup, summaries); - tracing::debug!( - approx_tokens = super::approx_tokens(&prompt), - "stage C front-page request" - ); - let mut response: FrontPageResponse = llm.complete_json(&prompt, temperature).await?; - response.from_the_editor = response.from_the_editor.trim().to_string(); - if response.from_the_editor.is_empty() { - return Err(LlmError::EmptyResponse); - } - // Keep only intros for sections that actually exist in the issue. - response - .section_intros - .retain(|name, text| lineup.section_order.contains(name) && !text.trim().is_empty()); - Ok(response) -} - -/// Render the front-page user prompt: the whole lineup with its abstracts (§3.6). -pub fn build_front_page_prompt(lineup: &Lineup, summaries: &BTreeMap) -> String { +pub fn build_brief_prompt(lineup: &Lineup, summaries: &BTreeMap) -> String { let mut prompt = String::with_capacity(4096); - prompt.push_str(FRONT_PAGE_INSTRUCTIONS); - let minutes: i64 = lineup - .picks - .iter() - .map(|p| p.article.reading_minutes()) - .sum(); + prompt.push_str(BRIEF_INSTRUCTIONS); let _ = write!( prompt, - "\n\nISSUE: {} · {} articles across {} sections · about {} minutes of reading\n\ - SECTIONS, in order: {}\n\nLINEUP\n", + "\n\nISSUE: {} · {} articles\n\nLINEUP\n", lineup.date, - lineup.picks.len(), - lineup.section_order.len(), - minutes, - lineup.section_order.join(" | ") + lineup.picks.len() ); for section in &lineup.section_order { - let _ = write!(prompt, "\n## {section}\n"); + let _ = writeln!(prompt, "\n## {section}"); for pick in lineup.section_picks(section) { - let _ = write!(prompt, "{}", render_pick(pick, summaries)); + let score = pick + .llm + .as_ref() + .map(|score| format!("{:.1}", score.score)) + .unwrap_or_else(|| "unscored".into()); + let summary = summaries + .get(&pick.article.id) + .cloned() + .unwrap_or_else(|| excerpt_summary(pick)); + let _ = writeln!( + prompt, + "- {}\n feed: {}\n why: {}\n score: {}\n summary: {}", + pick.article.title.trim(), + pick.article.feed_title.trim(), + pick.why.as_deref().unwrap_or("not supplied"), + score, + summary + ); } } prompt } -fn render_pick(pick: &Pick, summaries: &BTreeMap) -> String { - let a = &pick.article; - let mut block = String::with_capacity(400); - let _ = writeln!( - block, - "\n- {}{}", - a.title.trim(), - if pick.is_lead { " [LEAD STORY]" } else { "" } - ); - let _ = writeln!( - block, - " source: {} · {} words (~{} min){}", - if a.feed_title.is_empty() { - "unknown" - } else { - a.feed_title.trim() - }, - a.word_count, - a.reading_minutes(), - social_note(pick) - ); - let abstract_text = summaries - .get(&a.id) - .cloned() - .unwrap_or_else(|| excerpt_summary(pick)); - let _ = writeln!(block, " abstract: {abstract_text}"); - block -} - -fn social_note(pick: &Pick) -> String { - if pick.article.social.is_empty() { - return String::new(); +pub async fn brief( + llms: &Llms, + lineup: &Lineup, + summaries: &BTreeMap, + temperature: f32, +) -> Result { + let prompt = build_brief_prompt(lineup, summaries); + let Some(primary) = llms.editor_or_bulk() else { + return Err(LlmError::Api { + provider: "editorial", + message: "no provider configured".into(), + }); + }; + let response = match primary + .complete_json::(&prompt, temperature) + .await + { + Ok(response) => response, + Err(error) => { + let Some(fallback) = llms + .bulk + .as_ref() + .filter(|bulk| bulk.provider != primary.provider) + else { + return Err(error); + }; + tracing::warn!(%error, "brief failed on editor; retrying on bulk"); + fallback + .complete_json::(&prompt, temperature) + .await? + } + }; + let brief = response.brief.trim().to_string(); + if brief.is_empty() { + return Err(LlmError::EmptyResponse { + provider: primary.provider, + }); } - let parts: Vec = pick - .article - .social - .iter() - .map(|s| { - format!( - "{} {} pts/{} comments", - s.source.display_name(), - s.score, - s.num_comments - ) - }) - .collect(); - format!(" · {}", parts.join(", ")) + Ok(brief) } -// --------------------------------------------------------------------------- -// Fallbacks (§3.6, notes §6) -// --------------------------------------------------------------------------- - -/// The article's own opening words, used when no LLM summary exists (§3.6). pub fn excerpt_summary(pick: &Pick) -> String { let text = truncate_words( &prompt_text(&pick.article.content_html), @@ -326,17 +278,14 @@ pub fn excerpt_summary(pick: &Pick) -> String { } } -/// A plain, factual front page used when the model is unavailable (§3.6, notes §6). pub fn fallback_front_page_html(lineup: &Lineup) -> String { let minutes: i64 = lineup .picks .iter() - .map(|p| p.article.reading_minutes()) + .map(|pick| pick.article.reading_minutes()) .sum(); let mut text = format!( - "Today's issue collects {} articles across {} sections — about {} minutes of \ - reading. Editorial notes are unavailable for this issue, so the lineup speaks \ - for itself.", + "Today's issue collects {} articles across {} sections — about {} minutes of reading. Editorial notes are unavailable for this issue, so the lineup speaks for itself.", lineup.picks.len(), lineup.section_order.len(), minutes @@ -346,29 +295,15 @@ pub fn fallback_front_page_html(lineup: &Lineup) -> String { text, "\n\nLeading today: “{}” ({}).", lead.article.title.trim(), - if lead.article.feed_title.is_empty() { - "source unknown" - } else { - lead.article.feed_title.trim() - } - ); - } - if !lineup.section_order.is_empty() { - let _ = write!( - text, - "\n\nIn this issue: {}.", - lineup.section_order.join(", ") + lead.article.feed_title.trim() ); } text_to_paragraphs(&text) } -/// `--skip-llm` / budget-exceeded fallback: feed excerpts stand in for summaries -/// and the front page is a plain stats line (§3.6, notes §6). pub fn fallback_editorial(lineup: &Lineup) -> Editorial { Editorial { front_page_html: fallback_front_page_html(lineup), - section_intros: BTreeMap::new(), summaries: lineup .picks .iter() @@ -377,54 +312,34 @@ pub fn fallback_editorial(lineup: &Lineup) -> Editorial { } } -// --------------------------------------------------------------------------- -// Stage driver -// --------------------------------------------------------------------------- - -/// Stage C end to end: summaries, then one front-page call, with excerpts filling -/// every gap (§3.6). -pub async fn run(llm: &LlmClient, lineup: &Lineup, temperature: f32) -> Editorial { +pub async fn run( + llms: &Llms, + lineup: &Lineup, + config: &EditorialConfig, + temperature: f32, +) -> Editorial { if lineup.picks.is_empty() { return fallback_editorial(lineup); } - - let mut summaries = summarize_all(llm, lineup, temperature).await; - let missing: Vec<&Pick> = lineup - .picks - .iter() - .filter(|p| !summaries.contains_key(&p.article.id)) - .collect(); - if !missing.is_empty() { - tracing::warn!( - count = missing.len(), - "using feed excerpts as summaries for articles the model did not cover" - ); - for pick in missing { - summaries.insert(pick.article.id, excerpt_summary(pick)); - } + let mut summaries = summarize_all(llms, lineup, config, temperature).await; + for pick in &lineup.picks { + summaries + .entry(pick.article.id) + .or_insert_with(|| excerpt_summary(pick)); } - - let (front_page_html, section_intros) = - match front_page(llm, lineup, &summaries, temperature).await { - Ok(response) => ( - text_to_paragraphs(&response.from_the_editor), - response.section_intros, - ), - Err(e) => { - tracing::error!(error = %e, - "front-page generation failed; using the plain front page"); - (fallback_front_page_html(lineup), BTreeMap::new()) - } - }; - + let front_page_html = match brief(llms, lineup, &summaries, temperature).await { + Ok(text) => text_to_paragraphs(&text), + Err(error) => { + tracing::warn!(%error, "brief failed; using fallback front page"); + fallback_front_page_html(lineup) + } + }; Editorial { front_page_html, - section_intros, summaries, } } -/// Escape-and-wrap helper for callers rendering a summary straight into XHTML. pub fn summary_to_html(summary: &str) -> String { format!("

{}

", escape_html(summary.trim())) } @@ -432,15 +347,15 @@ pub fn summary_to_html(summary: &str) -> String { #[cfg(test)] mod tests { use super::*; - use crate::config::DeepseekConfig; - use crate::curate::llm::{MockBackend, UsageMeter}; + use crate::config::{AnthropicConfig, DeepseekConfig}; + use crate::curate::llm::{ChatBackend, MockBackend, PriceTable, UsageMeter}; use crate::curate::prefilter::tests::article; use crate::types::TokenUsage; use std::sync::Arc; - const FRONT_PAGE_FIXTURE: &str = include_str!(concat!( + const BRIEF_FIXTURE: &str = include_str!(concat!( env!("CARGO_MANIFEST_DIR"), - "/tests/fixtures/deepseek_front_page.json" + "/tests/fixtures/claude_brief.json" )); fn pick(id: i64, title: &str, section: &str, is_lead: bool) -> Pick { @@ -451,6 +366,7 @@ mod tests { section: section.into(), position: 1, is_lead, + why: Some(format!("the {title} piece you'd argue with")), summary: None, llm: None, discussion: None, @@ -468,15 +384,40 @@ mod tests { } } - fn client(backend: Arc, limit: f64) -> LlmClient { - LlmClient::with_backend( - "deepseek-v4-flash", + fn mock(provider: &'static str, backend: Arc, limit: f64) -> LlmClient { + let prices = if provider == "anthropic" { + PriceTable::anthropic(&AnthropicConfig::default()) + } else { + PriceTable::deepseek(&DeepseekConfig::default()) + }; + LlmClient::with_backend_options( + provider, + "model", "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), limit), - backend, + None, + UsageMeter::with_prices(prices, limit), + backend as Arc, ) } + fn bulk_only(backend: Arc, limit: f64) -> Llms { + Llms { + bulk: Some(mock("deepseek", backend, limit)), + editor: None, + } + } + + fn editor_and_bulk(editor: Arc, bulk: Arc) -> Llms { + Llms { + bulk: Some(mock("deepseek", bulk, 2.0)), + editor: Some(mock("anthropic", editor, 3.0)), + } + } + + fn config() -> EditorialConfig { + EditorialConfig::default() + } + #[tokio::test] async fn summary_prompt_carries_headline_and_truncated_body() { let backend = Arc::new(MockBackend::new()); @@ -484,9 +425,9 @@ mod tests { r#"{"summary": "A team moves 40TB of relational data off Postgres and documents every rollback."}"#, TokenUsage::default(), ); - let llm = client(Arc::clone(&backend), 2.0); + let llm = mock("deepseek", Arc::clone(&backend), 2.0); let body = format!("

{}

", "word ".repeat(20_000)); - let summary = summarize_article(&llm, "Migrating 40TB", &body, 0.8) + let summary = summarize_article(&llm, "Migrating 40TB", &body, 3_000, 0.8) .await .expect("summary"); assert!(summary.starts_with("A team moves 40TB")); @@ -495,60 +436,133 @@ mod tests { assert!(prompt.starts_with(SUMMARY_INSTRUCTIONS)); assert!(prompt.contains("HEADLINE: Migrating 40TB")); assert!(prompt.contains("(truncated for length)")); - // ~5k tokens ≈ 20k characters of body, not the full 100k. - assert!(prompt.len() < 26_000, "prompt was {} bytes", prompt.len()); + // 3k tokens ≈ 12k characters of body, not the full 100k. + assert!(prompt.len() < 16_000, "prompt was {} bytes", prompt.len()); } #[tokio::test] - async fn front_page_parses_and_filters_unknown_sections() { + async fn the_brief_is_parsed_and_rendered() { let backend = Arc::new(MockBackend::new()); - backend.push(FRONT_PAGE_FIXTURE, TokenUsage::default()); - let llm = client(Arc::clone(&backend), 2.0); + backend.push(BRIEF_FIXTURE, TokenUsage::default()); + let llms = bulk_only(Arc::clone(&backend), 2.0); let lineup = lineup(); let summaries = BTreeMap::from([ (1, "A migration story with numbers.".to_string()), (2, "Transit data, charted.".to_string()), ]); - let response = front_page(&llm, &lineup, &summaries, 0.8) - .await - .expect("front page"); - assert!(response.from_the_editor.split_whitespace().count() > 40); - assert_eq!(response.section_intros.len(), 2); - assert!(response.section_intros.contains_key("Top Stories")); - assert!( - !response.section_intros.contains_key("Niche Corner"), - "intros for absent sections are dropped" - ); + let text = brief(&llms, &lineup, &summaries, 0.8).await.expect("brief"); + assert!(text.split_whitespace().count() > 100); + assert!(text.contains("Migrating 40TB off Postgres")); let prompt = &backend.prompts()[0].user; - assert!(prompt.starts_with(FRONT_PAGE_INSTRUCTIONS)); + assert!(prompt.starts_with(BRIEF_INSTRUCTIONS)); assert!(prompt.contains("## Top Stories")); - assert!(prompt.contains("[LEAD STORY]")); - assert!(prompt.contains("abstract: A migration story with numbers.")); + assert!(prompt.contains("## Boston & Local")); + assert!(prompt.contains("- Migrating 40TB off Postgres")); + assert!(prompt.contains("why: the Migrating 40TB off Postgres piece you'd argue with")); + assert!(prompt.contains("summary: A migration story with numbers.")); + assert!(prompt.contains("score: unscored")); assert!(prompt.contains("2026-08-15")); + assert!( + !prompt.contains("section_intros"), + "section intros are gone" + ); } #[tokio::test] - async fn full_stage_c_produces_summaries_intros_and_front_page() { + async fn full_stage_c_produces_summaries_and_the_brief() { let backend = Arc::new(MockBackend::new()); backend.push(r#"{"summary": "First abstract."}"#, TokenUsage::default()); backend.push(r#"{"summary": "Second abstract."}"#, TokenUsage::default()); - backend.push(FRONT_PAGE_FIXTURE, TokenUsage::default()); - let llm = client(Arc::clone(&backend), 2.0); + backend.push(BRIEF_FIXTURE, TokenUsage::default()); + let llms = bulk_only(Arc::clone(&backend), 2.0); - let editorial = run(&llm, &lineup(), 0.8).await; - assert_eq!( - backend.calls(), - 3, - "one call per article plus the front page" - ); + let editorial = run(&llms, &lineup(), &config(), 0.8).await; + assert_eq!(backend.calls(), 3, "one call per article plus the brief"); assert_eq!(editorial.summaries.len(), 2); assert_eq!(editorial.summaries[&1], "First abstract."); assert!(editorial.front_page_html.starts_with("

")); assert!(editorial.front_page_html.contains("

")); + assert!( + editorial + .front_page_html + .contains("Migrating 40TB off Postgres") + ); assert!(!editorial.front_page_html.contains("")); // An empty lineup is still a valid editorial. diff --git a/src/curate/llm.rs b/src/curate/llm.rs index 20cbee0..90e6265 100644 --- a/src/curate/llm.rs +++ b/src/curate/llm.rs @@ -1,28 +1,23 @@ -//! DeepSeek client and token/cost accounting (spec §3.6). +//! Provider-neutral LLM clients, transports, retry, and token accounting (§4, §5). //! -//! The OpenAI-compatible chat-completions endpoint at `https://api.deepseek.com/v1`. -//! DeepSeek prefix-caches automatically, so the (identical, long) taste-profile -//! system prompt must come first in every request: cached input is $0.0028/M vs -//! $0.14/M. +//! Two transports speak to the wire directly through the shared `reqwest` +//! client (no vendor SDK; implementation notes, cross-cutting item 5 is stale): //! -//! **Why not `async-openai`** (spec §2 crate table): the published crate exposes -//! neither `Client` nor `types::chat` under any feature combination we could get -//! to build here, and it would drag in a second HTTP stack besides the shared -//! `reqwest` client (notes §4). [`DeepseekBackend`] therefore speaks the same -//! OpenAI-compatible wire protocol directly — about 80 lines, no new dependency, -//! and the request/response shapes are pinned by this module's tests. The -//! dependency was dropped from `Cargo.toml`; swapping a vendor SDK back in later -//! is a single [`ChatBackend`] impl and nothing else moves. +//! - [`DeepseekBackend`]: the OpenAI-compatible chat-completions endpoint. The +//! system prompt is the first message so DeepSeek's prefix cache hits. +//! - [`AnthropicBackend`]: `POST /v1/messages` with the system prompt as one +//! `cache_control: ephemeral` block, `output_config.effort`, and server-side +//! `fallbacks: "default"` (§4.2). No sampling parameters, no `thinking`, no +//! prefill — Opus 5 rejects them. A `stop_reason: "refusal"` (HTTP 200) is +//! [`LlmError::Refusal`], which the callers use to fall back to the bulk client. //! -//! Every call in the project goes through [`LlmClient`], which +//! Every call goes through [`LlmClient`], which sends the byte-identical system +//! prompt on every request, folds token usage into a per-provider [`UsageMeter`] +//! priced by a [`PriceTable`], and refuses further work once that provider's +//! `max_daily_usd` is spent. [`Llms`] pairs the bulk and editor clients. //! -//! 1. always sends [`LlmClient::system_prompt`] as the **first** message, byte for -//! byte identical across requests (that is what makes the prefix cache hit), -//! 2. folds the response's token usage into a shared [`UsageMeter`], and -//! 3. refuses further work once `max_daily_usd` has been spent (§3.6 guardrail). -//! -//! The network is reached through a [`ChatBackend`] so tests can inject canned -//! responses ([`MockBackend`]) without touching the wire (notes §6). +//! Tests inject [`MockBackend`] or a loopback `axum` listener; nothing here +//! touches the network under test. use std::future::Future; use std::pin::Pin; @@ -32,100 +27,150 @@ use std::sync::{Arc, Mutex}; use serde::Deserialize; use serde_json::json; -use crate::config::DeepseekConfig; +use crate::config::{AnthropicConfig, DeepseekConfig}; use crate::http::RetryPolicy; use crate::types::TokenUsage; -/// `response_format` value used for every structured call (§3.6). pub const JSON_OBJECT: &str = "json_object"; +const DEEPSEEK_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(180); +const ANTHROPIC_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(300); +const ANTHROPIC_VERSION: &str = "2023-06-01"; +const ANTHROPIC_BETA: &str = "server-side-fallback-2026-07-01"; #[derive(Debug, thiserror::Error)] pub enum LlmError { - #[error("deepseek api key is not configured (set DAILY_EPUB_DEEPSEEK__API_KEY)")] - MissingApiKey, - #[error("deepseek request failed: {0}")] - Api(String), - /// A 5xx/429/network failure: worth retrying (crate table "retry"). - #[error("deepseek request failed (transient): {0}")] - Transient(String), - #[error("deepseek returned an empty completion")] - EmptyResponse, - #[error("deepseek returned unparseable JSON: {0}")] + #[error("{provider} api key is not configured (set {env_var})")] + MissingApiKey { + provider: &'static str, + env_var: &'static str, + }, + #[error("{provider} request failed: {message}")] + Api { + provider: &'static str, + message: String, + }, + #[error("{provider} request failed (transient): {message}")] + Transient { + provider: &'static str, + message: String, + }, + #[error("{provider} returned a refusal")] + Refusal { provider: &'static str }, + #[error("{provider} returned an empty completion")] + EmptyResponse { provider: &'static str }, + #[error("llm returned unparseable JSON: {0}")] Json(#[from] serde_json::Error), - /// The `max_daily_usd` ceiling was reached: callers must skip remaining - /// editorial calls and fall back to feed excerpts, loudly (§3.6). #[error("daily cost ceiling of ${limit:.2} reached (spent ${spent:.4})")] BudgetExceeded { spent: f64, limit: f64 }, } impl LlmError { - /// True for failures the [`RetryPolicy`] should retry. pub fn is_transient(&self) -> bool { - matches!(self, LlmError::Transient(_)) + matches!(self, LlmError::Transient { .. }) + } + + fn api(provider: &'static str, message: impl Into) -> Self { + Self::Api { + provider, + message: message.into(), + } + } + + fn transient(provider: &'static str, message: impl Into) -> Self { + Self::Transient { + provider, + message: message.into(), + } } } -// --------------------------------------------------------------------------- -// Usage metering (§3.6 cost guardrail, notes §5) -// --------------------------------------------------------------------------- +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct PriceTable { + pub input_per_mtok: f64, + pub cache_write_per_mtok: f64, + pub cache_read_per_mtok: f64, + pub output_per_mtok: f64, +} + +impl PriceTable { + pub fn deepseek(cfg: &DeepseekConfig) -> Self { + Self { + input_per_mtok: cfg.price_input_per_mtok, + cache_write_per_mtok: 0.0, + cache_read_per_mtok: cfg.price_cached_input_per_mtok, + output_per_mtok: cfg.price_output_per_mtok, + } + } + + pub fn anthropic(cfg: &AnthropicConfig) -> Self { + Self { + input_per_mtok: cfg.price_input_per_mtok, + cache_write_per_mtok: cfg.price_cache_write_per_mtok, + cache_read_per_mtok: cfg.price_cache_read_per_mtok, + output_per_mtok: cfg.price_output_per_mtok, + } + } +} -/// Shared token/cost accumulator enforcing `max_daily_usd` (notes §5). -/// -/// Cloning shares the counters: one meter per run, cloned into every stage. #[derive(Debug, Clone)] pub struct UsageMeter { inner: Arc>, - /// Sticky: once the ceiling is crossed the run stays degraded (§3.6). exceeded: Arc, + prior_spend_usd: Arc>, limit_usd: f64, - price_input: f64, - price_cached: f64, - price_output: f64, + prices: PriceTable, } impl UsageMeter { + /// Compatibility constructor for the existing DeepSeek call sites. pub fn new(cfg: &DeepseekConfig, limit_usd: f64) -> Self { + Self::with_prices(PriceTable::deepseek(cfg), limit_usd) + } + + pub fn with_prices(prices: PriceTable, limit_usd: f64) -> Self { Self { inner: Arc::new(Mutex::new(TokenUsage::default())), exceeded: Arc::new(AtomicBool::new(false)), + prior_spend_usd: Arc::new(Mutex::new(0.0)), limit_usd, - price_input: cfg.price_input_per_mtok, - price_cached: cfg.price_cached_input_per_mtok, - price_output: cfg.price_output_per_mtok, + prices, } } - /// Seed the meter with spend already recorded for the day (§3.6): the - /// guardrail is a *daily* ceiling, not a per-run one. pub fn preload_cost(&self, spent_usd: f64) { - if spent_usd > 0.0 && self.limit_usd > 0.0 && spent_usd >= self.limit_usd { - self.trip("prior spend for today already exceeds the ceiling"); + let spent_usd = spent_usd.max(0.0); + match self.prior_spend_usd.lock() { + Ok(mut guard) => *guard = spent_usd, + Err(poisoned) => *poisoned.into_inner() = spent_usd, + } + if self.limit_usd > 0.0 && spent_usd >= self.limit_usd { + self.trip("prior spend for the run's UTC day reached the ceiling"); } } - /// Fold one response's usage in and return the running total. pub fn record(&self, usage: TokenUsage) -> TokenUsage { let total = match self.inner.lock() { Ok(mut guard) => { guard.add(usage); *guard } - // A poisoned mutex must not abort a run: accounting is advisory. Err(poisoned) => { let mut guard = poisoned.into_inner(); guard.add(usage); *guard } }; - let cost = self.cost_of(total); + let spent = self.spent_usd(); tracing::debug!( input = usage.input_tokens, - cached = usage.cached_tokens, + cache_write = usage.cache_write_tokens, + cache_read = usage.cached_tokens, output = usage.output_tokens, - total_cost_usd = cost, + live_cost_usd = self.cost_usd(), + day_spend_usd = spent, "recorded llm usage" ); - if self.limit_usd > 0.0 && cost > self.limit_usd && !self.exceeded.load(Ordering::SeqCst) { + if self.limit_usd > 0.0 && spent > self.limit_usd { self.trip("token spend crossed the ceiling"); } total @@ -134,10 +179,9 @@ impl UsageMeter { fn trip(&self, why: &str) { self.exceeded.store(true, Ordering::SeqCst); tracing::error!( - spent_usd = self.cost_usd(), + spent_usd = self.spent_usd(), limit_usd = self.limit_usd, - "LLM budget exceeded ({why}): remaining editorial calls will be skipped \ - and feed excerpts used instead" + "LLM budget exceeded ({why}); remaining calls for this provider are skipped" ); } @@ -148,29 +192,40 @@ impl UsageMeter { } } - fn cost_of(&self, usage: TokenUsage) -> f64 { - usage.cost_usd(self.price_input, self.price_cached, self.price_output) + pub fn cost_of(&self, usage: TokenUsage) -> f64 { + usage.cost_usd( + self.prices.input_per_mtok, + self.prices.cache_write_per_mtok, + self.prices.cache_read_per_mtok, + self.prices.output_per_mtok, + ) } pub fn cost_usd(&self) -> f64 { self.cost_of(self.total()) } + pub fn spent_usd(&self) -> f64 { + let prior = match self.prior_spend_usd.lock() { + Ok(guard) => *guard, + Err(poisoned) => *poisoned.into_inner(), + }; + prior + self.cost_usd() + } + pub fn limit_usd(&self) -> f64 { self.limit_usd } - /// True once the ceiling has been crossed — editorial stages check this and - /// silently degrade to excerpts (§3.6). pub fn budget_exceeded(&self) -> bool { self.exceeded.load(Ordering::SeqCst) } - /// `Err(BudgetExceeded)` once the run has spent more than `max_daily_usd` (§3.6). pub fn check_budget(&self) -> Result<(), LlmError> { - if self.budget_exceeded() { + if self.budget_exceeded() || (self.limit_usd > 0.0 && self.spent_usd() >= self.limit_usd) { + self.exceeded.store(true, Ordering::SeqCst); return Err(LlmError::BudgetExceeded { - spent: self.cost_usd(), + spent: self.spent_usd(), limit: self.limit_usd, }); } @@ -178,23 +233,16 @@ impl UsageMeter { } } -// --------------------------------------------------------------------------- -// Backend abstraction (notes §6: no network in tests) -// --------------------------------------------------------------------------- - -/// One chat completion request. The system prompt is an [`Arc`] so that the -/// identical bytes are reused for every call (DeepSeek prefix caching, §3.6). #[derive(Debug, Clone)] pub struct ChatRequest { pub model: String, pub system: Arc, pub user: String, pub temperature: f32, - /// Ask for `response_format: {"type": "json_object"}` (§3.6). pub json: bool, + pub effort: Option, } -/// One chat completion response, reduced to what the pipeline needs. #[derive(Debug, Clone, Default)] pub struct ChatCompletion { pub content: String, @@ -203,15 +251,10 @@ pub struct ChatCompletion { type BoxFuture<'a, T> = Pin + Send + 'a>>; -/// The seam between [`LlmClient`] and the network (notes §6). pub trait ChatBackend: std::fmt::Debug + Send + Sync { fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result>; } -/// LLM calls are slow; the shared 10s HTTP timeout would kill them (notes §4). -const LLM_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(180); - -/// The real thing: the OpenAI-compatible endpoint at `deepseek.base_url` (§3.6). #[derive(Debug, Clone)] pub struct DeepseekBackend { http: reqwest::Client, @@ -225,11 +268,14 @@ impl DeepseekBackend { .api_key .as_deref() .map(str::trim) - .filter(|k| !k.is_empty()) - .ok_or(LlmError::MissingApiKey)? + .filter(|key| !key.is_empty()) + .ok_or(LlmError::MissingApiKey { + provider: "deepseek", + env_var: "DAILY_EPUB_DEEPSEEK__API_KEY", + })? .to_string(); - let http = crate::http::build_client(LLM_TIMEOUT) - .map_err(|e| LlmError::Api(format!("building the deepseek http client: {e}")))?; + let http = crate::http::build_client(DEEPSEEK_TIMEOUT) + .map_err(|error| LlmError::api("deepseek", format!("building http client: {error}")))?; Ok(Self { http, endpoint: format!("{}/chat/completions", cfg.base_url.trim_end_matches('/')), @@ -244,7 +290,6 @@ impl ChatBackend for DeepseekBackend { let mut body = json!({ "model": req.model, "messages": [ - // FIRST and byte-identical across every request: prefix cache (§3.6). {"role": "system", "content": req.system.as_str()}, {"role": "user", "content": req.user}, ], @@ -252,11 +297,10 @@ impl ChatBackend for DeepseekBackend { "stream": false, }); if req.json - && let Some(obj) = body.as_object_mut() + && let Some(object) = body.as_object_mut() { - obj.insert("response_format".into(), json!({"type": JSON_OBJECT})); + object.insert("response_format".into(), json!({"type": JSON_OBJECT})); } - let response = self .http .post(&self.endpoint) @@ -264,60 +308,50 @@ impl ChatBackend for DeepseekBackend { .json(&body) .send() .await - .map_err(classify_reqwest_error)?; - + .map_err(|error| classify_reqwest_error("deepseek", error))?; let status = response.status(); if !status.is_success() { - let detail = response.text().await.unwrap_or_default(); - let detail = detail.chars().take(500).collect::(); - let msg = format!("{status}: {detail}"); - return Err(if status.is_server_error() || status.as_u16() == 429 { - LlmError::Transient(msg) - } else { - LlmError::Api(msg) - }); + return Err(classify_status("deepseek", status, response).await); } - - let parsed: ApiResponse = response.json().await.map_err(|e| { - LlmError::Api(format!("decoding the deepseek chat completion: {e}")) + let parsed: DeepseekResponse = response.json().await.map_err(|error| { + LlmError::api("deepseek", format!("decoding chat completion: {error}")) })?; let content = parsed .choices .into_iter() .next() - .and_then(|c| c.message.content) - .filter(|c| !c.trim().is_empty()) - .ok_or(LlmError::EmptyResponse)?; - let usage = parsed.usage.map(usage_from_api).unwrap_or_default(); + .and_then(|choice| choice.message.content) + .filter(|content| !content.trim().is_empty()) + .ok_or(LlmError::EmptyResponse { + provider: "deepseek", + })?; + let usage = parsed.usage.map(deepseek_usage).unwrap_or_default(); Ok(ChatCompletion { content, usage }) }) } } -/// The slice of the chat-completions response we consume. #[derive(Debug, Deserialize)] -struct ApiResponse { +struct DeepseekResponse { #[serde(default)] - choices: Vec, + choices: Vec, #[serde(default)] - usage: Option, + usage: Option, } #[derive(Debug, Deserialize)] -struct ApiChoice { - message: ApiMessage, +struct DeepseekChoice { + message: DeepseekMessage, } #[derive(Debug, Deserialize)] -struct ApiMessage { +struct DeepseekMessage { #[serde(default)] content: Option, } -/// DeepSeek reports cache hits both OpenAI-style (`prompt_tokens_details`) and -/// natively (`prompt_cache_hit_tokens`); we accept either (§3.6 pricing). #[derive(Debug, Default, Deserialize)] -struct ApiUsage { +struct DeepseekUsage { #[serde(default)] prompt_tokens: i64, #[serde(default)] @@ -325,95 +359,260 @@ struct ApiUsage { #[serde(default)] prompt_cache_hit_tokens: Option, #[serde(default)] - prompt_tokens_details: Option, + prompt_tokens_details: Option, } #[derive(Debug, Default, Deserialize)] -struct ApiPromptTokensDetails { +struct PromptTokenDetails { #[serde(default)] cached_tokens: Option, } -/// Split `prompt_tokens` into cache-miss and cache-hit halves (§3.6 pricing). -fn usage_from_api(u: ApiUsage) -> TokenUsage { - let cached = u +fn deepseek_usage(usage: DeepseekUsage) -> TokenUsage { + let cached = usage .prompt_tokens_details .as_ref() - .and_then(|d| d.cached_tokens) - .or(u.prompt_cache_hit_tokens) + .and_then(|details| details.cached_tokens) + .or(usage.prompt_cache_hit_tokens) .unwrap_or(0) .max(0); - let prompt = u.prompt_tokens.max(0); + let prompt = usage.prompt_tokens.max(0); let cached = cached.min(prompt); TokenUsage { input_tokens: prompt - cached, cached_tokens: cached, - output_tokens: u.completion_tokens.max(0), + cache_write_tokens: 0, + output_tokens: usage.completion_tokens.max(0), } } -fn classify_reqwest_error(err: reqwest::Error) -> LlmError { - if crate::http::is_retryable(&err) { - LlmError::Transient(err.to_string()) +#[derive(Debug, Clone)] +pub struct AnthropicBackend { + http: reqwest::Client, + endpoint: String, + api_key: String, +} + +impl AnthropicBackend { + pub fn new(cfg: &AnthropicConfig) -> Result { + let api_key = cfg + .api_key + .as_deref() + .map(str::trim) + .filter(|key| !key.is_empty()) + .ok_or(LlmError::MissingApiKey { + provider: "anthropic", + env_var: "DAILY_EPUB_ANTHROPIC__API_KEY", + })? + .to_string(); + let http = crate::http::build_client(ANTHROPIC_TIMEOUT).map_err(|error| { + LlmError::api("anthropic", format!("building http client: {error}")) + })?; + Ok(Self { + http, + endpoint: format!("{}/v1/messages", cfg.base_url.trim_end_matches('/')), + api_key, + }) + } +} + +impl ChatBackend for AnthropicBackend { + fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result> { + Box::pin(async move { + let body = json!({ + "model": req.model, + "max_tokens": 16_000, + "system": [{ + "type": "text", + "text": req.system.as_str(), + "cache_control": {"type": "ephemeral"}, + }], + "messages": [{"role": "user", "content": req.user}], + "output_config": {"effort": req.effort.as_deref().unwrap_or("high")}, + "fallbacks": "default", + }); + let response = self + .http + .post(&self.endpoint) + .header("x-api-key", &self.api_key) + .header("anthropic-version", ANTHROPIC_VERSION) + .header("anthropic-beta", ANTHROPIC_BETA) + .header(reqwest::header::CONTENT_TYPE, "application/json") + .json(&body) + .send() + .await + .map_err(|error| classify_reqwest_error("anthropic", error))?; + let status = response.status(); + if !status.is_success() { + return Err(classify_status("anthropic", status, response).await); + } + let parsed: AnthropicResponse = response.json().await.map_err(|error| { + LlmError::api("anthropic", format!("decoding messages response: {error}")) + })?; + if parsed.stop_reason.as_deref() == Some("refusal") { + return Err(LlmError::Refusal { + provider: "anthropic", + }); + } + let content = parsed + .content + .into_iter() + .filter(|block| block.kind == "text") + .filter_map(|block| block.text) + .collect::>() + .join(""); + if content.trim().is_empty() { + return Err(LlmError::EmptyResponse { + provider: "anthropic", + }); + } + Ok(ChatCompletion { + content, + usage: anthropic_usage(parsed.usage), + }) + }) + } +} + +#[derive(Debug, Deserialize)] +struct AnthropicResponse { + #[serde(default)] + content: Vec, + #[serde(default)] + stop_reason: Option, + #[serde(default)] + usage: AnthropicUsage, +} + +#[derive(Debug, Deserialize)] +struct AnthropicContent { + #[serde(rename = "type")] + kind: String, + #[serde(default)] + text: Option, +} + +#[derive(Debug, Default, Deserialize)] +struct AnthropicUsage { + #[serde(default)] + input_tokens: i64, + #[serde(default)] + cache_creation_input_tokens: i64, + #[serde(default)] + cache_read_input_tokens: i64, + #[serde(default)] + output_tokens: i64, +} + +fn anthropic_usage(usage: AnthropicUsage) -> TokenUsage { + TokenUsage { + input_tokens: usage.input_tokens.max(0), + cached_tokens: usage.cache_read_input_tokens.max(0), + cache_write_tokens: usage.cache_creation_input_tokens.max(0), + output_tokens: usage.output_tokens.max(0), + } +} + +async fn classify_status( + provider: &'static str, + status: reqwest::StatusCode, + response: reqwest::Response, +) -> LlmError { + let detail = response.text().await.unwrap_or_default(); + let message = format!("{status}: {}", detail.chars().take(500).collect::()); + if status.is_server_error() || status == reqwest::StatusCode::TOO_MANY_REQUESTS { + LlmError::transient(provider, message) } else { - LlmError::Api(err.to_string()) + LlmError::api(provider, message) } } -// --------------------------------------------------------------------------- -// Client -// --------------------------------------------------------------------------- +fn classify_reqwest_error(provider: &'static str, error: reqwest::Error) -> LlmError { + if crate::http::is_retryable(&error) { + LlmError::transient(provider, error.to_string()) + } else { + LlmError::api(provider, error.to_string()) + } +} -/// Every LLM call in the project goes through this client (notes §5). #[derive(Debug, Clone)] pub struct LlmClient { - /// The taste profile, sent as the first (cacheable) system message (§3.6). + pub provider: &'static str, pub system_prompt: Arc, pub model: String, + pub effort: Option, pub meter: UsageMeter, backend: Arc, retry: RetryPolicy, } impl LlmClient { - /// Build against the configured base URL; fails without an API key. pub fn new( cfg: &DeepseekConfig, system_prompt: String, meter: UsageMeter, ) -> Result { let backend = DeepseekBackend::new(cfg)?; - tracing::debug!( - base_url = %cfg.base_url, - model = %cfg.model, - system_prompt_chars = system_prompt.len(), - "deepseek client ready" - ); - Ok(Self::with_backend( + Ok(Self::with_backend_options( + "deepseek", &cfg.model, system_prompt, + None, + meter, + Arc::new(backend), + )) + } + + pub fn new_anthropic( + cfg: &AnthropicConfig, + system_prompt: String, + meter: UsageMeter, + ) -> Result { + let backend = AnthropicBackend::new(cfg)?; + Ok(Self::with_backend_options( + "anthropic", + &cfg.model, + system_prompt, + Some(cfg.effort.clone()), meter, Arc::new(backend), )) } - /// Construct around an arbitrary backend — the seam used by tests (notes §6). pub fn with_backend( model: &str, system_prompt: String, meter: UsageMeter, backend: Arc, + ) -> Self { + Self::with_backend_options("mock", model, system_prompt, None, meter, backend) + } + + pub fn with_backend_options( + provider: &'static str, + model: &str, + system_prompt: String, + effort: Option, + meter: UsageMeter, + backend: Arc, ) -> Self { Self { + provider, system_prompt: Arc::new(system_prompt), model: model.to_string(), + effort, meter, backend, retry: RetryPolicy::default(), } } - /// Raw completion: budget check → retry loop → usage accounting. + #[cfg(test)] + fn with_retry(mut self, retry: RetryPolicy) -> Self { + self.retry = retry; + self + } + pub async fn complete( &self, user_prompt: &str, @@ -421,25 +620,26 @@ impl LlmClient { json: bool, ) -> Result { self.meter.check_budget()?; - let req = ChatRequest { + let request = ChatRequest { model: self.model.clone(), system: Arc::clone(&self.system_prompt), user: user_prompt.to_string(), temperature, json, + effort: self.effort.clone(), }; let completion = self .retry - .run("deepseek chat completion", LlmError::is_transient, || { - self.backend.complete(req.clone()) - }) + .run( + &format!("{} chat completion", self.provider), + LlmError::is_transient, + || self.backend.complete(request.clone()), + ) .await?; self.meter.record(completion.usage); Ok(completion.content) } - /// One chat completion returning parsed JSON of type `T`, with the system - /// prompt first and `response_format: json_object` (§3.6). pub async fn complete_json( &self, user_prompt: &str, @@ -447,20 +647,20 @@ impl LlmClient { ) -> Result { let raw = self.complete(user_prompt, temperature, true).await?; let cleaned = strip_code_fence(&raw); - match serde_json::from_str::(cleaned) { - Ok(v) => Ok(v), - Err(e) => { + match serde_json::from_str(cleaned) { + Ok(value) => Ok(value), + Err(error) => { tracing::warn!( - error = %e, + provider = self.provider, + %error, preview = %cleaned.chars().take(400).collect::(), - "deepseek returned malformed JSON" + "llm returned malformed JSON" ); - Err(LlmError::Json(e)) + Err(LlmError::Json(error)) } } } - /// One plain-text completion (used for the front page / intros) (§3.6). pub async fn complete_text( &self, user_prompt: &str, @@ -470,7 +670,66 @@ impl LlmClient { } } -/// Models occasionally wrap JSON in ```` ```json ```` fences despite `json_object`. +/// The two provider clients the pipeline works with (§4.2). +/// +/// Both share the exact same system prompt string (§8.4). Each has its own +/// [`UsageMeter`] with its own price table and `max_daily_usd` (§5). +#[derive(Debug, Clone, Default)] +pub struct Llms { + /// DeepSeek — scoring, and the fallback for every editor call. + pub bulk: Option, + /// Claude — selection, summaries, the brief, the profile rebuild. + pub editor: Option, +} + +impl Llms { + /// Build both clients from config with one shared system prompt. + /// + /// A missing key or `anthropic.enabled = false` leaves that slot `None` with + /// a log line; nothing here is fatal because the paper always publishes (§17). + pub fn from_config( + deepseek: &DeepseekConfig, + anthropic: &AnthropicConfig, + system_prompt: String, + bulk_meter: UsageMeter, + editor_meter: UsageMeter, + ) -> Self { + let bulk = match LlmClient::new(deepseek, system_prompt.clone(), bulk_meter) { + Ok(client) => Some(client), + Err(error) => { + tracing::warn!(%error, "DeepSeek (bulk) is unavailable"); + None + } + }; + let editor = if anthropic.enabled { + match LlmClient::new_anthropic(anthropic, system_prompt, editor_meter) { + Ok(client) => Some(client), + Err(error) => { + tracing::warn!(%error, "Anthropic (editor) is unavailable; editor work falls back to bulk"); + None + } + } + } else { + tracing::info!("anthropic.enabled = false; editor work runs on the bulk provider"); + None + }; + Self { bulk, editor } + } + + /// The editor when configured and its meter is not tripped, else bulk. + pub fn editor_or_bulk(&self) -> Option<&LlmClient> { + self.editor + .as_ref() + .filter(|client| !client.meter.budget_exceeded()) + .or(self.bulk.as_ref()) + } + + /// True when no provider at all is available (`--skip-llm` or no keys). + pub fn is_empty(&self) -> bool { + self.bulk.is_none() && self.editor.is_none() + } +} + pub fn strip_code_fence(raw: &str) -> &str { let trimmed = raw.trim(); let Some(rest) = trimmed.strip_prefix("```") else { @@ -483,15 +742,9 @@ pub fn strip_code_fence(raw: &str) -> &str { .trim() } -// --------------------------------------------------------------------------- -// Test backend -// --------------------------------------------------------------------------- - -/// Canned-response backend for tests: pops scripted replies in order (notes §6). #[derive(Debug, Default)] pub struct MockBackend { - scripted: Mutex>>, - /// Every prompt the code under test sent, in order. + scripted: Mutex>>, pub seen: Mutex>, } @@ -500,44 +753,49 @@ impl MockBackend { Self::default() } - /// Queue a successful reply carrying `usage` tokens. pub fn push(&self, content: impl Into, usage: TokenUsage) { - if let Ok(mut q) = self.scripted.lock() { - q.push_back(Ok(ChatCompletion { + if let Ok(mut queue) = self.scripted.lock() { + queue.push_back(Ok(ChatCompletion { content: content.into(), usage, })); } } - /// Queue a permanent (non-retryable) failure. pub fn push_error(&self, message: impl Into) { - if let Ok(mut q) = self.scripted.lock() { - q.push_back(Err(message.into())); + self.push_llm_error(LlmError::api("mock", message)); + } + + pub fn push_llm_error(&self, error: LlmError) { + if let Ok(mut queue) = self.scripted.lock() { + queue.push_back(Err(error)); } } pub fn calls(&self) -> usize { - self.seen.lock().map(|s| s.len()).unwrap_or(0) + self.seen.lock().map(|seen| seen.len()).unwrap_or(0) } pub fn prompts(&self) -> Vec { - self.seen.lock().map(|s| s.clone()).unwrap_or_default() + self.seen + .lock() + .map(|seen| seen.clone()) + .unwrap_or_default() } } impl ChatBackend for MockBackend { fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result> { Box::pin(async move { - let next = self.scripted.lock().ok().and_then(|mut q| q.pop_front()); + let next = self + .scripted + .lock() + .ok() + .and_then(|mut queue| queue.pop_front()); if let Ok(mut seen) = self.seen.lock() { seen.push(req); } - match next { - Some(Ok(c)) => Ok(c), - Some(Err(msg)) => Err(LlmError::Api(msg)), - None => Err(LlmError::Api("mock backend ran out of responses".into())), - } + next.unwrap_or_else(|| Err(LlmError::api("mock", "mock backend ran out of responses"))) }) } } @@ -545,6 +803,13 @@ impl ChatBackend for MockBackend { #[cfg(test)] mod tests { use super::*; + use std::collections::VecDeque; + use std::time::Duration; + + use axum::extract::State; + use axum::http::{HeaderMap, StatusCode}; + use axum::routing::post; + use axum::{Json, Router}; fn cfg() -> DeepseekConfig { DeepseekConfig::default() @@ -554,10 +819,15 @@ mod tests { TokenUsage { input_tokens: input, cached_tokens: cached, + cache_write_tokens: 0, output_tokens: output, } } + // ----------------------------------------------------------------------- + // Meter and pricing + // ----------------------------------------------------------------------- + #[test] fn meter_accumulates_and_prices() { let meter = UsageMeter::new(&cfg(), 2.0); @@ -567,11 +837,26 @@ mod tests { assert_eq!(total.input_tokens, 1_000_000); assert_eq!(total.cached_tokens, 1_000_000); assert_eq!(total.output_tokens, 1_000_000); + // The DeepSeek path prices exactly as before the cache-write counter. assert!((meter.cost_usd() - 0.4228).abs() < 1e-9); assert!(!meter.budget_exceeded()); assert!(meter.check_budget().is_ok()); } + #[test] + fn anthropic_price_table_charges_cache_reads_and_writes() { + let meter = + UsageMeter::with_prices(PriceTable::anthropic(&AnthropicConfig::default()), 100.0); + meter.record(TokenUsage { + input_tokens: 1_000_000, + cached_tokens: 1_000_000, + cache_write_tokens: 1_000_000, + output_tokens: 1_000_000, + }); + // 5 + 0.5 + 6.25 + 25 + assert!((meter.cost_usd() - 36.75).abs() < 1e-9); + } + #[test] fn meter_trips_the_budget_flag_and_stays_tripped() { // Ceiling of $0.10; 1M cache-miss input tokens costs $0.14. @@ -591,31 +876,50 @@ mod tests { let meter = UsageMeter::new(&cfg(), 1.0); meter.preload_cost(0.5); assert!(!meter.budget_exceeded()); + assert!((meter.spent_usd() - 0.5).abs() < 1e-9); meter.preload_cost(1.5); assert!(meter.budget_exceeded()); } #[test] - fn usage_split_uses_prompt_token_details() { - let u: ApiUsage = serde_json::from_str( + fn deepseek_usage_split_uses_prompt_token_details() { + let u: DeepseekUsage = serde_json::from_str( r#"{"prompt_tokens": 1000, "completion_tokens": 120, "total_tokens": 1120, "prompt_tokens_details": {"cached_tokens": 800}}"#, ) .expect("fixture usage"); - assert_eq!(usage_from_api(u), tokens(200, 800, 120)); + assert_eq!(deepseek_usage(u), tokens(200, 800, 120)); } #[test] - fn usage_falls_back_to_deepseek_native_cache_fields() { - let u: ApiUsage = serde_json::from_str( + fn deepseek_usage_falls_back_to_native_cache_fields() { + let u: DeepseekUsage = serde_json::from_str( r#"{"prompt_tokens": 500, "completion_tokens": 40, "prompt_cache_hit_tokens": 448, "prompt_cache_miss_tokens": 52}"#, ) .expect("fixture usage"); - assert_eq!(usage_from_api(u), tokens(52, 448, 40)); + assert_eq!(deepseek_usage(u), tokens(52, 448, 40)); // Missing usage is not an error, just zero. - let empty: ApiUsage = serde_json::from_str("{}").expect("empty usage"); - assert_eq!(usage_from_api(empty), TokenUsage::default()); + let empty: DeepseekUsage = serde_json::from_str("{}").expect("empty usage"); + assert_eq!(deepseek_usage(empty), TokenUsage::default()); + } + + #[test] + fn anthropic_usage_maps_cache_fields() { + let u: AnthropicUsage = serde_json::from_str( + r#"{"input_tokens": 120, "cache_creation_input_tokens": 3000, + "cache_read_input_tokens": 0, "output_tokens": 800}"#, + ) + .expect("fixture usage"); + assert_eq!( + anthropic_usage(u), + TokenUsage { + input_tokens: 120, + cached_tokens: 0, + cache_write_tokens: 3000, + output_tokens: 800, + } + ); } #[test] @@ -625,6 +929,10 @@ mod tests { assert_eq!(strip_code_fence("```\n{\"a\":1}\n```"), "{\"a\":1}"); } + // ----------------------------------------------------------------------- + // LlmClient over the mock backend + // ----------------------------------------------------------------------- + fn client(backend: Arc, limit: f64) -> LlmClient { LlmClient::with_backend( "deepseek-v4-flash", @@ -694,15 +1002,381 @@ mod tests { assert!(matches!(out, Err(LlmError::Json(_)))); } + #[tokio::test] + async fn refusals_are_not_retried_and_keep_their_variant() { + let backend = Arc::new(MockBackend::new()); + backend.push_llm_error(LlmError::Refusal { + provider: "anthropic", + }); + backend.push("{}", TokenUsage::default()); + let llm = client(Arc::clone(&backend), 2.0).with_retry(RetryPolicy { + max_attempts: 3, + base_delay: Duration::from_millis(1), + max_delay: Duration::from_millis(2), + }); + let err = llm.complete_text("x", 0.3).await.expect_err("refusal"); + assert!(matches!( + err, + LlmError::Refusal { + provider: "anthropic" + } + )); + assert_eq!(backend.calls(), 1, "a refusal is terminal for that client"); + } + #[test] - fn missing_api_key_is_reported() { - let cfg = DeepseekConfig { + fn missing_api_keys_name_their_provider() { + let deepseek = DeepseekConfig { api_key: Some(" ".into()), ..DeepseekConfig::default() }; + let err = DeepseekBackend::new(&deepseek).expect_err("blank key"); assert!(matches!( - DeepseekBackend::new(&cfg), - Err(LlmError::MissingApiKey) + err, + LlmError::MissingApiKey { + provider: "deepseek", + .. + } + )); + assert!(err.to_string().contains("DAILY_EPUB_DEEPSEEK__API_KEY")); + + let anthropic = AnthropicConfig::default(); + let err = AnthropicBackend::new(&anthropic).expect_err("no key"); + assert!(matches!( + err, + LlmError::MissingApiKey { + provider: "anthropic", + .. + } + )); + assert!(err.to_string().contains("DAILY_EPUB_ANTHROPIC__API_KEY")); + } + + // ----------------------------------------------------------------------- + // Llms + // ----------------------------------------------------------------------- + + fn mock_client(provider: &'static str, limit: f64) -> (LlmClient, Arc) { + let backend = Arc::new(MockBackend::new()); + let prices = if provider == "anthropic" { + PriceTable::anthropic(&AnthropicConfig::default()) + } else { + PriceTable::deepseek(&cfg()) + }; + let client = LlmClient::with_backend_options( + provider, + "model", + "SYSTEM".into(), + None, + UsageMeter::with_prices(prices, limit), + Arc::clone(&backend) as Arc, + ); + (client, backend) + } + + #[test] + fn editor_or_bulk_prefers_an_untripped_editor() { + let (bulk, _) = mock_client("deepseek", 2.0); + let (editor, _) = mock_client("anthropic", 3.0); + let llms = Llms { + bulk: Some(bulk), + editor: Some(editor), + }; + assert_eq!(llms.editor_or_bulk().map(|c| c.provider), Some("anthropic")); + // Trip the editor's meter: bulk takes over. + llms.editor + .as_ref() + .expect("editor") + .meter + .preload_cost(10.0); + assert_eq!(llms.editor_or_bulk().map(|c| c.provider), Some("deepseek")); + // No bulk and a tripped editor means no client at all. + let only_editor = Llms { + bulk: None, + editor: llms.editor.clone(), + }; + assert!(only_editor.editor_or_bulk().is_none()); + assert!(Llms::default().is_empty()); + assert!(Llms::default().editor_or_bulk().is_none()); + } + + #[test] + fn from_config_without_keys_yields_no_clients() { + let llms = Llms::from_config( + &cfg(), + &AnthropicConfig::default(), + "SYSTEM".into(), + UsageMeter::new(&cfg(), 1.0), + UsageMeter::with_prices(PriceTable::anthropic(&AnthropicConfig::default()), 1.0), + ); + assert!(llms.is_empty()); + } + + // ----------------------------------------------------------------------- + // AnthropicBackend against a loopback listener (§4.2, §20) + // ----------------------------------------------------------------------- + + #[derive(Clone, Default)] + struct FakeAnthropic { + seen: Arc>>, + scripted: Arc>>, + } + + impl FakeAnthropic { + fn push(&self, status: StatusCode, body: serde_json::Value) { + self.scripted + .lock() + .expect("script lock") + .push_back((status, body)); + } + + fn requests(&self) -> Vec<(HeaderMap, serde_json::Value)> { + self.seen.lock().expect("seen lock").clone() + } + } + + async fn handle( + State(fake): State, + headers: HeaderMap, + Json(body): Json, + ) -> (StatusCode, Json) { + fake.seen.lock().expect("seen lock").push((headers, body)); + let (status, body) = fake + .scripted + .lock() + .expect("script lock") + .pop_front() + .unwrap_or(( + StatusCode::INTERNAL_SERVER_ERROR, + json!({"error": "unscripted"}), + )); + (status, Json(body)) + } + + async fn serve(fake: FakeAnthropic) -> String { + let app = Router::new() + .route("/v1/messages", post(handle)) + .with_state(fake); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0") + .await + .expect("loopback listener"); + let addr = listener.local_addr().expect("local addr"); + tokio::spawn(async move { + let _ = axum::serve(listener, app).await; + }); + format!("http://{addr}") + } + + async fn anthropic_client(fake: FakeAnthropic, limit: f64) -> LlmClient { + let base_url = serve(fake).await; + let config = AnthropicConfig { + base_url, + api_key: Some("test-key-never-logged".into()), + effort: "medium".into(), + ..AnthropicConfig::default() + }; + LlmClient::new_anthropic( + &config, + "PROFILE SYSTEM PROMPT".into(), + UsageMeter::with_prices(PriceTable::anthropic(&config), limit), + ) + .expect("client") + .with_retry(RetryPolicy { + max_attempts: 3, + base_delay: Duration::from_millis(1), + max_delay: Duration::from_millis(2), + }) + } + + fn ok_message(text: &str, stop_reason: &str) -> serde_json::Value { + json!({ + "id": "msg_01", + "type": "message", + "role": "assistant", + "model": "claude-opus-5", + "content": [ + {"type": "thinking", "thinking": ""}, + {"type": "text", "text": text} + ], + "stop_reason": stop_reason, + "usage": { + "input_tokens": 1_000_000, + "cache_creation_input_tokens": 1_000_000, + "cache_read_input_tokens": 1_000_000, + "output_tokens": 1_000_000 + } + }) + } + + #[tokio::test] + async fn anthropic_request_has_the_documented_shape() { + let fake = FakeAnthropic::default(); + fake.push( + StatusCode::OK, + ok_message("```json\n{\"ok\": true}\n```", "end_turn"), + ); + let llm = anthropic_client(fake.clone(), 100.0).await; + + let out: serde_json::Value = llm + .complete_json("the task", 0.7) + .await + .expect("completion"); + assert_eq!(out, json!({"ok": true})); + + let requests = fake.requests(); + assert_eq!(requests.len(), 1); + let (headers, body) = &requests[0]; + assert_eq!( + headers.get("x-api-key").and_then(|v| v.to_str().ok()), + Some("test-key-never-logged") + ); + assert_eq!( + headers + .get("anthropic-version") + .and_then(|v| v.to_str().ok()), + Some("2023-06-01") + ); + assert_eq!( + headers.get("anthropic-beta").and_then(|v| v.to_str().ok()), + Some("server-side-fallback-2026-07-01") + ); + assert!( + headers + .get("content-type") + .and_then(|v| v.to_str().ok()) + .is_some_and(|v| v.starts_with("application/json")) + ); + + assert_eq!(body["model"], "claude-opus-5"); + assert_eq!(body["max_tokens"], 16_000); + assert_eq!(body["system"][0]["type"], "text"); + assert_eq!(body["system"][0]["text"], "PROFILE SYSTEM PROMPT"); + assert_eq!(body["system"][0]["cache_control"]["type"], "ephemeral"); + assert_eq!(body["messages"][0]["role"], "user"); + assert_eq!(body["messages"][0]["content"], "the task"); + assert_eq!(body["output_config"]["effort"], "medium"); + assert_eq!(body["fallbacks"], "default"); + for forbidden in [ + "temperature", + "top_p", + "top_k", + "thinking", + "response_format", + ] { + assert!( + body.get(forbidden).is_none(), + "{forbidden} must not be sent" + ); + } + assert_eq!( + body["messages"].as_array().map(Vec::len), + Some(1), + "no prefill" + ); + + // Usage was priced with the cache read/write rates: 5 + 6.25 + 0.5 + 25. + assert_eq!( + llm.meter.total(), + TokenUsage { + input_tokens: 1_000_000, + cached_tokens: 1_000_000, + cache_write_tokens: 1_000_000, + output_tokens: 1_000_000, + } + ); + assert!((llm.meter.cost_usd() - 36.75).abs() < 1e-9); + } + + #[tokio::test] + async fn anthropic_refusal_surfaces_as_the_fallback_error() { + let fake = FakeAnthropic::default(); + fake.push( + StatusCode::OK, + json!({ + "content": [], + "stop_reason": "refusal", + "stop_details": {"type": "refusal", "category": "cyber"}, + "usage": {"input_tokens": 0, "output_tokens": 0} + }), + ); + let llm = anthropic_client(fake.clone(), 100.0).await; + let err = llm.complete_text("x", 0.3).await.expect_err("refusal"); + assert!(matches!( + err, + LlmError::Refusal { + provider: "anthropic" + } + )); + assert!(!err.is_transient()); + assert_eq!(fake.requests().len(), 1, "a refusal is never retried"); + } + + #[tokio::test] + async fn anthropic_429_is_retried_but_400_is_not() { + let fake = FakeAnthropic::default(); + fake.push( + StatusCode::TOO_MANY_REQUESTS, + json!({"type": "error", "error": {"type": "rate_limit_error"}}), + ); + fake.push( + StatusCode::OK, + ok_message("{\"after\": \"retry\"}", "end_turn"), + ); + let llm = anthropic_client(fake.clone(), 100.0).await; + let text = llm.complete_text("x", 0.3).await.expect("second attempt"); + assert_eq!(text, "{\"after\": \"retry\"}"); + assert_eq!(fake.requests().len(), 2); + + let fake = FakeAnthropic::default(); + fake.push( + StatusCode::BAD_REQUEST, + json!({"type": "error", "error": {"type": "invalid_request_error", "message": "nope"}}), + ); + let llm = anthropic_client(fake.clone(), 100.0).await; + let err = llm.complete_text("x", 0.3).await.expect_err("400"); + assert!(matches!( + err, + LlmError::Api { + provider: "anthropic", + .. + } + )); + assert!(err.to_string().contains("400")); + assert_eq!(fake.requests().len(), 1, "400 is never retried"); + } + + #[tokio::test] + async fn anthropic_concatenates_text_blocks_and_rejects_empty_output() { + let fake = FakeAnthropic::default(); + fake.push( + StatusCode::OK, + json!({ + "content": [ + {"type": "text", "text": "{\"a\": "}, + {"type": "thinking", "thinking": "..."}, + {"type": "text", "text": "1}"} + ], + "stop_reason": "end_turn", + "usage": {"input_tokens": 10, "output_tokens": 5} + }), + ); + fake.push( + StatusCode::OK, + json!({ + "content": [{"type": "thinking", "thinking": ""}], + "stop_reason": "end_turn", + "usage": {"input_tokens": 10, "output_tokens": 0} + }), + ); + let llm = anthropic_client(fake.clone(), 100.0).await; + let out: serde_json::Value = llm.complete_json("x", 0.3).await.expect("joined"); + assert_eq!(out, json!({"a": 1})); + let err = llm.complete_text("y", 0.3).await.expect_err("empty"); + assert!(matches!( + err, + LlmError::EmptyResponse { + provider: "anthropic" + } )); } } diff --git a/src/curate/mod.rs b/src/curate/mod.rs index 93c4130..72487e8 100644 --- a/src/curate/mod.rs +++ b/src/curate/mod.rs @@ -6,8 +6,9 @@ //! //! [`Curator`] is the thin orchestration layer the `generate` pipeline calls; the //! interesting logic lives in the stage modules. Every stage is safe to run with -//! `llm == None` (`--skip-llm`): the prefilter order stands in for selection and -//! feed excerpts stand in for summaries (notes §6). +//! no provider at all (`--skip-llm`): the prefilter order stands in for selection +//! and feed excerpts stand in for summaries (notes §6). Scoring runs on the bulk +//! client; selection and editorial on the editor with per-call bulk fallback. pub mod editorial; pub mod llm; @@ -26,14 +27,15 @@ use crate::types::{Article, Editorial, Lineup, ScoredArticle}; pub struct Curator { pub config: Config, pub db: Db, - pub llm: Option, + pub llms: llm::Llms, } impl Curator { - /// `llm == None` corresponds to `--skip-llm`: prefilter order is used for - /// selection and feed excerpts stand in for summaries (notes §6). - pub fn new(config: Config, db: Db, llm: Option) -> Self { - Self { config, db, llm } + /// An empty [`llm::Llms`] corresponds to `--skip-llm`: prefilter order is + /// used for selection and feed excerpts stand in for summaries (notes §6). + /// With only `bulk`, every editor call runs on DeepSeek (§4.2). + pub fn new(config: Config, db: Db, llms: llm::Llms) -> Self { + Self { config, db, llms } } /// Heuristic pre-filter: 300–500 articles → `prefilter_keep` (§3.5). @@ -75,7 +77,7 @@ impl Curator { /// /// A no-op under `--skip-llm`. Scores are persisted per `(article, date)`. pub async fn score(&self, candidates: &mut [ScoredArticle], date: Date) -> anyhow::Result<()> { - let Some(llm) = self.llm.as_ref() else { + let Some(llm) = self.llms.bulk.as_ref() else { tracing::info!("--skip-llm: stage A scoring skipped"); return Ok(()); }; @@ -86,6 +88,7 @@ impl Curator { llm, candidates, self.config.deepseek.score_batch_size, + self.config.deepseek.max_concurrent_requests, &self.config.curation.sections, self.config.deepseek.score_temperature, ) @@ -116,23 +119,29 @@ impl Curator { date: Date, ) -> anyhow::Result { let sections = &self.config.curation.sections; - let target = self.config.target_article_count; - let Some(llm) = self.llm.as_ref() else { - tracing::info!("--skip-llm: selecting by prefilter order"); - return Ok(select::select_without_llm( - candidates, sections, target, date, - )); - }; - let span = tracing::info_span!("llm_select", candidates = candidates.len()); + let soft_target = self.config.target_article_count; + let hard_max = self.config.curation.max_article_count; + let span = tracing::info_span!("llm_editor", candidates = candidates.len()); let _guard = span.enter(); - - match select::select(llm, candidates.clone(), sections, target, date).await { + match select::select( + &self.llms, + candidates.clone(), + sections, + soft_target, + hard_max, + date, + ) + .await + { Ok(lineup) => Ok(lineup), - Err(e) => { - tracing::error!(error = %e, - "stage B selection failed; falling back to prefilter order"); + Err(error) => { + tracing::error!(%error, "editor and bulk fallback failed; selecting heuristically"); Ok(select::select_without_llm( - candidates, sections, target, date, + candidates, + sections, + soft_target, + hard_max, + date, )) } } @@ -142,13 +151,19 @@ impl Curator { /// /// Never fails the run: a budget trip or an API error degrades to excerpts. pub async fn editorial(&self, lineup: &Lineup) -> anyhow::Result { - let Some(llm) = self.llm.as_ref() else { + if self.llms.editor_or_bulk().is_none() { tracing::info!("--skip-llm: using feed excerpts as summaries"); return Ok(editorial::fallback_editorial(lineup)); - }; + } let span = tracing::info_span!("llm_editorial", picks = lineup.picks.len()); let _guard = span.enter(); - Ok(editorial::run(llm, lineup, self.config.deepseek.editorial_temperature).await) + Ok(editorial::run( + &self.llms, + lineup, + &self.config.editorial, + self.config.deepseek.editorial_temperature, + ) + .await) } } diff --git a/src/curate/score.rs b/src/curate/score.rs index a427f6f..27a80eb 100644 --- a/src/curate/score.rs +++ b/src/curate/score.rs @@ -10,6 +10,7 @@ use std::collections::HashMap; use std::fmt::Write as _; +use futures::{StreamExt, stream}; use serde::{Deserialize, Serialize}; use serde_json::Value; @@ -331,6 +332,7 @@ pub async fn score_all( llm: &LlmClient, candidates: &mut [ScoredArticle], batch_size: usize, + max_concurrent_requests: usize, sections: &[String], temperature: f32, ) -> Result { @@ -339,61 +341,47 @@ pub async fn score_all( } let batch_size = batch_size.max(1); let batches = candidates.len().div_ceil(batch_size); - let mut scores: HashMap = HashMap::with_capacity(candidates.len()); + let prompts = candidates + .chunks(batch_size) + .enumerate() + .map(|(index, batch)| (index, batch.len(), build_batch_prompt(batch, sections))) + .collect::>(); - for (n, batch) in candidates.chunks(batch_size).enumerate() { - if let Err(e) = llm.meter.check_budget() { - tracing::error!( - error = %e, - batch = n + 1, - of = batches, - unscored = candidates.len() - scores.len(), - "COST CEILING HIT during stage A scoring — remaining batches skipped; \ - the lineup will fall back to heuristic ranking for them" - ); - break; - } - let prompt = build_batch_prompt(batch, sections); - tracing::debug!( - batch = n + 1, - of = batches, - articles = batch.len(), - approx_tokens = super::approx_tokens(&prompt), - "stage A request" - ); - match llm.complete(&prompt, temperature, true).await { - Ok(raw) => { - let items = parse_score_response(&raw); - if items.is_empty() { - tracing::warn!( - batch = n + 1, - of = batches, - "stage A batch returned no scores" - ); - } - for item in items { - scores.insert(item.id, item.into()); - } - } - Err(e) => { - tracing::warn!(batch = n + 1, of = batches, error = %e, - "stage A batch failed; its articles stay unscored"); + let results = stream::iter(prompts) + .map(|(index, article_count, prompt)| async move { + if let Err(error) = llm.meter.check_budget() { + tracing::warn!(batch = index + 1, of = batches, %error, "bulk budget tripped; skipping stage A batch"); + return (index, Vec::new()); } + tracing::debug!(batch = index + 1, of = batches, articles = article_count, approx_tokens = super::approx_tokens(&prompt), "stage A request"); + let items = match llm.complete(&prompt, temperature, true).await { + Ok(raw) => parse_score_response(&raw), + Err(error) => { + tracing::warn!(batch = index + 1, of = batches, %error, "stage A batch failed; its articles stay unscored"); + Vec::new() + } + }; + (index, items) + }) + .buffer_unordered(max_concurrent_requests.max(1)) + .collect::>() + .await; + + let mut scores: HashMap = HashMap::with_capacity(candidates.len()); + for (_, items) in results { + for item in items { + scores.insert(item.id, item.into()); } } - - let mut applied = 0usize; - for candidate in candidates.iter_mut() { + let mut applied = 0; + for candidate in candidates { if let Some(score) = scores.remove(&candidate.article.id) { candidate.llm = Some(score); applied += 1; } } if !scores.is_empty() { - tracing::warn!( - unknown_ids = scores.len(), - "stage A returned scores for ids that were not in the batch" - ); + tracing::warn!(unknown_ids = scores.len(), "stage A returned unknown ids"); } Ok(applied) } @@ -535,7 +523,7 @@ mod tests { candidate(2, "Two", 1000), candidate(3, "Three", 1000), ]; - let scored = score_all(&llm, &mut candidates, 2, §ions(), 0.3) + let scored = score_all(&llm, &mut candidates, 2, 4, §ions(), 0.3) .await .expect("scoring"); assert_eq!(scored, 3); @@ -556,7 +544,7 @@ mod tests { ); let llm = client(Arc::clone(&backend), 2.0); let mut candidates = vec![candidate(1, "One", 900), candidate(2, "Two", 900)]; - let scored = score_all(&llm, &mut candidates, 1, §ions(), 0.3) + let scored = score_all(&llm, &mut candidates, 1, 4, §ions(), 0.3) .await .expect("scoring must not abort"); assert_eq!(scored, 1); @@ -573,6 +561,7 @@ mod tests { TokenUsage { input_tokens: 1_000_000, cached_tokens: 0, + cache_write_tokens: 0, output_tokens: 0, }, ); @@ -582,7 +571,7 @@ mod tests { ); let llm = client(Arc::clone(&backend), 0.05); let mut candidates = vec![candidate(1, "One", 900), candidate(2, "Two", 900)]; - let scored = score_all(&llm, &mut candidates, 1, §ions(), 0.3) + let scored = score_all(&llm, &mut candidates, 1, 4, §ions(), 0.3) .await .expect("scoring"); assert_eq!(scored, 1, "only the first batch ran"); diff --git a/src/curate/select.rs b/src/curate/select.rs index 5f7fa06..62d103b 100644 --- a/src/curate/select.rs +++ b/src/curate/select.rs @@ -1,13 +1,19 @@ -//! Stage B — lineup selection (spec §3.6). +//! The editor — lineup selection (plan §13). //! -//! One call: send the top ~40 candidates by [`ScoredArticle::combined_score`] with -//! their rationales; the model returns the final 15–25 picks, each with a section -//! from the configured palette, an ordering, and exactly one `lead_story`. +//! One call on the editor client (Claude), falling back to the same prompt on the +//! bulk client (DeepSeek), then to [`select_without_llm`]. The shortlist is the +//! top candidates by [`ScoredArticle::combined_score`] with their Stage A +//! rationales; the model returns picks, each with a section from the configured +//! palette, an ordering, exactly one `lead_story`, and a one-line `why` that is +//! printed under the headline. //! //! The model's answer is treated as a proposal, never as gospel: sections are //! validated against the palette, the lead is forced to be unique, auto-include -//! feeds are re-inserted if they were dropped, and the size is clamped to -//! `target_article_count ± 5`. +//! feeds are re-inserted if they were dropped, duplicate ids are dropped, and the +//! size is trimmed to `hard_max`. There is **no minimum**: a nine-pick answer is +//! published as nine (the "top up" branch is gone). `--max-articles N` is a +//! ceiling: `hard_max = min(curation.max_article_count, N)` and +//! `soft_target = min(target_article_count, hard_max)`. use std::collections::{BTreeMap, HashMap, HashSet}; use std::fmt::Write as _; @@ -15,19 +21,16 @@ use std::fmt::Write as _; use jiff::civil::Date; use serde::{Deserialize, Serialize}; -use super::llm::{LlmClient, LlmError, strip_code_fence}; +use super::llm::{LlmError, Llms, strip_code_fence}; use super::{prompt_text, truncate_words}; -use crate::types::{ArticleId, Lineup, Pick, ScoredArticle, SourceKind, WORLD_BRIEFING_SECTION}; +use crate::types::{ArticleId, Lineup, Pick, ScoredArticle, WORLD_BRIEFING_SECTION}; -/// How many candidates are offered to stage B (§3.6). +/// How many candidates are offered to the editor (§13; step 5 raises this to the diversified shortlist). pub const SHORTLIST_SIZE: usize = 40; -/// How far the final count may drift from `target_article_count` (§3.6: 15–25 -/// around a default target of 20). -pub const TARGET_TOLERANCE: usize = 5; -/// Words of lead-in text shown per candidate in the stage-B prompt. -const BLURB_WORDS: usize = 45; +/// Words of lead-in text shown per candidate in the editor prompt (§13). +const BLURB_WORDS: usize = 60; -/// One element of the stage-B JSON response (§3.6). +/// One element of the editor's JSON response (§13). #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct SelectionItem { pub id: ArticleId, @@ -36,6 +39,8 @@ pub struct SelectionItem { pub position: i64, #[serde(default)] pub lead_story: bool, + #[serde(default)] + pub why: Option, } /// Envelope the model is asked to return. @@ -45,49 +50,59 @@ pub struct SelectionResponse { pub picks: Vec, } -/// The invariant instruction block for stage B (§3.6). -pub const SELECT_INSTRUCTIONS: &str = "\ -TASK: assemble today's issue of The Daily EPUB from the shortlist below. +/// The invariant instruction block for the editor (§13), with `{soft_target}` and +/// `{hard_max}` substituted at render time. +pub const EDITOR_INSTRUCTIONS: &str = r#"TASK: assemble today's issue of The Daily EPUB from the shortlist below. -You are choosing what one specific reader — the profile in your system prompt — \ -will actually read on an e-ink screen over breakfast. Build a paper, not a \ -ranking: it should have a shape, a range of subjects, and a clear front page. +You are choosing what one specific reader — the profile, learned adjustments and +recent verdicts in your system prompt — will read on an e-ink screen over breakfast. +Build a paper, not a ranking: it should have a shape, a range of subjects, and a +clear front page. RULES -1. Pick articles by id from the shortlist only. Never invent an id. -2. Give every pick a section from the palette below, spelled exactly as given. -3. Number picks within each section from 1 upward, best first. -4. Flag exactly one pick as \"lead_story\": true — the day's strongest, most \ -substantial piece. It must sit in the first section you use. -5. Any candidate marked \"always-include\" MUST appear; place it in \"From the \ -Blogroll\" unless it clearly belongs elsewhere. -6. Do not select two articles that tell the same story; keep the better one. +1. Pick by id from the shortlist only. +2. Every pick gets a section from the palette, spelled exactly. +3. Number picks within a section from 1, best first. +4. Exactly one pick is "lead_story": true, in the first section you use. +5. Candidates flagged always-include MUST appear. +6. Never select two articles that tell the same story. +7. SIZE: aim for about {soft_target}; never more than {hard_max}; there is NO minimum. + If only nine pieces deserve the reader's morning, publish nine. Never pad. +8. For every pick write "why": at most 14 words, specific to this article and this + reader, in the second person is fine ("the Postgres failover story you'd argue with"). + It is printed under the headline. EDITORIAL JUDGEMENT -- Favour depth over coverage: a slim issue of excellent pieces beats a full one \ -padded with filler. Drop anything you would not defend. -- Mix the day up. Several long technical dives in a row is a bad breakfast; \ -alternate register and subject across sections. -- Keep the local and ultra-niche picks — a Boston story and a small-scene story \ -are worth more here than a third AI-industry item. -- Score is evidence, not an instruction: overrule it when the paper reads better \ -for it, and say so through your placement. -- Leave a section out entirely rather than padding it; empty sections are dropped. +- Depth over coverage. Drop anything you would not defend to him in person. +- Diversity is a feature: do not let one subject, one format, or one feed dominate, + even if it is what he has been loving lately. A paper of eight AI posts is a failure + even if each is good. The "recent verdicts" tell you his taste; they do not tell you + to repeat it. +- Keep the local and ultra-niche picks when they are good; they are worth more here + than a third industry item. +- Candidates flagged exploration were included on purpose to test the edges of his + taste; take one if it is genuinely good, ignore it otherwise. +- Scores are evidence, not instructions. Overrule them when the paper reads better. -Return JSON exactly in this shape and nothing else: -{\"picks\": [{\"id\": 123, \"section\": \"Top Stories\", \"position\": 1, \ -\"lead_story\": true}]}"; +Return JSON exactly: +{"picks": [{"id": 123, "section": "Top Stories", "position": 1, "lead_story": true, "why": "…"}]}"#; -/// Render the stage-B user prompt (§3.6). -pub fn build_prompt(shortlist: &[ScoredArticle], sections: &[String], target: usize) -> String { - let (min, max) = size_bounds(target); +/// Render the editor's user prompt (§13). +pub fn build_prompt( + shortlist: &[ScoredArticle], + sections: &[String], + soft_target: usize, + hard_max: usize, +) -> String { + let instructions = EDITOR_INSTRUCTIONS + .replace("{soft_target}", &soft_target.to_string()) + .replace("{hard_max}", &hard_max.to_string()); let mut prompt = String::with_capacity(2048 + shortlist.len() * 400); - prompt.push_str(SELECT_INSTRUCTIONS); + prompt.push_str(&instructions); let _ = write!( prompt, "\n\nSECTION PALETTE (exact strings, use only these): {}\n\ Reserved and unavailable: \"{WORLD_BRIEFING_SECTION}\" is compiled separately.\n\n\ - SIZE: choose {target} articles; never fewer than {min} and never more than {max}.\n\n\ SHORTLIST ({} candidates, best-ranked first)\n", sections.join(" | "), shortlist.len() @@ -124,7 +139,7 @@ fn render_candidate(candidate: &ScoredArticle) -> String { Some(llm) => { let _ = writeln!( block, - "score: {:.1} ({}) — {}", + "score: {:.1} · {} — {}", llm.score, if llm.category.is_empty() { "uncategorized" @@ -135,24 +150,19 @@ fn render_candidate(candidate: &ScoredArticle) -> String { ); } None => { - let _ = writeln!( - block, - "score: unscored (heuristic rank {:.0}/100)", - candidate.prefilter_score - ); + let _ = writeln!(block, "score: unscored"); } } - let _ = writeln!( - block, - "signals: social {:.2}; via {}{}", - candidate.social_score, - source_kinds(candidate), - if candidate.auto_include { - "; ALWAYS-INCLUDE" - } else { - "" - } - ); + let mut flags = Vec::new(); + if candidate.auto_include { + flags.push("always-include"); + } + if a.excerpt_only { + flags.push("excerpt only"); + } + if !flags.is_empty() { + let _ = writeln!(block, "flags: {}", flags.join(" | ")); + } let blurb = truncate_words(&prompt_text(&a.content_html), BLURB_WORDS); if !blurb.is_empty() { let _ = writeln!(block, "opening: {blurb}"); @@ -160,37 +170,6 @@ fn render_candidate(candidate: &ScoredArticle) -> String { block } -fn source_kinds(candidate: &ScoredArticle) -> String { - let mut kinds: Vec<&str> = candidate - .article - .sources - .iter() - .map(|s| match s.kind { - SourceKind::Scour => "scour", - SourceKind::HnFrontpage => "hn_frontpage", - SourceKind::Lobsters => "lobsters", - SourceKind::Reddit => "reddit", - SourceKind::Feed => "feed", - }) - .collect(); - kinds.sort_unstable(); - kinds.dedup(); - if kinds.is_empty() { - "feed".into() - } else { - kinds.join("+") - } -} - -/// `target ± TARGET_TOLERANCE`, floored at one article (§3.6). -pub fn size_bounds(target: usize) -> (usize, usize) { - let target = target.max(1); - ( - target.saturating_sub(TARGET_TOLERANCE).max(1), - target + TARGET_TOLERANCE, - ) -} - // --------------------------------------------------------------------------- // Section validation (§3.6: the model may only use the configured palette) // --------------------------------------------------------------------------- @@ -244,7 +223,7 @@ fn words_of(s: &str) -> HashSet { .collect() } -/// Section guess from feed metadata, used by `--skip-llm` and by top-ups (§3.6). +/// Section guess from feed metadata, used by [`select_without_llm`] (§3.6). pub fn heuristic_section(candidate: &ScoredArticle, sections: &[String]) -> String { if candidate.auto_include { return resolve_section("From the Blogroll", sections); @@ -346,7 +325,7 @@ pub fn heuristic_section(candidate: &ScoredArticle, sections: &[String]) -> Stri /// Keys the model might wrap the array in. const ARRAY_KEYS: &[&str] = &["picks", "lineup", "articles", "selection", "items"]; -/// Lenient parse of the stage-B response (§3.6). +/// Lenient parse of the editor response (§13). `why` is capped at 14 words. pub fn parse_selection_response(raw: &str) -> Vec { let cleaned = strip_code_fence(raw); let value: serde_json::Value = match serde_json::from_str(cleaned) { @@ -405,6 +384,16 @@ pub fn parse_selection_response(raw: &str) -> Vec { .or_else(|| v.as_str().map(|s| s.eq_ignore_ascii_case("true"))) }) .unwrap_or(false), + why: obj + .get("why") + .and_then(serde_json::Value::as_str) + .map(|why| { + why.split_whitespace() + .take(14) + .collect::>() + .join(" ") + }) + .filter(|why| !why.is_empty()), }); } out @@ -414,96 +403,131 @@ pub fn parse_selection_response(raw: &str) -> Vec { // Stage driver // --------------------------------------------------------------------------- -/// Ask the model for the day's lineup, validating that every section is from the -/// configured palette and exactly one pick is the lead (§3.6). +/// Ask the editor for the day's lineup (§13). +/// +/// Editor first, then the same prompt on the bulk client, then +/// [`select_without_llm`]; never an error unless a mock is misconfigured. pub async fn select( - llm: &LlmClient, + llms: &Llms, candidates: Vec, sections: &[String], - target: usize, + soft_target: usize, + hard_max: usize, date: Date, ) -> Result { if candidates.is_empty() { - tracing::warn!("stage B had no candidates"); return Ok(Lineup { date, picks: Vec::new(), section_order: Vec::new(), }); } - if let Err(e) = llm.meter.check_budget() { - tracing::error!(error = %e, - "COST CEILING HIT before stage B selection — falling back to heuristic ranking"); - return Ok(select_without_llm(candidates, sections, target, date)); - } - - let shortlist = shortlist(&candidates, target); - let prompt = build_prompt(&shortlist, sections, target); + let Some(primary) = llms.editor_or_bulk() else { + return Ok(select_without_llm( + candidates, + sections, + soft_target, + hard_max, + date, + )); + }; + let shortlist = shortlist(&candidates, hard_max); + let prompt = build_prompt(&shortlist, sections, soft_target, hard_max); tracing::debug!( shortlist = shortlist.len(), approx_tokens = super::approx_tokens(&prompt), - "stage B request" + "editor request" ); - let raw = llm.complete(&prompt, llm_temperature(), true).await?; + let raw = match complete_with_fallback(llms, primary, &prompt).await { + Ok(raw) => raw, + Err(error) => { + tracing::error!(%error, "editor and bulk fallback both failed; selecting heuristically"); + return Ok(select_without_llm( + candidates, + sections, + soft_target, + hard_max, + date, + )); + } + }; let items = parse_selection_response(&raw); if items.is_empty() { - tracing::error!("stage B returned no usable picks; falling back to heuristic ranking"); - return Ok(select_without_llm(candidates, sections, target, date)); + tracing::error!("editor returned no usable picks; falling back to heuristic ranking"); + return Ok(select_without_llm( + candidates, + sections, + soft_target, + hard_max, + date, + )); } let by_id: HashMap = candidates.iter().map(|c| (c.article.id, c)).collect(); - let mut chosen: Vec<(SelectionItem, ScoredArticle)> = Vec::with_capacity(items.len()); - let mut seen: HashSet = HashSet::new(); + let mut chosen = Vec::with_capacity(items.len()); + let mut seen = HashSet::new(); for item in items { if !seen.insert(item.id) { - tracing::warn!(id = item.id, "stage B picked the same article twice"); + tracing::warn!(id = item.id, "editor picked the same article twice"); continue; } match by_id.get(&item.id) { Some(candidate) => chosen.push((item, (*candidate).clone())), - None => tracing::warn!(id = item.id, "stage B invented an id that was not offered"), + None => tracing::warn!(id = item.id, "editor invented an id that was not offered"), } } - - // Auto-include feeds can never be dropped (§3.5). for candidate in &candidates { if candidate.auto_include && seen.insert(candidate.article.id) { - tracing::info!( - id = candidate.article.id, - title = %candidate.article.title, - "re-inserting an always-include article the model dropped" - ); chosen.push(( SelectionItem { id: candidate.article.id, section: "From the Blogroll".into(), position: i64::MAX, lead_story: false, + why: Some("A standing source you always want represented".into()), }, candidate.clone(), )); } } - - let lineup = assemble(chosen, &candidates, sections, target, date); - tracing::info!( - picks = lineup.picks.len(), - sections = lineup.section_order.len(), - lead = lineup.lead().map(|p| p.article.id), - "stage B lineup ready" - ); - Ok(lineup) + Ok(assemble(chosen, sections, hard_max, date)) } -/// Stage B runs at the scoring temperature: this is a judgement call, not prose. -fn llm_temperature() -> f32 { - 0.4 +/// The editor runs at the scoring temperature: this is a judgement call, not +/// prose. The Anthropic backend ignores it (§4.2). +const EDITOR_TEMPERATURE: f32 = 0.4; + +/// One attempt on `primary`; on any error (refusal, budget, API) the same prompt +/// goes to the bulk client when that is a different provider (§13, §17). +async fn complete_with_fallback( + llms: &Llms, + primary: &super::llm::LlmClient, + prompt: &str, +) -> Result { + match primary.complete(prompt, EDITOR_TEMPERATURE, true).await { + Ok(raw) => Ok(raw), + Err(primary_error) => { + let fallback = llms + .bulk + .as_ref() + .filter(|bulk| primary.provider != bulk.provider); + let Some(fallback) = fallback else { + return Err(primary_error); + }; + tracing::warn!( + error = %primary_error, + provider = primary.provider, + "editor failed; retrying the same prompt on bulk" + ); + fallback.complete(prompt, EDITOR_TEMPERATURE, true).await + } + } } -/// Top [`SHORTLIST_SIZE`] candidates by combined score, always including the -/// auto-includes (§3.6). +/// Top [`SHORTLIST_SIZE`] (or `2 × hard_max`) candidates by combined score, +/// always including the auto-includes. fn shortlist(candidates: &[ScoredArticle], target: usize) -> Vec { let mut ranked: Vec = candidates.to_vec(); sort_by_combined(&mut ranked); @@ -526,19 +550,16 @@ fn sort_by_combined(candidates: &mut [ScoredArticle]) { }); } -/// Turn validated picks into a [`Lineup`]: clamp the size, force a single lead, -/// order the sections and renumber positions (§3.6). +/// Turn validated picks into a [`Lineup`]: trim to `hard_max`, force a single +/// lead, order the sections and renumber positions (§13). No minimum size. fn assemble( mut chosen: Vec<(SelectionItem, ScoredArticle)>, - all: &[ScoredArticle], sections: &[String], - target: usize, + hard_max: usize, date: Date, ) -> Lineup { - let (min, max) = size_bounds(target); - // Too many: drop the weakest non-auto-include picks. - if chosen.len() > max { + if chosen.len() > hard_max { chosen.sort_by(|a, b| { b.1.auto_include.cmp(&a.1.auto_include).then_with(|| { b.1.combined_score() @@ -546,35 +567,9 @@ fn assemble( .unwrap_or(std::cmp::Ordering::Equal) }) }); - let dropped = chosen.len() - max; - chosen.truncate(max); - tracing::info!(dropped, max, "trimmed the lineup to the size ceiling"); - } - - // Too few: top up from the best unpicked candidates. - if chosen.len() < min { - let taken: HashSet = chosen.iter().map(|(i, _)| i.id).collect(); - let mut rest: Vec = all - .iter() - .filter(|c| !taken.contains(&c.article.id)) - .cloned() - .collect(); - sort_by_combined(&mut rest); - let wanted = min - chosen.len(); - let added = rest.len().min(wanted); - for candidate in rest.into_iter().take(wanted) { - let section = heuristic_section(&candidate, sections); - chosen.push(( - SelectionItem { - id: candidate.article.id, - section, - position: i64::MAX, - lead_story: false, - }, - candidate, - )); - } - tracing::info!(added, min, "topped the lineup up to the size floor"); + let dropped = chosen.len() - hard_max; + chosen.truncate(hard_max); + tracing::info!(dropped, hard_max, "trimmed the lineup to the size ceiling"); } // Normalize sections and pick the section order. @@ -632,6 +627,7 @@ fn assemble( section: item.section, position: *position, is_lead: Some(item.id) == lead_id, + why: item.why, summary: None, llm: candidate.llm.clone(), discussion: None, @@ -647,52 +643,46 @@ fn assemble( } } -/// `--skip-llm` fallback: take the top `target` by prefilter score and bucket them -/// into sections by feed category (notes §6). +/// Heuristic fallback (`--skip-llm`, no provider, or both providers failed): the +/// top `soft_target` by prefilter score plus the auto-includes, bucketed into +/// sections by feed category, trimmed to `hard_max` (notes §6). pub fn select_without_llm( candidates: Vec, sections: &[String], - target: usize, + soft_target: usize, + hard_max: usize, date: Date, ) -> Lineup { - let mut ranked = candidates.clone(); + let mut ranked = candidates; super::prefilter::sort_by_prefilter(&mut ranked); - - let mut chosen: Vec<(SelectionItem, ScoredArticle)> = Vec::new(); - let mut seen: HashSet = HashSet::new(); - for candidate in ranked.into_iter() { - let auto = candidate.auto_include; - if chosen.len() >= target && !auto { + let mut chosen = Vec::new(); + let mut seen = HashSet::new(); + for candidate in ranked { + if chosen.len() >= soft_target && !candidate.auto_include { continue; } if !seen.insert(candidate.article.id) { continue; } - let section = heuristic_section(&candidate, sections); chosen.push(( SelectionItem { id: candidate.article.id, - section, + section: heuristic_section(&candidate, sections), position: chosen.len() as i64 + 1, lead_story: false, + why: None, }, candidate, )); } - let lineup = assemble(chosen, &candidates, sections, target, date); - tracing::info!( - picks = lineup.picks.len(), - sections = lineup.section_order.len(), - "skip-llm lineup ready" - ); - lineup + assemble(chosen, sections, hard_max, date) } #[cfg(test)] mod tests { use super::*; - use crate::config::{CurationConfig, DeepseekConfig}; - use crate::curate::llm::{MockBackend, UsageMeter}; + use crate::config::{AnthropicConfig, CurationConfig, DeepseekConfig}; + use crate::curate::llm::{ChatBackend, LlmClient, MockBackend, PriceTable, UsageMeter}; use crate::curate::prefilter::tests::article; use crate::types::{LlmScore, TokenUsage}; use std::sync::Arc; @@ -738,6 +728,49 @@ mod tests { .collect() } + fn mock(provider: &'static str, backend: Arc, limit: f64) -> LlmClient { + let prices = if provider == "anthropic" { + PriceTable::anthropic(&AnthropicConfig::default()) + } else { + PriceTable::deepseek(&DeepseekConfig::default()) + }; + LlmClient::with_backend_options( + provider, + "model", + "SYSTEM".into(), + None, + UsageMeter::with_prices(prices, limit), + backend as Arc, + ) + } + + /// DeepSeek only — the shape of a run without an Anthropic key. + fn bulk_only(backend: Arc) -> Llms { + Llms { + bulk: Some(mock("deepseek", backend, 2.0)), + editor: None, + } + } + + fn editor_and_bulk(editor: Arc, bulk: Arc) -> Llms { + Llms { + bulk: Some(mock("deepseek", bulk, 2.0)), + editor: Some(mock("anthropic", editor, 3.0)), + } + } + + fn picks_json(n: i64) -> String { + let picks: Vec = (1..=n) + .map(|i| { + format!( + r#"{{"id":{i},"section":"Top Stories","position":{i},"lead_story":{},"why":"pick {i} because"}}"#, + i == 1 + ) + }) + .collect(); + format!(r#"{{"picks":[{}]}}"#, picks.join(",")) + } + #[test] fn section_resolution_maps_onto_the_palette() { let s = sections(); @@ -786,33 +819,64 @@ mod tests { assert!(items[0].lead_story); assert_eq!(items[0].section, "Top Stories"); assert_eq!(items.iter().filter(|i| i.lead_story).count(), 1); + assert!(items[0].why.as_deref().is_some_and(|w| !w.is_empty())); // Junk entries in the fixture are dropped, not fatal. assert!(items.iter().all(|i| i.id != 0)); } #[test] - fn size_bounds_follow_the_spec() { - assert_eq!(size_bounds(20), (15, 25)); - assert_eq!(size_bounds(6), (1, 11)); - assert_eq!(size_bounds(0), (1, 6)); + fn why_lines_are_optional_and_capped_at_fourteen_words() { + let long = (1..=30) + .map(|i| format!("w{i}")) + .collect::>() + .join(" "); + let items = parse_selection_response(&format!( + r#"{{"picks":[{{"id":1,"section":"Top Stories","why":"{long}"}}, + {{"id":2,"section":"Top Stories","why":" "}}, + {{"id":3,"section":"Top Stories"}}]}}"# + )); + assert_eq!(items.len(), 3); + assert_eq!( + items[0] + .why + .as_deref() + .map(|w| w.split_whitespace().count()), + Some(14) + ); + assert!(items[1].why.is_none()); + assert!(items[2].why.is_none()); + } + + #[test] + fn the_prompt_substitutes_the_size_targets() { + let prompt = build_prompt(&candidates(3), §ions(), 6, 11); + assert!(prompt.contains("aim for about 6; never more than 11; there is NO minimum")); + assert!(!prompt.contains("{soft_target}") && !prompt.contains("{hard_max}")); + assert!(prompt.contains("--- id: 1\n")); + assert!(prompt.contains("score: 9.9 · Tech & Engineering — solid")); + assert!(prompt.contains("opening: ")); + assert!( + !prompt.contains("combined"), + "the numeric blend stays out of the prompt" + ); + let mut flagged = candidates(1); + flagged[0].auto_include = true; + flagged[0].article.excerpt_only = true; + let prompt = build_prompt(&flagged, §ions(), 6, 11); + assert!(prompt.contains("flags: always-include | excerpt only")); } #[tokio::test] async fn selection_builds_a_valid_lineup() { let backend = Arc::new(MockBackend::new()); backend.push(LINEUP_FIXTURE, TokenUsage::default()); - let llm = LlmClient::with_backend( - "deepseek-v4-flash", - "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), 2.0), - backend.clone(), - ); + let llms = bulk_only(Arc::clone(&backend)); // ids 101..=112 so the fixture's picks resolve. let pool: Vec = (101..=112) .map(|i| candidate(i, &format!("Article {i}"), 800, 7.0)) .collect(); - let lineup = select(&llm, pool, §ions(), 6, date()) + let lineup = select(&llms, pool, §ions(), 6, 11, date()) .await .expect("selection"); @@ -845,9 +909,32 @@ mod tests { ); // The prompt carried the shortlist and the palette. let prompt = &backend.prompts()[0].user; - assert!(prompt.starts_with(SELECT_INSTRUCTIONS)); + assert!(prompt.starts_with("TASK: assemble today's issue of The Daily EPUB")); assert!(prompt.contains("--- id: 101")); - assert!(prompt.contains("never fewer than 1 and never more than 11")); + assert!(prompt.contains("aim for about 6; never more than 11")); + } + + #[tokio::test] + async fn why_lines_land_on_picks() { + let backend = Arc::new(MockBackend::new()); + backend.push(picks_json(3), TokenUsage::default()); + let lineup = select( + &bulk_only(backend), + candidates(5), + §ions(), + 3, + 5, + date(), + ) + .await + .expect("selection"); + assert_eq!(lineup.picks.len(), 3); + for pick in &lineup.picks { + assert_eq!( + pick.why.as_deref(), + Some(format!("pick {} because", pick.article.id).as_str()) + ); + } } #[tokio::test] @@ -856,19 +943,22 @@ mod tests { backend.push( r#"{"picks":[{"id":9999,"section":"Top Stories","position":1,"lead_story":true}, {"id":1,"section":"Sportsball","position":2}, - {"id":2,"section":"Niche Corner","position":1}]}"#, + {"id":2,"section":"Niche Corner","position":1}, + {"id":2,"section":"Niche Corner","position":2}]}"#, TokenUsage::default(), ); - let llm = LlmClient::with_backend( - "deepseek-v4-flash", - "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), 2.0), - backend, - ); - let lineup = select(&llm, candidates(6), §ions(), 2, date()) - .await - .expect("selection"); + let lineup = select( + &bulk_only(backend), + candidates(6), + §ions(), + 2, + 7, + date(), + ) + .await + .expect("selection"); assert!(lineup.picks.iter().all(|p| p.article.id != 9999)); + assert_eq!(lineup.picks.len(), 2, "the duplicate id was dropped"); assert_eq!(lineup.picks.iter().filter(|p| p.is_lead).count(), 1); for pick in &lineup.picks { assert!(sections().contains(&pick.section)); @@ -876,93 +966,148 @@ mod tests { } #[tokio::test] - async fn oversized_and_undersized_answers_are_clamped() { - // Undersized: the model returns one pick but the floor is 5. + async fn a_nine_pick_answer_is_published_as_nine() { + // Soft target 20, ceiling 28, thirty candidates: the model picks nine. let backend = Arc::new(MockBackend::new()); - backend.push( - r#"{"picks":[{"id":1,"section":"Top Stories","position":1,"lead_story":true}]}"#, - TokenUsage::default(), - ); - let llm = LlmClient::with_backend( - "deepseek-v4-flash", - "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), 2.0), - backend, - ); - let lineup = select(&llm, candidates(30), §ions(), 10, date()) - .await - .expect("selection"); - assert!(lineup.picks.len() >= 5, "{}", lineup.picks.len()); - - // Oversized: 30 picks against a target of 6 (ceiling 11). - let picks: Vec = (1..=30) - .map(|i| format!(r#"{{"id":{i},"section":"Top Stories","position":{i}}}"#)) - .collect(); - let backend = Arc::new(MockBackend::new()); - backend.push( - format!(r#"{{"picks":[{}]}}"#, picks.join(",")), - TokenUsage::default(), - ); - let llm = LlmClient::with_backend( - "deepseek-v4-flash", - "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), 2.0), - backend, - ); - let lineup = select(&llm, candidates(30), §ions(), 6, date()) - .await - .expect("selection"); - assert_eq!(lineup.picks.len(), 11); + backend.push(picks_json(9), TokenUsage::default()); + let lineup = select( + &bulk_only(backend), + candidates(30), + §ions(), + 20, + 28, + date(), + ) + .await + .expect("selection"); + assert_eq!(lineup.picks.len(), 9, "no top-up, no padding"); } #[tokio::test] - async fn always_include_articles_are_reinserted() { + async fn hard_max_trims_oversized_answers_by_ranking() { let backend = Arc::new(MockBackend::new()); - backend.push( - r#"{"picks":[{"id":1,"section":"Top Stories","position":1,"lead_story":true}]}"#, - TokenUsage::default(), - ); - let llm = LlmClient::with_backend( - "deepseek-v4-flash", - "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), 2.0), - backend, - ); - let mut pool = candidates(3); - pool[2].auto_include = true; - let lineup = select(&llm, pool, §ions(), 1, date()) + backend.push(picks_json(30), TokenUsage::default()); + let lineup = select( + &bulk_only(backend), + candidates(30), + §ions(), + 6, + 11, + date(), + ) + .await + .expect("selection"); + assert_eq!(lineup.picks.len(), 11); + // The strongest by today's ranking key survive: ids 1..=11 score highest. + let mut ids: Vec = lineup.picks.iter().map(|p| p.article.id).collect(); + ids.sort_unstable(); + assert_eq!(ids, (1..=11).collect::>()); + } + + #[tokio::test] + async fn always_include_articles_are_reinserted_and_survive_the_trim() { + let backend = Arc::new(MockBackend::new()); + backend.push(picks_json(4), TokenUsage::default()); + let mut pool = candidates(30); + pool[29].auto_include = true; // id 30, the weakest by score + let lineup = select(&bulk_only(backend), pool, §ions(), 2, 4, date()) .await .expect("selection"); let ids: Vec = lineup.picks.iter().map(|p| p.article.id).collect(); - assert!(ids.contains(&3), "auto-include must survive: {ids:?}"); - assert_eq!( - lineup - .picks - .iter() - .find(|p| p.article.id == 3) - .map(|p| p.section.as_str()), - Some("From the Blogroll") - ); + assert!(ids.contains(&30), "auto-include must survive: {ids:?}"); + assert_eq!(lineup.picks.len(), 4, "the ceiling still holds"); + let reinserted = lineup + .picks + .iter() + .find(|p| p.article.id == 30) + .expect("reinserted"); + assert_eq!(reinserted.section, "From the Blogroll"); + assert!(reinserted.why.is_some()); } #[tokio::test] - async fn a_tripped_budget_falls_back_without_calling_the_model() { + async fn refusal_on_the_editor_falls_back_to_bulk_with_the_same_prompt() { + let editor = Arc::new(MockBackend::new()); + editor.push_llm_error(LlmError::Refusal { + provider: "anthropic", + }); + let bulk = Arc::new(MockBackend::new()); + bulk.push(picks_json(5), TokenUsage::default()); + let llms = editor_and_bulk(Arc::clone(&editor), Arc::clone(&bulk)); + + let lineup = select(&llms, candidates(10), §ions(), 5, 10, date()) + .await + .expect("selection"); + assert_eq!(lineup.picks.len(), 5); + assert_eq!(editor.calls(), 1); + assert_eq!(bulk.calls(), 1); + assert_eq!( + editor.prompts()[0].user, + bulk.prompts()[0].user, + "the bulk client gets the identical prompt" + ); + assert_eq!(editor.prompts()[0].system, bulk.prompts()[0].system); + } + + #[tokio::test] + async fn an_error_on_both_providers_selects_heuristically() { + let editor = Arc::new(MockBackend::new()); + editor.push_error("500 opus is down"); + let bulk = Arc::new(MockBackend::new()); + bulk.push_error("500 deepseek is down too"); + let llms = editor_and_bulk(Arc::clone(&editor), Arc::clone(&bulk)); + let lineup = select(&llms, candidates(10), §ions(), 4, 10, date()) + .await + .expect("heuristic fallback"); + assert_eq!(lineup.picks.len(), 4); + assert_eq!(editor.calls(), 1); + assert_eq!(bulk.calls(), 1); + } + + #[tokio::test] + async fn a_tripped_editor_budget_goes_straight_to_bulk() { + let editor = Arc::new(MockBackend::new()); + let bulk = Arc::new(MockBackend::new()); + bulk.push(picks_json(3), TokenUsage::default()); + let llms = editor_and_bulk(Arc::clone(&editor), Arc::clone(&bulk)); + llms.editor + .as_ref() + .expect("editor") + .meter + .preload_cost(10.0); + let lineup = select(&llms, candidates(10), §ions(), 3, 10, date()) + .await + .expect("selection"); + assert_eq!(lineup.picks.len(), 3); + assert_eq!(editor.calls(), 0, "a tripped editor is never called"); + assert_eq!(bulk.calls(), 1); + } + + #[tokio::test] + async fn a_tripped_bulk_budget_falls_back_without_calling_the_model() { let backend = Arc::new(MockBackend::new()); - let meter = UsageMeter::new(&DeepseekConfig::default(), 0.001); - meter.record(TokenUsage { - input_tokens: 1_000_000, + let llms = bulk_only(Arc::clone(&backend)); + llms.bulk.as_ref().expect("bulk").meter.record(TokenUsage { + input_tokens: 100_000_000, cached_tokens: 0, + cache_write_tokens: 0, output_tokens: 0, }); - let llm = - LlmClient::with_backend("deepseek-v4-flash", "SYSTEM".into(), meter, backend.clone()); - let lineup = select(&llm, candidates(20), §ions(), 6, date()) + let lineup = select(&llms, candidates(20), §ions(), 6, 28, date()) .await .expect("fallback"); assert_eq!(backend.calls(), 0); assert_eq!(lineup.picks.len(), 6); } + #[tokio::test] + async fn no_provider_selects_heuristically() { + let lineup = select(&Llms::default(), candidates(20), §ions(), 6, 28, date()) + .await + .expect("fallback"); + assert_eq!(lineup.picks.len(), 6); + } + #[test] fn skip_llm_lineup_uses_prefilter_order() { let mut pool = candidates(10); @@ -971,7 +1116,7 @@ mod tests { pool[9].auto_include = true; // id 10 is a personal blog pool[9].prefilter_score = 1.0; - let lineup = select_without_llm(pool, §ions(), 4, date()); + let lineup = select_without_llm(pool, §ions(), 4, 28, date()); assert_eq!(lineup.picks.len(), 5, "4 picks + the auto-include"); assert_eq!(lineup.lead().map(|p| p.article.id), Some(8)); assert_eq!(lineup.picks.iter().filter(|p| p.is_lead).count(), 1); @@ -984,13 +1129,23 @@ mod tests { for pick in &lineup.picks { assert!(sections().contains(&pick.section)); assert!(pick.summary.is_none()); + assert!(pick.why.is_none()); } assert!(!lineup.section_order.is_empty()); } + #[test] + fn heuristic_selection_respects_the_ceiling() { + let mut pool = candidates(10); + pool[9].auto_include = true; + let lineup = select_without_llm(pool, §ions(), 10, 4, date()); + assert_eq!(lineup.picks.len(), 4); + assert!(lineup.picks.iter().any(|p| p.article.id == 10)); + } + #[test] fn empty_input_yields_an_empty_lineup() { - let lineup = select_without_llm(Vec::new(), §ions(), 20, date()); + let lineup = select_without_llm(Vec::new(), §ions(), 20, 28, date()); assert!(lineup.picks.is_empty()); assert!(lineup.section_order.is_empty()); assert!(lineup.lead().is_none()); diff --git a/src/db.rs b/src/db.rs index 70eb550..0ded1ab 100644 --- a/src/db.rs +++ b/src/db.rs @@ -5,6 +5,7 @@ //! (implementation notes §2). Pipeline writes are idempotent upserts so that //! `generate --date X` can be re-run safely; feedback events are append-only. +use std::collections::BTreeMap; use std::path::Path; use std::str::FromStr; use std::time::Duration; @@ -43,6 +44,12 @@ pub enum DbError { }, #[error("malformed value in column `{column}`: {value}")] Decode { column: &'static str, value: String }, + #[error("malformed JSON in column `{column}`: {source}")] + Json { + column: &'static str, + #[source] + source: serde_json::Error, + }, } type Result = std::result::Result; @@ -480,8 +487,8 @@ impl Db { .await?; for pick in picks { sqlx::query( - "INSERT INTO issue_articles (issue_date, article_id, section, position, is_lead, summary) - VALUES (?, ?, ?, ?, ?, ?)", + "INSERT INTO issue_articles (issue_date, article_id, section, position, is_lead, summary, why) + VALUES (?, ?, ?, ?, ?, ?, ?)", ) .bind(date.to_string()) .bind(pick.article.id) @@ -489,6 +496,7 @@ impl Db { .bind(pick.position) .bind(pick.is_lead) .bind(pick.summary.as_deref()) + .bind(pick.why.as_deref()) .execute(&mut *tx) .await?; } @@ -656,7 +664,7 @@ impl Db { sqlx::query( "UPDATE runs SET finished_at = ?, entries_fetched = ?, candidates = ?, selected = ?, input_tokens = ?, cached_tokens = ?, output_tokens = ?, cost_usd = ?, - status = ?, error = ? + status = ?, error = ?, provider_costs_json = ?, config_json = ? WHERE id = ?", ) .bind(report.finished_at.map(fmt_ts)) @@ -669,20 +677,55 @@ impl Db { .bind(report.cost_usd) .bind(report.status.as_str()) .bind(report.error.as_deref()) + .bind( + serde_json::to_string(&report.provider_costs).map_err(|source| DbError::Json { + column: "runs.provider_costs_json", + source, + })?, + ) + .bind( + serde_json::to_string(&report.config_json).map_err(|source| DbError::Json { + column: "runs.config_json", + source, + })?, + ) .bind(id) .execute(&self.pool) .await?; Ok(()) } - /// Total spend recorded for a date, for the `max_daily_usd` guardrail (§3.6). - pub async fn spend_for_date(&self, date: Date) -> Result { - let row = - sqlx::query("SELECT COALESCE(SUM(cost_usd), 0.0) AS total FROM runs WHERE date = ?") - .bind(date.to_string()) - .fetch_one(&self.pool) - .await?; - Ok(row.get::("total")) + /// Earlier provider spend on the UTC date containing this run's start (§5). + pub async fn provider_spend_for_utc_day( + &self, + started_at: Timestamp, + ) -> Result> { + let utc_date = started_at + .to_zoned(jiff::tz::TimeZone::UTC) + .date() + .to_string(); + let rows = sqlx::query( + "SELECT provider_costs_json FROM runs + WHERE substr(started_at, 1, 10) = ? AND started_at < ? + AND provider_costs_json IS NOT NULL", + ) + .bind(utc_date) + .bind(fmt_ts(started_at)) + .fetch_all(&self.pool) + .await?; + let mut totals = BTreeMap::new(); + for row in rows { + let raw = row.get::("provider_costs_json"); + let providers: BTreeMap = + serde_json::from_str(&raw).map_err(|source| DbError::Json { + column: "runs.provider_costs_json", + source, + })?; + for (provider, usage) in providers { + *totals.entry(provider).or_insert(0.0) += usage.cost_usd; + } + } + Ok(totals) } } @@ -988,7 +1031,7 @@ mod tests { report.counts.entries_fetched = 412; report.counts.candidates = 120; report.counts.selected = 20; - report.finish(ts("2026-08-15T05:36:00Z"), 0.14, 0.0028, 0.28); + report.finish(ts("2026-08-15T05:36:00Z")); db.finish_run(run_id, &report).await.unwrap(); db.upsert_issue( @@ -1006,7 +1049,82 @@ mod tests { let next: Date = "2026-08-16".parse().unwrap(); assert_eq!(db.next_issue_number(next).await.unwrap(), 2); assert_eq!(db.recent_reports(5).await.unwrap().len(), 1); - assert_eq!(db.spend_for_date(date).await.unwrap(), 0.0); + // No provider spend was recorded, so the budget-day preload is empty. + let spend = db + .provider_spend_for_utc_day(ts("2026-08-15T23:00:00Z")) + .await + .unwrap(); + assert!(spend.values().all(|usd| *usd == 0.0)); + } + + async fn record_run(db: &Db, date: Date, started: &str, deepseek: f64, anthropic: f64) { + use crate::report::{ProviderUsage, RunReport}; + let started_at = ts(started); + let run_id = db.start_run(date, started_at).await.unwrap(); + let mut report = RunReport::new(date, started_at); + report.provider_costs.insert( + "deepseek".into(), + ProviderUsage { + usage: crate::types::TokenUsage { + input_tokens: 10, + ..Default::default() + }, + cost_usd: deepseek, + }, + ); + report.provider_costs.insert( + "anthropic".into(), + ProviderUsage { + usage: crate::types::TokenUsage::default(), + cost_usd: anthropic, + }, + ); + report.config_json = serde_json::json!({"models": {"editor": "claude-opus-5"}}); + report.finish(started_at); + db.finish_run(run_id, &report).await.unwrap(); + } + + #[tokio::test] + async fn provider_spend_is_summed_by_the_utc_day_of_started_at() { + let (_dir, db) = temp_db().await; + let date: Date = "2026-08-15".parse().unwrap(); + record_run(&db, date, "2026-08-15T03:00:00Z", 0.10, 0.50).await; + record_run(&db, date, "2026-08-15T23:30:00Z", 0.05, 0.0).await; + // Nominal issue date 08-15 in New York, but already 08-16 in UTC: a + // different budget day (§5). + record_run(&db, date, "2026-08-16T01:00:00Z", 1.0, 1.0).await; + + let spend = db + .provider_spend_for_utc_day(ts("2026-08-15T23:45:00Z")) + .await + .unwrap(); + assert!((spend["deepseek"] - 0.15).abs() < 1e-9); + assert!((spend["anthropic"] - 0.5).abs() < 1e-9); + // Only runs that started earlier than this one count. + let spend = db + .provider_spend_for_utc_day(ts("2026-08-15T12:00:00Z")) + .await + .unwrap(); + assert!((spend["deepseek"] - 0.10).abs() < 1e-9); + let spend = db + .provider_spend_for_utc_day(ts("2026-08-17T12:00:00Z")) + .await + .unwrap(); + assert!(spend.is_empty()); + + // `provider_costs_json` and `config_json` were written and round-trip. + let row = + sqlx::query("SELECT provider_costs_json, config_json FROM runs ORDER BY id LIMIT 1") + .fetch_one(&db.pool) + .await + .unwrap(); + let costs: BTreeMap = + serde_json::from_str(&row.get::("provider_costs_json")).unwrap(); + assert_eq!(costs["deepseek"].usage.input_tokens, 10); + assert!((costs["anthropic"].cost_usd - 0.5).abs() < 1e-9); + let config: serde_json::Value = + serde_json::from_str(&row.get::("config_json")).unwrap(); + assert_eq!(config["models"]["editor"], "claude-opus-5"); } #[tokio::test] diff --git a/src/epub/build.rs b/src/epub/build.rs index 8e4575d..eb8120c 100644 --- a/src/epub/build.rs +++ b/src/epub/build.rs @@ -66,12 +66,7 @@ pub fn render_all( ]; for name in section_names(issue) { - let intro = issue - .editorial - .section_intros - .get(&name) - .map(|s| s.as_str()); - chapters.push(render_section_page(&name, intro)?); + chapters.push(render_section_page(&name)?); for pick in issue.lineup.section_picks(&name) { chapters.push(render_article( issue, diff --git a/src/epub/chapters.rs b/src/epub/chapters.rs index 1328ac8..06664d0 100644 --- a/src/epub/chapters.rs +++ b/src/epub/chapters.rs @@ -39,6 +39,7 @@ struct IndexEntry { source: String, reading_minutes: i64, summary: String, + why: Option, } struct IndexSection { @@ -59,7 +60,6 @@ struct InThisIssue { struct SectionPage { title: String, name: String, - intro: Option, } struct RatingLinks { @@ -76,6 +76,7 @@ struct ArticleChapter { byline: Option, meta_line: String, social_line: Option, + why: Option, summary: Option, excerpt_only: bool, body_html: String, @@ -108,7 +109,10 @@ struct ColophonChapter { issue_number: i64, display_date: String, generated_at: String, - model: String, + bulk_model: String, + editor_model: String, + summaries_model: String, + provider_costs: Vec, entries_fetched: i64, feeds_seen: i64, candidates: i64, @@ -120,6 +124,11 @@ struct ColophonChapter { generator_version: String, } +struct ProviderCostLine { + provider: String, + cost: String, +} + // --------------------------------------------------------------------------- // Chapters (§3.10) // --------------------------------------------------------------------------- @@ -274,6 +283,7 @@ pub fn render_in_this_issue(issue: &Issue) -> Result { source: pick.article.feed_title.clone(), reading_minutes: pick.article.reading_minutes(), summary: summary_for(issue, pick).unwrap_or_default().to_string(), + why: pick.why.clone(), }) .collect(); sections.push(IndexSection { name, entries }); @@ -287,6 +297,7 @@ pub fn render_in_this_issue(issue: &Issue) -> Result { source: "Wikipedia Current Events".into(), reading_minutes: 3, summary: "The day's events, as recorded by the Current Events portal.".into(), + why: None, }], }); } @@ -304,14 +315,11 @@ pub fn render_in_this_issue(issue: &Issue) -> Result { }) } -/// A section title page: name + LLM intro (§3.10). -pub fn render_section_page(name: &str, intro: Option<&str>) -> Result { +/// A section title page: the name only (§14.2 removed the LLM intros). +pub fn render_section_page(name: &str) -> Result { let tpl = SectionPage { title: name.to_string(), name: name.to_string(), - intro: intro - .map(|s| s.trim().to_string()) - .filter(|s| !s.is_empty()), }; Ok(Chapter { id: format!("sec-{name}"), @@ -364,6 +372,7 @@ pub fn render_article( byline: article.author.as_ref().map(|a| format!("By {a}")), meta_line: meta_parts.join(" \u{00b7} "), social_line: social_line(&article.social), + why: pick.why.clone(), summary: summary_for(issue, pick).map(str::to_string), excerpt_only: article.excerpt_only, body_html: prepare_body(&article.content_html, images_), @@ -429,16 +438,23 @@ pub fn render_world_briefing(issue: &Issue) -> Result, EpubError /// Colophon: generation timestamp, models used, token cost, feed counts (§3.10). pub fn render_colophon(issue: &Issue) -> Result { let colophon = &issue.colophon; + let provider_costs = colophon + .provider_costs + .iter() + .map(|(provider, cost)| ProviderCostLine { + provider: provider.clone(), + cost: format!("${cost:.4}"), + }) + .collect(); let tpl = ColophonChapter { title: "Colophon".into(), issue_number: issue.meta.issue_number, display_date: issue.meta.display_date.clone(), generated_at: issue.meta.generated_at.to_string(), - model: if colophon.model.is_empty() { - "none (heuristic selection)".into() - } else { - colophon.model.clone() - }, + bulk_model: colophon.models.bulk.clone(), + editor_model: colophon.models.editor.clone(), + summaries_model: colophon.models.summaries.clone(), + provider_costs, entries_fetched: colophon.entries_fetched, feeds_seen: colophon.feeds_seen, candidates: colophon.candidates, diff --git a/src/epub/fixtures.rs b/src/epub/fixtures.rs index 9a35e91..1cd8278 100644 --- a/src/epub/fixtures.rs +++ b/src/epub/fixtures.rs @@ -86,6 +86,7 @@ pub fn issue() -> Issue { section: "Top Stories".into(), position: 0, is_lead: true, + why: Some("The systems story with enough operational detail to matter".into()), summary: Some("What it argues, and why it is worth the time.".into()), llm: None, discussion: Some(discussion(1, 1001)), @@ -95,12 +96,11 @@ pub fn issue() -> Issue { section: "Niche Corner".into(), position: 0, is_lead: false, + why: Some("A small-scene delight outside the usual technical orbit".into()), summary: None, llm: None, discussion: None, }; - let mut section_intros = BTreeMap::new(); - section_intros.insert("Top Stories".to_string(), "The day in brief.".to_string()); let mut summaries = BTreeMap::new(); summaries.insert(2, "A short abstract for the second piece.".to_string()); @@ -122,7 +122,6 @@ pub fn issue() -> Issue { }, editorial: Editorial { front_page_html: "

Two stories today, both worth your coffee.

".into(), - section_intros, summaries, }, world_briefing: Some(WorldBriefing { @@ -141,7 +140,15 @@ pub fn issue() -> Issue { }], }), colophon: Colophon { - model: "deepseek-v4-flash".into(), + provider_costs: BTreeMap::from([ + ("deepseek".into(), 0.0231), + ("anthropic".into(), 0.05), + ]), + models: Models { + bulk: "deepseek-v4-flash".into(), + editor: "claude-opus-5".into(), + summaries: "claude-opus-5".into(), + }, entries_fetched: 431, feeds_seen: 92, candidates: 120, diff --git a/src/epub/templates/chapter.xhtml b/src/epub/templates/chapter.xhtml index 29632ca..5777156 100644 --- a/src/epub/templates/chapter.xhtml +++ b/src/epub/templates/chapter.xhtml @@ -7,6 +7,9 @@

{{ line }}

{% endif %}

{{ meta_line }}

+{% if let Some(text) = why %} +

Why it's here: {{ text }}

+{% endif %} {% if let Some(line) = social_line %} {% endif %} diff --git a/src/epub/templates/colophon.xhtml b/src/epub/templates/colophon.xhtml index 441f8d3..98cb1b7 100644 --- a/src/epub/templates/colophon.xhtml +++ b/src/epub/templates/colophon.xhtml @@ -9,12 +9,17 @@

Issue: No. {{ issue_number }} · {{ display_date }}

Generated: {{ generated_at }}

-

Curation model: {{ model }}

+

Bulk model: {{ bulk_model }}

+

Editor model: {{ editor_model }}

+

Summaries model: {{ summaries_model }}

Entries considered: {{ entries_fetched }} from {{ feeds_seen }} feeds

Candidates scored: {{ candidates }}

Articles selected: {{ article_count }} across {{ section_count }} sections

Words: {{ total_words }} · {{ reading_line }}

-

Token cost: {{ cost_usd }}

+{% for line in provider_costs %} +

{{ line.provider }} cost: {{ line.cost }}

+{% endfor %} +

Total token cost: {{ cost_usd }}

Generator: {{ generator_version }}

Article text belongs to its authors and publications; excerpts and links are diff --git a/src/epub/templates/front_page.xhtml b/src/epub/templates/front_page.xhtml index a6f2e8a..25899e3 100644 --- a/src/epub/templates/front_page.xhtml +++ b/src/epub/templates/front_page.xhtml @@ -4,7 +4,7 @@

The Daily EPUB

{{ display_date }} · No. {{ issue_number }}

-

From the Editor

+

The Brief

{{ body_html|safe }}
diff --git a/src/epub/templates/in_this_issue.xhtml b/src/epub/templates/in_this_issue.xhtml index c2f15bc..5f6fd7d 100644 --- a/src/epub/templates/in_this_issue.xhtml +++ b/src/epub/templates/in_this_issue.xhtml @@ -12,6 +12,9 @@

{{ entry.source }} · {{ entry.reading_minutes }} min read

{% if !entry.summary.is_empty() %}

{{ entry.summary }}

+{% endif %} +{% if let Some(text) = entry.why %} +

Why it's here: {{ text }}

{% endif %} {% endfor %} diff --git a/src/epub/templates/section.xhtml b/src/epub/templates/section.xhtml index 3f424de..09f01c5 100644 --- a/src/epub/templates/section.xhtml +++ b/src/epub/templates/section.xhtml @@ -3,7 +3,4 @@ {% block content %}

{{ name }}

-{% if let Some(text) = intro %} -

{{ text }}

-{% endif %} {% endblock %} diff --git a/src/epub/templates/style-x4.css b/src/epub/templates/style-x4.css index 5c858f3..2b1be76 100644 --- a/src/epub/templates/style-x4.css +++ b/src/epub/templates/style-x4.css @@ -190,3 +190,5 @@ p.comment-line { .fact-line { margin: 0 0 0.35em 0; } + +.why, .index-why { font-size: 0.9em; font-style: italic; } diff --git a/src/epub/templates/style.css b/src/epub/templates/style.css index 4aef901..0cfe02c 100644 --- a/src/epub/templates/style.css +++ b/src/epub/templates/style.css @@ -296,3 +296,5 @@ blockquote.comment blockquote.comment { .fact-line { margin: 0 0 0.35em 0; } + +.why, .index-why { font-size: 0.9em; font-style: italic; } diff --git a/src/main.rs b/src/main.rs index cd9cdcf..916c081 100644 --- a/src/main.rs +++ b/src/main.rs @@ -270,13 +270,30 @@ fn print_report(report: &RunReport) { report.counts.duplicates_merged, report.counts.entries_dropped, ); + if report.counts.llm_unscored > 0 { + println!( + "curation: {} scored · {} unscored · {} selected", + report.counts.llm_scored, report.counts.llm_unscored, report.counts.selected, + ); + } println!( - "tokens: {} input · {} cached · {} output = ${:.4}", + "tokens: {} input · {} cache read · {} cache write · {} output = ${:.4}", report.usage.input_tokens, report.usage.cached_tokens, + report.usage.cache_write_tokens, report.usage.output_tokens, report.cost_usd, ); + for (provider, usage) in &report.provider_costs { + println!( + " {provider}: {} input · {} cache read · {} cache write · {} output = ${:.4}", + usage.usage.input_tokens, + usage.usage.cached_tokens, + usage.usage.cache_write_tokens, + usage.usage.output_tokens, + usage.cost_usd, + ); + } for warning in &report.warnings { println!("warning: {warning}"); } @@ -316,8 +333,15 @@ fn print_lineup(issue: &daily_epub::types::Issue) { // Other subcommands // --------------------------------------------------------------------------- +/// `profile rebuild` runs on the editor when configured, else bulk (§14.3). async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> { - let meter = curate::llm::UsageMeter::new(&config.deepseek, config.max_daily_usd); + use curate::llm::{Llms, PriceTable, UsageMeter}; + let bulk_meter = + UsageMeter::with_prices(PriceTable::deepseek(&config.deepseek), config.max_daily_usd); + let editor_meter = UsageMeter::with_prices( + PriceTable::anthropic(&config.anthropic), + config.anthropic.max_daily_usd, + ); let profile = curate::profile::load_or_build( db, &config.interests_opml, @@ -325,10 +349,22 @@ async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> { config.curation.feedback.verdicts_in_prompt, ) .await?; - let llm = curate::llm::LlmClient::new(&config.deepseek, profile.text, meter)?; + let llms = Llms::from_config( + &config.deepseek, + &config.anthropic, + profile.text, + bulk_meter, + editor_meter, + ); + let Some(llm) = llms.editor_or_bulk() else { + anyhow::bail!( + "no LLM provider is configured; set DAILY_EPUB_ANTHROPIC__API_KEY or DAILY_EPUB_DEEPSEEK__API_KEY" + ); + }; + tracing::info!(provider = llm.provider, model = %llm.model, "rebuilding the profile"); let rebuilt = curate::profile::rebuild( db, - &llm, + llm, &config.interests_opml, &config.profile_path, config.curation.feedback.verdicts_in_prompt, diff --git a/src/pipeline.rs b/src/pipeline.rs index 5155cc6..8862575 100644 --- a/src/pipeline.rs +++ b/src/pipeline.rs @@ -31,15 +31,15 @@ use jiff::civil::Date; use jiff::{Timestamp, Zoned}; use crate::config::Config; -use crate::curate::llm::{LlmClient, UsageMeter}; +use crate::curate::llm::{Llms, PriceTable, UsageMeter}; use crate::curate::{Curator, editorial, profile}; use crate::db::Db; use crate::extract::Extractor; use crate::miniflux::MinifluxClient; use crate::publish::Published; -use crate::report::{RunReport, RunStatus}; +use crate::report::{ProviderUsage, RunReport, RunStatus}; use crate::types::{ - Article, Artifact, Colophon, Edition, Issue, IssueMeta, Lineup, reading_minutes, + Article, Artifact, Colophon, Edition, Issue, IssueMeta, Lineup, Models, reading_minutes, }; use crate::{comments, dedupe, epub, http, miniflux, publish, social, world}; @@ -185,7 +185,7 @@ pub async fn generate(config: &Config, db: &Db, opts: &GenerateOptions) -> Resul let started_at = Timestamp::now(); let (window_start, window_end) = ingest_window(config, date)?; let out_dir = opts.out.clone().unwrap_or_else(|| config.out_dir.clone()); - let target = opts.max_articles.unwrap_or(config.target_article_count); + let (soft_target, hard_max) = issue_size_bounds(config, opts.max_articles); let span = tracing::info_span!("generate", %date, dry_run = opts.dry_run); let _guard = span.enter(); @@ -193,16 +193,19 @@ pub async fn generate(config: &Config, db: &Db, opts: &GenerateOptions) -> Resul %window_start, %window_end, lookback_hours = config.lookback_hours, - target, + soft_target, + hard_max, skip_llm = opts.skip_llm, out = %out_dir.display(), "starting run" ); + log_resolved_providers(config, opts.skip_llm); let run_id = db.start_run(date, started_at).await?; let mut report = RunReport::new(date, started_at); report.window_start = Some(window_start); report.window_end = Some(window_end); + report.config_json = resolved_run_config(config, soft_target, hard_max); if opts.dry_run { report.status = RunStatus::DryRun; } @@ -211,19 +214,16 @@ pub async fn generate(config: &Config, db: &Db, opts: &GenerateOptions) -> Resul config, db, date, - target, + soft_target, + hard_max, + started_at, out_dir, dry_run: opts.dry_run, skip_llm: opts.skip_llm, }; let stages = match run_stages(&ctx, window_start, window_end, &mut report).await { Ok(stages) => { - report.finish( - Timestamp::now(), - config.deepseek.price_input_per_mtok, - config.deepseek.price_cached_input_per_mtok, - config.deepseek.price_output_per_mtok, - ); + report.finish(Timestamp::now()); stages } Err(e) => { @@ -276,7 +276,9 @@ struct StageContext<'a> { config: &'a Config, db: &'a Db, date: Date, - target: usize, + soft_target: usize, + hard_max: usize, + started_at: Timestamp, out_dir: PathBuf, dry_run: bool, skip_llm: bool, @@ -369,23 +371,28 @@ async fn run_stages( // --- Stage 6: heuristic pre-filter (§3.5) --- let stage = Timestamp::now(); - let meter = UsageMeter::new(&config.deepseek, config.max_daily_usd); - // `max_daily_usd` is a ceiling for the *day*, not for one invocation, so a - // re-run inherits what earlier runs for this date already spent (§3.6). - match db.spend_for_date(date).await { - Ok(spent) if spent > 0.0 => { - tracing::info!(spent, "preloading today's recorded DeepSeek spend"); - meter.preload_cost(spent); + let bulk_meter = + UsageMeter::with_prices(PriceTable::deepseek(&config.deepseek), config.max_daily_usd); + let editor_meter = UsageMeter::with_prices( + PriceTable::anthropic(&config.anthropic), + config.anthropic.max_daily_usd, + ); + match db.provider_spend_for_utc_day(ctx.started_at).await { + Ok(spend) => { + bulk_meter.preload_cost(spend.get("deepseek").copied().unwrap_or(0.0)); + editor_meter.preload_cost(spend.get("anthropic").copied().unwrap_or(0.0)); + } + Err(error) => { + tracing::warn!(%error, "could not preload provider spend; starting from zero") } - Ok(_) => {} - Err(e) => tracing::warn!(error = %e, "could not read today's spend; starting from zero"), } - let llm = build_llm(ctx, &meter, report).await; - let llm_available = llm.is_some(); + let llms = build_llms(ctx, &bulk_meter, &editor_meter, report).await; + let bulk_available = llms.bulk.is_some(); let mut curator_config = config.clone(); - curator_config.target_article_count = ctx.target; - let curator = Curator::new(curator_config, db.clone(), llm); + curator_config.target_article_count = ctx.soft_target; + curator_config.curation.max_article_count = ctx.hard_max; + let curator = Curator::new(curator_config, db.clone(), llms); let mut candidates = curator .prefilter(articles, date) @@ -396,12 +403,13 @@ async fn run_stages( // --- Stage 7: LLM scoring, then selection (§3.6 A + B) --- let stage = Timestamp::now(); - if llm_available && let Err(e) = curator.score(&mut candidates, date).await { + if bulk_available && let Err(e) = curator.score(&mut candidates, date).await { // A dead API or a tripped budget must not cost us the issue: selection // degrades to prefilter order exactly as `--skip-llm` does. report.warn(format!("LLM scoring failed; ranking heuristically: {e:#}")); } report.counts.llm_scored = candidates.iter().filter(|c| c.llm.is_some()).count() as i64; + report.counts.llm_unscored = report.counts.candidates - report.counts.llm_scored; let mut lineup = curator .select(candidates, date) @@ -440,7 +448,7 @@ async fn run_stages( if config.world_briefing { match world_briefing.as_mut() { Some(briefing) => { - for warning in world::enrich(&http, briefing, curator.llm.as_ref()).await { + for warning in world::enrich(&http, briefing, curator.llms.bulk.as_ref()).await { report.warn(warning); } } @@ -454,19 +462,55 @@ async fn run_stages( .next_issue_number(date) .await .context("computing the issue number")?; + report.provider_costs.insert( + "deepseek".into(), + ProviderUsage { + usage: bulk_meter.total(), + cost_usd: bulk_meter.cost_usd(), + }, + ); + report.provider_costs.insert( + "anthropic".into(), + ProviderUsage { + usage: editor_meter.total(), + cost_usd: editor_meter.cost_usd(), + }, + ); + let summary_model = match config.editorial.summary_model { + crate::config::SummaryModel::Editor if curator.llms.editor.is_some() => { + config.anthropic.model.clone() + } + _ if curator.llms.bulk.is_some() => config.deepseek.model.clone(), + _ => "none".into(), + }; + let provider_costs = report + .provider_costs + .iter() + .map(|(provider, usage)| (provider.clone(), usage.cost_usd)) + .collect(); let colophon = Colophon { - model: if llm_available { - config.deepseek.model.clone() - } else { - "none (--skip-llm)".into() + provider_costs, + models: Models { + bulk: if bulk_available { + config.deepseek.model.clone() + } else { + "none".into() + }, + editor: if curator.llms.editor.is_some() { + config.anthropic.model.clone() + } else if bulk_available { + format!("{} (bulk fallback)", config.deepseek.model) + } else { + "none".into() + }, + summaries: summary_model, }, entries_fetched: report.counts.entries_fetched, feeds_seen: report.counts.feeds_seen, candidates: report.counts.candidates, - cost_usd: meter.cost_usd(), + cost_usd: bulk_meter.cost_usd() + editor_meter.cost_usd(), generator_version: format!("daily-epub {}", crate::VERSION), }; - report.usage = meter.total(); let issue = build_issue( date, issue_number, @@ -582,11 +626,12 @@ async fn record_issue(db: &Db, issue: &Issue, published: &Published) -> Result<( /// /// Returns `None` for `--skip-llm` and for every configuration/API problem: the /// caller then curates heuristically instead of failing the run (§3.6). -async fn build_llm( +async fn build_llms( ctx: &StageContext<'_>, - meter: &UsageMeter, + bulk_meter: &UsageMeter, + editor_meter: &UsageMeter, report: &mut RunReport, -) -> Option { +) -> Llms { let profile = match profile::load_or_build( ctx.db, &ctx.config.interests_opml, @@ -596,32 +641,36 @@ async fn build_llm( .await { Ok(profile) => profile, - Err(e) => { + Err(error) => { report.warn(format!( - "could not build the taste profile; curating heuristically: {e:#}" + "could not build the taste profile; curating heuristically: {error:#}" )); - return None; + return Llms::default(); } }; if ctx.skip_llm { - tracing::info!("--skip-llm: profile rebuilt; no DeepSeek call will be made"); - return None; + tracing::info!("--skip-llm: profile rebuilt; no provider calls will be made"); + return Llms::default(); } - let client = match LlmClient::new(&ctx.config.deepseek, profile.text, meter.clone()) { - Ok(client) => client, - Err(e) => { - report.warn(format!( - "DeepSeek is unavailable; curating heuristically: {e}" - )); - return None; - } + + let make_clients = |prompt: String| { + Llms::from_config( + &ctx.config.deepseek, + &ctx.config.anthropic, + prompt, + bulk_meter.clone(), + editor_meter.clone(), + ) }; - // Weekly rewrite of the "learned adjustments" section (§3.6c). It changes the - // system prompt, so the client is rebuilt around the new profile. + let mut llms = make_clients(profile.text); + let Some(rebuild_client) = llms.editor_or_bulk() else { + report.warn("no LLM provider is available; curating heuristically"); + return llms; + }; match profile::weekly_rebuild_if_due( ctx.db, - &client, + rebuild_client, &ctx.config.interests_opml, &ctx.config.profile_path, ctx.config.curation.feedback.verdicts_in_prompt, @@ -629,21 +678,78 @@ async fn build_llm( .await { Ok(Some(rebuilt)) => { - tracing::info!(version = rebuilt.version, "taste profile rebuilt"); - match LlmClient::new(&ctx.config.deepseek, rebuilt.text, meter.clone()) { - Ok(refreshed) => Some(refreshed), - Err(e) => { - tracing::warn!(error = %e, "keeping the previous profile client"); - Some(client) - } - } - } - Ok(None) => Some(client), - Err(e) => { - report.warn(format!("weekly profile rebuild failed: {e:#}")); - Some(client) + tracing::info!( + version = rebuilt.version, + "taste profile rebuilt with editor-or-bulk" + ); + llms = make_clients(rebuilt.text); } + Ok(None) => {} + Err(error) => report.warn(format!("weekly profile rebuild failed: {error:#}")), } + llms +} + +/// `--max-articles N` is a ceiling, never a target (§13): the hard ceiling is +/// the smaller of `curation.max_article_count` and `N`, and the soft target +/// never exceeds it. Returns `(soft_target, hard_max)`. +pub fn issue_size_bounds(config: &Config, max_articles: Option) -> (usize, usize) { + let hard_max = max_articles.map_or(config.curation.max_article_count, |ceiling| { + ceiling.min(config.curation.max_article_count) + }); + (config.target_article_count.min(hard_max), hard_max) +} + +/// Startup line naming the resolved models and whether each provider is on +/// (§19): the root config ignores unknown sections, so an `[anthropics]` typo +/// would otherwise be silent. Keys are never logged, only their presence. +fn log_resolved_providers(config: &Config, skip_llm: bool) { + let has_key = |key: Option<&str>| key.is_some_and(|k| !k.trim().is_empty()); + tracing::info!( + bulk_model = %config.deepseek.model, + bulk_enabled = !skip_llm && has_key(config.deepseek.api_key.as_deref()), + bulk_max_daily_usd = config.max_daily_usd, + editor_model = %config.anthropic.model, + editor_enabled = !skip_llm + && config.anthropic.enabled + && has_key(config.anthropic.api_key.as_deref()), + editor_effort = %config.anthropic.effort, + editor_max_daily_usd = config.anthropic.max_daily_usd, + summary_model = ?config.editorial.summary_model, + "resolved providers" + ); +} + +/// Prompt versions recorded per run so old telemetry stays interpretable (§7.6). +/// Bump a number when the corresponding instruction block changes. +const PROMPT_VERSIONS: &[(&str, u32)] = &[ + ("score", 1), + ("editor", 2), + ("summary", 1), + ("brief", 2), + ("profile", 2), +]; + +/// The resolved `[curation]`, `[editorial]`, model names and prompt versions +/// written to `runs.config_json` (§7.6, §19). Never includes keys. +fn resolved_run_config(config: &Config, soft_target: usize, hard_max: usize) -> serde_json::Value { + let mut curation = config.curation.clone(); + curation.max_article_count = hard_max; + serde_json::json!({ + "target_article_count": soft_target, + "prefilter_keep": config.prefilter_keep, + "curation": curation, + "editorial": config.editorial, + "models": { + "bulk": config.deepseek.model, + "editor": if config.anthropic.enabled { config.anthropic.model.as_str() } else { "disabled" }, + "editor_effort": config.anthropic.effort, + }, + "prompt_versions": PROMPT_VERSIONS + .iter() + .map(|(name, version)| ((*name).to_string(), serde_json::Value::from(*version))) + .collect::>(), + }) } fn elapsed_ms(since: Timestamp) -> i64 { @@ -689,6 +795,41 @@ mod tests { assert!(resolve_date(&config, None).is_ok()); } + #[test] + fn max_articles_is_a_ceiling_not_a_target() { + let config = Config { + target_article_count: 20, + ..Config::default() + }; + assert_eq!(config.curation.max_article_count, 28); + assert_eq!(issue_size_bounds(&config, None), (20, 28)); + // A ceiling below the target drags the target down with it. + assert_eq!(issue_size_bounds(&config, Some(6)), (6, 6)); + // A ceiling above the configured maximum does not raise it. + assert_eq!(issue_size_bounds(&config, Some(40)), (20, 28)); + assert_eq!(issue_size_bounds(&config, Some(24)), (20, 24)); + } + + #[test] + fn run_config_json_records_the_resolved_settings_and_no_keys() { + let mut config = Config::default(); + config.anthropic.api_key = Some("sk-secret".into()); + config.deepseek.api_key = Some("ds-secret".into()); + let value = resolved_run_config(&config, 6, 6); + assert_eq!(value["target_article_count"], 6); + assert_eq!(value["curation"]["max_article_count"], 6); + assert_eq!(value["editorial"]["summary_model"], "editor"); + assert_eq!(value["editorial"]["summary_input_tokens"], 3000); + assert_eq!(value["models"]["bulk"], "deepseek-v4-flash"); + assert_eq!(value["models"]["editor"], "claude-opus-5"); + assert!(value["prompt_versions"]["editor"].is_number()); + let text = value.to_string(); + assert!( + !text.contains("secret"), + "keys must never reach the database" + ); + } + #[test] fn issue_meta_is_derived_from_the_lineup() { let lineup = crate::epub::build::fixtures::issue().lineup; diff --git a/src/report.rs b/src/report.rs index ef0a5e6..389c354 100644 --- a/src/report.rs +++ b/src/report.rs @@ -71,6 +71,8 @@ pub struct StageCounts { pub candidates: i64, /// Articles scored by the LLM (§3.6 stage A). pub llm_scored: i64, + /// Candidates left unscored after failures or a bulk-provider budget trip (§5). + pub llm_unscored: i64, /// Articles in the final lineup (§3.6 stage B). pub selected: i64, /// Discussion chapters rendered (§3.7). @@ -93,6 +95,14 @@ impl StageTimings { } } +/// Usage and computed cost for one provider in this run (§5). +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct ProviderUsage { + #[serde(flatten)] + pub usage: TokenUsage, + pub cost_usd: f64, +} + /// The full summary of one `generate` invocation (§3.13). #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct RunReport { @@ -101,7 +111,12 @@ pub struct RunReport { pub finished_at: Option, pub status: RunStatus, pub counts: StageCounts, + /// Aggregate usage retained for the legacy `runs` columns. pub usage: TokenUsage, + /// Provider-keyed usage and cost written to `runs.provider_costs_json`. + pub provider_costs: BTreeMap, + /// Resolved curation/editorial/model settings for this run. + pub config_json: serde_json::Value, pub cost_usd: f64, pub timings: StageTimings, /// Ingest window actually used, RFC3339 (§3.1). @@ -124,6 +139,8 @@ impl RunReport { status: RunStatus::Running, counts: StageCounts::default(), usage: TokenUsage::default(), + provider_costs: BTreeMap::new(), + config_json: serde_json::Value::Null, cost_usd: 0.0, timings: StageTimings::default(), window_start: None, @@ -146,16 +163,15 @@ impl RunReport { self.error = Some(err.to_string()); } - /// Stamp the end time, compute cost from [`TokenUsage`] and settle the status. - pub fn finish( - &mut self, - finished_at: Timestamp, - price_input: f64, - price_cached: f64, - price_output: f64, - ) { + /// Stamp the end time, total provider costs and settle the status. + pub fn finish(&mut self, finished_at: Timestamp) { self.finished_at = Some(finished_at); - self.cost_usd = self.usage.cost_usd(price_input, price_cached, price_output); + self.usage = TokenUsage::default(); + self.cost_usd = 0.0; + for provider in self.provider_costs.values() { + self.usage.add(provider.usage); + self.cost_usd += provider.cost_usd; + } if self.status == RunStatus::Running { self.status = if self.warnings.is_empty() { RunStatus::Ok @@ -215,17 +231,37 @@ mod tests { s.parse().unwrap() } + fn usage(input: i64, cached: i64, cache_write: i64, output: i64) -> TokenUsage { + TokenUsage { + input_tokens: input, + cached_tokens: cached, + cache_write_tokens: cache_write, + output_tokens: output, + } + } + #[test] - fn finish_computes_cost_and_status() { + fn finish_totals_provider_costs_and_settles_status() { let mut r = RunReport::new("2026-08-15".parse().unwrap(), ts("2026-08-15T05:30:00Z")); - r.usage.add(TokenUsage { - input_tokens: 1_000_000, - cached_tokens: 1_000_000, - output_tokens: 1_000_000, - }); - r.finish(ts("2026-08-15T05:36:00Z"), 0.14, 0.0028, 0.28); + r.provider_costs.insert( + "deepseek".into(), + ProviderUsage { + usage: usage(1_000_000, 1_000_000, 0, 1_000_000), + cost_usd: 0.4228, + }, + ); + r.provider_costs.insert( + "anthropic".into(), + ProviderUsage { + usage: usage(100, 3_000, 2_000, 800), + cost_usd: 0.05, + }, + ); + r.finish(ts("2026-08-15T05:36:00Z")); assert_eq!(r.status, RunStatus::Ok); - assert!((r.cost_usd - 0.4228).abs() < 1e-9); + assert!((r.cost_usd - 0.4728).abs() < 1e-9); + // The legacy aggregate columns are the sum across providers. + assert_eq!(r.usage, usage(1_000_100, 1_003_000, 2_000, 1_000_800)); assert_eq!(r.duration_secs(), Some(360)); } @@ -233,7 +269,7 @@ mod tests { fn warnings_degrade_the_run() { let mut r = RunReport::new("2026-08-15".parse().unwrap(), ts("2026-08-15T05:30:00Z")); r.warn("xtc converter missing"); - r.finish(ts("2026-08-15T05:31:00Z"), 0.14, 0.0028, 0.28); + r.finish(ts("2026-08-15T05:31:00Z")); assert_eq!(r.status, RunStatus::Degraded); assert_eq!(r.warnings.len(), 1); } @@ -242,15 +278,31 @@ mod tests { fn serializes_round_trip() { let mut r = RunReport::new("2026-08-15".parse().unwrap(), ts("2026-08-15T05:30:00Z")); r.counts.entries_fetched = 412; + r.counts.llm_unscored = 3; r.per_feed_counts.insert("Hacker News".into(), 30); r.per_feed_counts.insert("Lobsters".into(), 12); r.timings.record("ingest", 1500); - r.finish(ts("2026-08-15T05:31:00Z"), 0.14, 0.0028, 0.28); + r.config_json = serde_json::json!({"models": {"editor": "claude-opus-5"}}); + r.provider_costs.insert( + "anthropic".into(), + ProviderUsage { + usage: usage(1, 2, 3, 4), + cost_usd: 0.01, + }, + ); + r.finish(ts("2026-08-15T05:31:00Z")); let json = r.to_json(); let back: RunReport = serde_json::from_str(&json).unwrap(); assert_eq!(back, r); assert_eq!(back.top_feeds(1), vec![("Hacker News", 30)]); assert_eq!(back.timings.total_ms(), 1500); assert!(back.summary_line().contains("412 entries")); + // `ProviderUsage` flattens the token counts next to the cost. + let value: serde_json::Value = serde_json::from_str(&json).unwrap(); + assert_eq!( + value["provider_costs"]["anthropic"]["cache_write_tokens"], + 3 + ); + assert_eq!(value["provider_costs"]["anthropic"]["cost_usd"], 0.01); } } diff --git a/src/types.rs b/src/types.rs index 32a6949..1db0840 100644 --- a/src/types.rs +++ b/src/types.rs @@ -292,6 +292,8 @@ pub struct Pick { /// Order within the section, ascending. pub position: i64, pub is_lead: bool, + /// Editor-written reason, at most 14 words (§13). + pub why: Option, /// Newspaper-abstract summary from stage C; `None` until editorial runs. pub summary: Option, pub llm: Option, @@ -331,8 +333,6 @@ impl Lineup { pub struct Editorial { /// "From the Editor", 250–400 words, already sanitized XHTML. pub front_page_html: String, - /// Section name → 2–3 sentence intro. - pub section_intros: BTreeMap, /// Article id → 2–3 sentence newspaper abstract. pub summaries: BTreeMap, } @@ -525,10 +525,19 @@ pub struct Issue { pub colophon: Colophon, } +/// Resolved model names printed in the colophon (§15.1). +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct Models { + pub bulk: String, + pub editor: String, + pub summaries: String, +} + /// Back-matter facts printed in the colophon chapter (§3.10). #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] pub struct Colophon { - pub model: String, + pub provider_costs: BTreeMap, + pub models: Models, pub entries_fetched: i64, pub feeds_seen: i64, pub candidates: i64, @@ -653,8 +662,10 @@ pub struct RatedArticle { pub struct TokenUsage { /// Cache-miss input tokens (billed at the full input rate). pub input_tokens: i64, - /// Prefix-cache hits (billed at the cached rate). + /// Prefix-cache reads (billed at the provider's cache-read rate). pub cached_tokens: i64, + /// Tokens written into a prompt cache (Anthropic only). + pub cache_write_tokens: i64, pub output_tokens: i64, } @@ -662,13 +673,21 @@ impl TokenUsage { pub fn add(&mut self, other: TokenUsage) { self.input_tokens += other.input_tokens; self.cached_tokens += other.cached_tokens; + self.cache_write_tokens += other.cache_write_tokens; self.output_tokens += other.output_tokens; } /// USD cost given the per-1M-token prices from `[deepseek]` config (§3.6). - pub fn cost_usd(&self, price_input: f64, price_cached: f64, price_output: f64) -> f64 { + pub fn cost_usd( + &self, + price_input: f64, + price_cache_write: f64, + price_cache_read: f64, + price_output: f64, + ) -> f64 { (self.input_tokens as f64 * price_input - + self.cached_tokens as f64 * price_cached + + self.cache_write_tokens as f64 * price_cache_write + + self.cached_tokens as f64 * price_cache_read + self.output_tokens as f64 * price_output) / 1_000_000.0 } diff --git a/tests/e2e_pipeline.rs b/tests/e2e_pipeline.rs index 81c07a1..4444c9a 100644 --- a/tests/e2e_pipeline.rs +++ b/tests/e2e_pipeline.rs @@ -18,18 +18,19 @@ //! no article in the fixtures carries an image, so the EPUB builder's image //! downloader has nothing to fetch. +use std::collections::BTreeMap; use std::path::Path; use jiff::Timestamp; use jiff::civil::Date; use daily_epub::config::{Config, PublishConfig, ServerConfig, XtcConfig}; -use daily_epub::curate::llm::{LlmClient, MockBackend, UsageMeter}; +use daily_epub::curate::llm::{LlmClient, Llms, MockBackend, UsageMeter}; use daily_epub::curate::{Curator, editorial, prefilter}; use daily_epub::db::Db; use daily_epub::extract::Extractor; use daily_epub::types::{ - Article, Colophon, Edition, Entry, Issue, Lineup, ScoredArticle, SourceKind, Vote, + Article, Colophon, Edition, Entry, Issue, Lineup, Models, ScoredArticle, SourceKind, Vote, }; use daily_epub::{auth, dedupe, epub, miniflux, pipeline, publish}; @@ -400,7 +401,7 @@ async fn skip_llm_pipeline_produces_a_published_issue() { let articles = ingest_dedupe_extract_persist(&db).await; // --- Stages 6–7 with no LLM at all (notes §6) --- - let curator = Curator::new(cfg.clone(), db.clone(), None); + let curator = Curator::new(cfg.clone(), db.clone(), Llms::default()); let candidates = curator .prefilter(articles, date()) .await @@ -432,7 +433,12 @@ async fn skip_llm_pipeline_produces_a_published_issue() { ); let colophon = Colophon { - model: "none (--skip-llm)".into(), + provider_costs: BTreeMap::new(), + models: Models { + bulk: "none".into(), + editor: "none".into(), + summaries: "none".into(), + }, entries_fetched: 8, feeds_seen: 8, candidates: 5, @@ -504,6 +510,7 @@ async fn llm_pipeline_runs_against_a_mock_backend() { let usage = daily_epub::types::TokenUsage { input_tokens: 1000, cached_tokens: 500, + cache_write_tokens: 0, output_tokens: 200, }; let scores: Vec = ids @@ -544,8 +551,7 @@ async fn llm_pipeline_runs_against_a_mock_backend() { ); } backend.push( - r#"{"from_the_editor": "Today's issue leans on storage internals.\n\nRead on.", - "section_intros": {"Top Stories": "The day in one place."}}"#, + r#"{"brief": "Today's issue leans on storage internals.\n\nRead on."}"#, usage, ); @@ -556,7 +562,14 @@ async fn llm_pipeline_runs_against_a_mock_backend() { meter.clone(), backend.clone(), ); - let curator = Curator::new(cfg.clone(), db.clone(), Some(llm)); + let curator = Curator::new( + cfg.clone(), + db.clone(), + Llms { + bulk: Some(llm), + editor: None, + }, + ); let mut candidates = candidates; curator @@ -590,12 +603,9 @@ async fn llm_pipeline_runs_against_a_mock_backend() { "the model's summaries were used, not excerpts" ); assert!(editorial_doc.front_page_html.contains("storage internals")); - assert_eq!( - editorial_doc - .section_intros - .get("Top Stories") - .map(String::as_str), - Some("The day in one place.") + assert!( + lineup.picks.iter().all(|p| p.why.is_none()), + "the scripted editor gave no why lines" ); // Every scripted response was consumed, and the meter priced them (§3.6). @@ -615,7 +625,12 @@ async fn llm_pipeline_runs_against_a_mock_backend() { // And it all assembles, builds and publishes like the skip-llm route does. let colophon = Colophon { - model: cfg.deepseek.model.clone(), + provider_costs: BTreeMap::from([("deepseek".to_string(), meter.cost_usd())]), + models: Models { + bulk: cfg.deepseek.model.clone(), + editor: format!("{} (bulk fallback)", cfg.deepseek.model), + summaries: cfg.deepseek.model.clone(), + }, entries_fetched: 8, feeds_seen: 8, candidates: 5, @@ -625,6 +640,6 @@ async fn llm_pipeline_runs_against_a_mock_backend() { let mut lineup = lineup; pipeline::apply_summaries(&mut lineup, &editorial_doc); let issue = assemble_build_publish(&db, &cfg, lineup, colophon).await; - assert_eq!(issue.colophon.model, cfg.deepseek.model); + assert_eq!(issue.colophon.models.bulk, cfg.deepseek.model); assert!(issue.colophon.cost_usd > 0.0); } diff --git a/tests/fixtures/claude_brief.json b/tests/fixtures/claude_brief.json new file mode 100644 index 0000000..de6f7b5 --- /dev/null +++ b/tests/fixtures/claude_brief.json @@ -0,0 +1,3 @@ +{ + "brief": "The lead, \"Migrating 40TB off Postgres\", is the rare migration write-up that keeps its failures in: two aborted cutovers, the rollback that took longer than the move, and a bill at the end. Read it first while the coffee is hot; it rewards attention and it is long.\n\nThe local desk answers with \"The MBTA slow-zone dataset\", which finally puts the T's own numbers into a shape a rider can argue with, and the charts do more persuading than a year of press releases. \"A failover story you'd argue with\" rounds out the engineering pages with a Postgres HA design that disagrees with the lead on almost every point, which is exactly why the two belong in the same issue. The issue is shorter than usual because a thin Friday is a good excuse to finish the long one properly rather than skim six." +} diff --git a/tests/fixtures/deepseek_front_page.json b/tests/fixtures/deepseek_front_page.json deleted file mode 100644 index 89016d8..0000000 --- a/tests/fixtures/deepseek_front_page.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "from_the_editor": "Two of today's pieces are, underneath, the same story: what it costs to move data you no longer trust. The lead — a team hauling forty terabytes off Postgres, rollback plans and all — is the version with the invoices attached, and it earns the front page by refusing to tidy up its failures. Read it first, while the coffee is hot; it rewards attention and it is long.\n\nThe local desk supplies the counterpoint. Somebody has finally put the MBTA's slow-zone data into a shape a rider can argue with, and the charts do more persuading than a year of press releases. It is a short read and a satisfying one, and it pairs unreasonably well with the migration story: both are about institutions discovering what they actually have.\n\nThe rest of the issue is quieter than usual. That is not a complaint — a thin Friday is a good excuse to finish the long one properly rather than skimming six. If you only get through the lead today, you will not have missed much else.", - "section_intros": { - "Top Stories": "The day's most substantial piece: a full account of a forty-terabyte migration, with the failures left in. It is long, technical and unusually honest about what went wrong.", - "Boston & Local": "Transit data gets the treatment it deserves. A rider-built analysis of MBTA slow zones, with charts you can check yourself and a methodology section that holds up.", - "Niche Corner": "A section the model wrote an intro for even though nothing was placed in it today — a stray thread about tape-drive firmware and the people who still maintain it. The issue drops intros for sections that never ran." - } -} diff --git a/tests/fixtures/deepseek_lineup.json b/tests/fixtures/deepseek_lineup.json index 648ff0a..48d4220 100644 --- a/tests/fixtures/deepseek_lineup.json +++ b/tests/fixtures/deepseek_lineup.json @@ -1,10 +1,10 @@ { "picks": [ - { "id": 101, "section": "Top Stories", "position": 1, "lead_story": true }, - { "id": 102, "section": "Tech & Engineering", "position": 1, "lead_story": false }, - { "id": 103, "section": "Tech & Engineering", "position": 2, "lead_story": false }, - { "id": 104, "section": "AI & Machine Learning", "position": 1, "lead_story": false }, - { "id": 105, "section": "Boston & Local", "position": 1, "lead_story": false }, + { "id": 101, "section": "Top Stories", "position": 1, "lead_story": true, "why": "The migration post-mortem with the invoices still attached" }, + { "id": 102, "section": "Tech & Engineering", "position": 1, "lead_story": false, "why": "A failover story you'd argue with over coffee" }, + { "id": 103, "section": "Tech & Engineering", "position": 2, "lead_story": false, "why": "Rare first-hand detail on a tool you use daily" }, + { "id": 104, "section": "AI & Machine Learning", "position": 1, "lead_story": false, "why": "The one benchmark piece this week that shows its work" }, + { "id": 105, "section": "Boston & Local", "position": 1, "lead_story": false, "why": "MBTA slow zones charted by a rider, not a press office" }, { "id": 106, "section": "Culture & Essays", "position": 1 }, { "section": "Niche Corner", "position": 2, "lead_story": false }, "the model sometimes trails off like this" diff --git a/tests/m3_curation.rs b/tests/m3_curation.rs index da0d886..895a985 100644 --- a/tests/m3_curation.rs +++ b/tests/m3_curation.rs @@ -17,7 +17,7 @@ use std::collections::BTreeSet; use std::path::{Path, PathBuf}; use std::process::Command; -use daily_epub::curate::editorial::FrontPageResponse; +use daily_epub::curate::editorial::BriefResponse; use daily_epub::curate::profile; use daily_epub::curate::score::parse_score_response; use daily_epub::curate::select::parse_selection_response; @@ -130,35 +130,41 @@ fn stage_b_fixture_parses_into_a_lineup() { ); } -/// Stage C's front-page response must deserialize into a 250–400 word editor's -/// note plus per-section intros (§3.6). +/// The Brief must deserialize into 120–200 words of plain prose that names at +/// least three picks by title (§14.2). Section intros are gone. #[test] -fn stage_c_fixture_parses_into_a_front_page() { - let response: FrontPageResponse = serde_json::from_str(&fixture("deepseek_front_page.json")) - .expect("the front-page fixture must match FrontPageResponse"); +fn stage_c_fixture_parses_into_the_brief() { + let response: BriefResponse = serde_json::from_str(&fixture("claude_brief.json")) + .expect("the brief fixture must match BriefResponse"); - let words = response.from_the_editor.split_whitespace().count(); + let words = response.brief.split_whitespace().count(); assert!( - (150..=450).contains(&words), - "From the Editor is {words} words; the prompt asks for 250-400" + (100..=220).contains(&words), + "The Brief is {words} words; the prompt asks for 120-200" ); assert!( - response.from_the_editor.contains("\n\n"), - "the prompt asks for 2-4 blank-line separated paragraphs" + !response.brief.contains("- ") && !response.brief.contains('#'), + "no bullets or headings in the brief" ); + let titles = response.brief.matches('"').count() / 2; assert!( - !response.from_the_editor.contains("- "), - "no bullet lists on the front page" + titles >= 3, + "the brief names at least three picks; found {titles}" ); - - assert!(response.section_intros.len() >= 2); - for (section, intro) in &response.section_intros { - let words = intro.split_whitespace().count(); + for banned in [ + "delve", + "dive", + "explore", + "a mix of", + "something for everyone", + ] { assert!( - (10..=90).contains(&words), - "intro for {section} is {words} words; the prompt asks for 35-60" + !response.brief.to_lowercase().contains(banned), + "banned phrase {banned}" ); } + let value: serde_json::Value = serde_json::from_str(&fixture("claude_brief.json")).unwrap(); + assert!(value.get("section_intros").is_none()); } /// The taste profile is seeded from this file; a broken export would silently diff --git a/tests/m4_epub.rs b/tests/m4_epub.rs index 3879267..d517e93 100644 --- a/tests/m4_epub.rs +++ b/tests/m4_epub.rs @@ -320,7 +320,14 @@ fn colophon_facts_are_x4_safe_distinct_paragraphs() { for edition in [Edition::Standard, Edition::X4] { let (_dir, _, zip) = build_edition_to_bytes(&issue, edition); let colophon = read_entry(&zip, "OEBPS/colophon.xhtml"); - assert_eq!(colophon.matches("

").count(), 9); + // Issue, generated, three model lines, entries, candidates, articles, + // words, two per-provider cost lines, the total, generator (§15.1). + assert_eq!(colophon.matches("

").count(), 13); + assert!(colophon.contains("Editor model: claude-opus-5")); + assert!(colophon.contains("Bulk model: deepseek-v4-flash")); + assert!(colophon.contains("anthropic cost: $0.0500")); + assert!(colophon.contains("deepseek cost: $0.0231")); + assert!(colophon.contains("Total token cost:")); assert!(!colophon.contains("