Initial commit: The Daily EPUB full implementation
Full implementation of a personalized daily newspaper delivered as an EPUB. Articles are pulled from a local self-hosted Miniflux instance, enriched with comments, summarized and filtered by DeepSeek AI, and then assembled into two EPUB editions: standard and optimized for the Xteink X4 e-ink reader. Both are served by the local self-hosted BookOrbit OPDS server in a separate library. Then the X4 edition is futher converted to XTC format and served over a separate OPDS server hosted by the Rust binary. Runs are tracked in a local SQLite database so runs are idempotent per date. Full documentation of the plan is in docs/plans and setup and install instructions are in the README.md file.
This commit is contained in:
@@ -0,0 +1,634 @@
|
||||
//! Stage C — summaries, section intros and the front page (spec §3.6).
|
||||
//!
|
||||
//! Voice: warm, literate, a little playful; never fabricates facts that are not
|
||||
//! present in the summaries.
|
||||
//!
|
||||
//! Everything here is best-effort. If the cost ceiling trips mid-way (§3.6) or a
|
||||
//! call fails, the affected article silently falls back to its own opening words
|
||||
//! and the run continues — an issue with plain excerpts is far better than no
|
||||
//! issue at all.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::fmt::Write as _;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::llm::{LlmClient, LlmError};
|
||||
use super::{escape_html, html_to_text, text_to_paragraphs, truncate_tokens, truncate_words};
|
||||
use crate::types::{ArticleId, Editorial, Lineup, Pick};
|
||||
|
||||
/// Article text is truncated to roughly this many tokens per summary call (§3.6).
|
||||
pub const SUMMARY_INPUT_TOKEN_BUDGET: usize = 5000;
|
||||
/// Target length of the "From the Editor" front page, in words (§3.6).
|
||||
pub const FRONT_PAGE_WORDS: (usize, usize) = (250, 400);
|
||||
/// Words of body text used when a summary has to fall back to the excerpt.
|
||||
pub const FALLBACK_SUMMARY_WORDS: usize = 45;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Prompts (reusable instructions here; per-call material in the user message)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Per-article summary instructions (§3.6 stage C).
|
||||
pub const SUMMARY_INSTRUCTIONS: &str = "\
|
||||
TASK: write the newspaper abstract for one article in today's issue.
|
||||
|
||||
Two or three sentences, 40–70 words, present tense, third person. It runs under \
|
||||
the headline in the \"In This Issue\" page, so the reader decides from it alone \
|
||||
whether to open the piece.
|
||||
|
||||
DO
|
||||
- Say what the article actually argues, reports or builds — the specific claim, \
|
||||
number, method or story, not the topic.
|
||||
- Add the one detail that makes it worth his time: the surprising result, the \
|
||||
scale, the person involved, the unusual method.
|
||||
- Match the piece's register: a technical post-mortem gets a technical abstract, \
|
||||
an essay gets an essayistic one.
|
||||
- Stay strictly inside the supplied text.
|
||||
|
||||
DO NOT
|
||||
- Tease (\"you won't believe what happens next\"), moralize, or address the \
|
||||
reader as \"you\".
|
||||
- Open with \"This article…\", \"The author…\", \"In this post…\", or repeat the \
|
||||
headline's words.
|
||||
- Invent facts, names, numbers or conclusions that are not in the text. If the \
|
||||
text is a truncated excerpt, summarize only what is there and say it is an \
|
||||
excerpt.
|
||||
- Recommend, rate or editorialize — that is the front page's job.
|
||||
|
||||
Return JSON exactly: {\"summary\": \"<two or three sentences>\"}";
|
||||
|
||||
/// Front-page + section-intro instructions (§3.6 stage C).
|
||||
pub const FRONT_PAGE_INSTRUCTIONS: &str = "\
|
||||
TASK: write the front page of today's issue of The Daily EPUB.
|
||||
|
||||
You are given the whole lineup: sections, headlines, sources and the abstract \
|
||||
written for each article. Everything you write must come from those abstracts — \
|
||||
you have not read the articles themselves, and inventing a fact would be worse \
|
||||
than saying less.
|
||||
|
||||
Produce two things.
|
||||
|
||||
1. \"from_the_editor\" — 250 to 400 words of prose addressed to the paper's one \
|
||||
reader. Find the two or three threads that actually run through today's lineup \
|
||||
(a shared question, an argument between two pieces, an accidental theme) and use \
|
||||
them to guide the read: what to start with over coffee, what to save for the \
|
||||
commute, what rewards patience. Name the lead story and say why it leads. It is \
|
||||
fine — good, even — to note when a day is quiet or lopsided. Voice: warm, \
|
||||
literate, lightly playful, never breathless; a real editor writing to someone \
|
||||
whose taste he knows. No bullet lists, no headings, no emoji, 2–4 paragraphs \
|
||||
separated by a blank line.
|
||||
|
||||
2. \"section_intros\" — for EACH section name given below, two or three \
|
||||
sentences (35–60 words) introducing what is in it today. Concrete, specific to \
|
||||
these articles, no filler like \"a variety of interesting stories\". Use the \
|
||||
section names exactly as spelled in the lineup.
|
||||
|
||||
Return JSON exactly:
|
||||
{\"from_the_editor\": \"<paragraphs separated by \\n\\n>\", \
|
||||
\"section_intros\": {\"<section name>\": \"<2-3 sentences>\"}}";
|
||||
|
||||
/// The single front-page call's JSON response (§3.6).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct FrontPageResponse {
|
||||
/// "From the Editor", 250–400 words.
|
||||
pub from_the_editor: String,
|
||||
/// Section name → 2–3 sentence intro.
|
||||
#[serde(default)]
|
||||
pub section_intros: BTreeMap<String, String>,
|
||||
}
|
||||
|
||||
/// The per-article summary call's JSON response.
|
||||
#[derive(Debug, Clone, Default, Deserialize)]
|
||||
struct SummaryResponse {
|
||||
#[serde(default)]
|
||||
summary: String,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Per-article summaries
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One 2–3 sentence newspaper abstract: what it argues, why it's worth reading (§3.6).
|
||||
pub async fn summarize_article(
|
||||
llm: &LlmClient,
|
||||
title: &str,
|
||||
body_html: &str,
|
||||
temperature: f32,
|
||||
) -> Result<String, LlmError> {
|
||||
llm.meter.check_budget()?;
|
||||
let body = truncate_tokens(&html_to_text(body_html), SUMMARY_INPUT_TOKEN_BUDGET);
|
||||
let mut prompt = String::with_capacity(body.len() + SUMMARY_INSTRUCTIONS.len() + 256);
|
||||
prompt.push_str(SUMMARY_INSTRUCTIONS);
|
||||
let _ = write!(
|
||||
prompt,
|
||||
"\n\nHEADLINE: {}\n\nARTICLE TEXT{}:\n{}\n",
|
||||
title.trim(),
|
||||
if body.ends_with('…') {
|
||||
" (truncated for length)"
|
||||
} else {
|
||||
""
|
||||
},
|
||||
if body.is_empty() {
|
||||
"(no body text was extracted; summarize from the headline alone and say the \
|
||||
full text was unavailable)"
|
||||
} else {
|
||||
&body
|
||||
}
|
||||
);
|
||||
let response: SummaryResponse = llm.complete_json(&prompt, temperature).await?;
|
||||
let summary = response.summary.trim().to_string();
|
||||
if summary.is_empty() {
|
||||
return Err(LlmError::EmptyResponse);
|
||||
}
|
||||
Ok(summary)
|
||||
}
|
||||
|
||||
/// Summarize every pick, returning `article_id → summary` (§3.6).
|
||||
///
|
||||
/// Stops early and returns what it has when the cost guardrail trips (§3.6).
|
||||
pub async fn summarize_all(
|
||||
llm: &LlmClient,
|
||||
lineup: &Lineup,
|
||||
temperature: f32,
|
||||
) -> BTreeMap<ArticleId, String> {
|
||||
let mut out = BTreeMap::new();
|
||||
for (n, pick) in lineup.picks.iter().enumerate() {
|
||||
if llm.meter.budget_exceeded() {
|
||||
tracing::error!(
|
||||
summarized = out.len(),
|
||||
remaining = lineup.picks.len() - n,
|
||||
spent_usd = llm.meter.cost_usd(),
|
||||
"COST CEILING HIT during stage C — the remaining articles fall back to \
|
||||
feed excerpts as summaries"
|
||||
);
|
||||
break;
|
||||
}
|
||||
match summarize_article(
|
||||
llm,
|
||||
&pick.article.title,
|
||||
&pick.article.content_html,
|
||||
temperature,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(summary) => {
|
||||
out.insert(pick.article.id, summary);
|
||||
}
|
||||
Err(LlmError::BudgetExceeded { spent, limit }) => {
|
||||
tracing::error!(spent, limit, "COST CEILING HIT during stage C");
|
||||
break;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
article_id = pick.article.id,
|
||||
title = %pick.article.title,
|
||||
error = %e,
|
||||
"summary failed; falling back to the article's own opening"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
tracing::info!(
|
||||
summarized = out.len(),
|
||||
picks = lineup.picks.len(),
|
||||
"stage C summaries complete"
|
||||
);
|
||||
out
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Front page
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The single front-page + section-intro call (§3.6).
|
||||
pub async fn front_page(
|
||||
llm: &LlmClient,
|
||||
lineup: &Lineup,
|
||||
summaries: &BTreeMap<ArticleId, String>,
|
||||
temperature: f32,
|
||||
) -> Result<FrontPageResponse, LlmError> {
|
||||
llm.meter.check_budget()?;
|
||||
let prompt = build_front_page_prompt(lineup, summaries);
|
||||
tracing::debug!(
|
||||
approx_tokens = super::approx_tokens(&prompt),
|
||||
"stage C front-page request"
|
||||
);
|
||||
let mut response: FrontPageResponse = llm.complete_json(&prompt, temperature).await?;
|
||||
response.from_the_editor = response.from_the_editor.trim().to_string();
|
||||
if response.from_the_editor.is_empty() {
|
||||
return Err(LlmError::EmptyResponse);
|
||||
}
|
||||
// Keep only intros for sections that actually exist in the issue.
|
||||
response
|
||||
.section_intros
|
||||
.retain(|name, text| lineup.section_order.contains(name) && !text.trim().is_empty());
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
/// Render the front-page user prompt: the whole lineup with its abstracts (§3.6).
|
||||
pub fn build_front_page_prompt(lineup: &Lineup, summaries: &BTreeMap<ArticleId, String>) -> String {
|
||||
let mut prompt = String::with_capacity(4096);
|
||||
prompt.push_str(FRONT_PAGE_INSTRUCTIONS);
|
||||
let minutes: i64 = lineup
|
||||
.picks
|
||||
.iter()
|
||||
.map(|p| p.article.reading_minutes())
|
||||
.sum();
|
||||
let _ = write!(
|
||||
prompt,
|
||||
"\n\nISSUE: {} · {} articles across {} sections · about {} minutes of reading\n\
|
||||
SECTIONS, in order: {}\n\nLINEUP\n",
|
||||
lineup.date,
|
||||
lineup.picks.len(),
|
||||
lineup.section_order.len(),
|
||||
minutes,
|
||||
lineup.section_order.join(" | ")
|
||||
);
|
||||
for section in &lineup.section_order {
|
||||
let _ = write!(prompt, "\n## {section}\n");
|
||||
for pick in lineup.section_picks(section) {
|
||||
let _ = write!(prompt, "{}", render_pick(pick, summaries));
|
||||
}
|
||||
}
|
||||
prompt
|
||||
}
|
||||
|
||||
fn render_pick(pick: &Pick, summaries: &BTreeMap<ArticleId, String>) -> String {
|
||||
let a = &pick.article;
|
||||
let mut block = String::with_capacity(400);
|
||||
let _ = writeln!(
|
||||
block,
|
||||
"\n- {}{}",
|
||||
a.title.trim(),
|
||||
if pick.is_lead { " [LEAD STORY]" } else { "" }
|
||||
);
|
||||
let _ = writeln!(
|
||||
block,
|
||||
" source: {} · {} words (~{} min){}",
|
||||
if a.feed_title.is_empty() {
|
||||
"unknown"
|
||||
} else {
|
||||
a.feed_title.trim()
|
||||
},
|
||||
a.word_count,
|
||||
a.reading_minutes(),
|
||||
social_note(pick)
|
||||
);
|
||||
let abstract_text = summaries
|
||||
.get(&a.id)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| excerpt_summary(pick));
|
||||
let _ = writeln!(block, " abstract: {abstract_text}");
|
||||
block
|
||||
}
|
||||
|
||||
fn social_note(pick: &Pick) -> String {
|
||||
if pick.article.social.is_empty() {
|
||||
return String::new();
|
||||
}
|
||||
let parts: Vec<String> = pick
|
||||
.article
|
||||
.social
|
||||
.iter()
|
||||
.map(|s| {
|
||||
format!(
|
||||
"{} {} pts/{} comments",
|
||||
s.source.display_name(),
|
||||
s.score,
|
||||
s.num_comments
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
format!(" · {}", parts.join(", "))
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fallbacks (§3.6, notes §6)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The article's own opening words, used when no LLM summary exists (§3.6).
|
||||
pub fn excerpt_summary(pick: &Pick) -> String {
|
||||
let text = truncate_words(
|
||||
&html_to_text(&pick.article.content_html),
|
||||
FALLBACK_SUMMARY_WORDS,
|
||||
);
|
||||
if text.is_empty() {
|
||||
format!(
|
||||
"From {}. (No preview text was available; open the article to read it.)",
|
||||
if pick.article.feed_title.is_empty() {
|
||||
"an unknown feed"
|
||||
} else {
|
||||
pick.article.feed_title.trim()
|
||||
}
|
||||
)
|
||||
} else {
|
||||
text
|
||||
}
|
||||
}
|
||||
|
||||
/// A plain, factual front page used when the model is unavailable (§3.6, notes §6).
|
||||
pub fn fallback_front_page_html(lineup: &Lineup) -> String {
|
||||
let minutes: i64 = lineup
|
||||
.picks
|
||||
.iter()
|
||||
.map(|p| p.article.reading_minutes())
|
||||
.sum();
|
||||
let mut text = format!(
|
||||
"Today's issue collects {} articles across {} sections — about {} minutes of \
|
||||
reading. Editorial notes are unavailable for this issue, so the lineup speaks \
|
||||
for itself.",
|
||||
lineup.picks.len(),
|
||||
lineup.section_order.len(),
|
||||
minutes
|
||||
);
|
||||
if let Some(lead) = lineup.lead() {
|
||||
let _ = write!(
|
||||
text,
|
||||
"\n\nLeading today: “{}” ({}).",
|
||||
lead.article.title.trim(),
|
||||
if lead.article.feed_title.is_empty() {
|
||||
"source unknown"
|
||||
} else {
|
||||
lead.article.feed_title.trim()
|
||||
}
|
||||
);
|
||||
}
|
||||
if !lineup.section_order.is_empty() {
|
||||
let _ = write!(
|
||||
text,
|
||||
"\n\nIn this issue: {}.",
|
||||
lineup.section_order.join(", ")
|
||||
);
|
||||
}
|
||||
text_to_paragraphs(&text)
|
||||
}
|
||||
|
||||
/// `--skip-llm` / budget-exceeded fallback: feed excerpts stand in for summaries
|
||||
/// and the front page is a plain stats line (§3.6, notes §6).
|
||||
pub fn fallback_editorial(lineup: &Lineup) -> Editorial {
|
||||
Editorial {
|
||||
front_page_html: fallback_front_page_html(lineup),
|
||||
section_intros: BTreeMap::new(),
|
||||
summaries: lineup
|
||||
.picks
|
||||
.iter()
|
||||
.map(|pick| (pick.article.id, excerpt_summary(pick)))
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Stage driver
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Stage C end to end: summaries, then one front-page call, with excerpts filling
|
||||
/// every gap (§3.6).
|
||||
pub async fn run(llm: &LlmClient, lineup: &Lineup, temperature: f32) -> Editorial {
|
||||
if lineup.picks.is_empty() {
|
||||
return fallback_editorial(lineup);
|
||||
}
|
||||
|
||||
let mut summaries = summarize_all(llm, lineup, temperature).await;
|
||||
let missing: Vec<&Pick> = lineup
|
||||
.picks
|
||||
.iter()
|
||||
.filter(|p| !summaries.contains_key(&p.article.id))
|
||||
.collect();
|
||||
if !missing.is_empty() {
|
||||
tracing::warn!(
|
||||
count = missing.len(),
|
||||
"using feed excerpts as summaries for articles the model did not cover"
|
||||
);
|
||||
for pick in missing {
|
||||
summaries.insert(pick.article.id, excerpt_summary(pick));
|
||||
}
|
||||
}
|
||||
|
||||
let (front_page_html, section_intros) =
|
||||
match front_page(llm, lineup, &summaries, temperature).await {
|
||||
Ok(response) => (
|
||||
text_to_paragraphs(&response.from_the_editor),
|
||||
response.section_intros,
|
||||
),
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e,
|
||||
"front-page generation failed; using the plain front page");
|
||||
(fallback_front_page_html(lineup), BTreeMap::new())
|
||||
}
|
||||
};
|
||||
|
||||
Editorial {
|
||||
front_page_html,
|
||||
section_intros,
|
||||
summaries,
|
||||
}
|
||||
}
|
||||
|
||||
/// Escape-and-wrap helper for callers rendering a summary straight into XHTML.
|
||||
pub fn summary_to_html(summary: &str) -> String {
|
||||
format!("<p>{}</p>", escape_html(summary.trim()))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::config::DeepseekConfig;
|
||||
use crate::curate::llm::{MockBackend, UsageMeter};
|
||||
use crate::curate::prefilter::tests::article;
|
||||
use crate::types::TokenUsage;
|
||||
use std::sync::Arc;
|
||||
|
||||
const FRONT_PAGE_FIXTURE: &str = include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/tests/fixtures/deepseek_front_page.json"
|
||||
));
|
||||
|
||||
fn pick(id: i64, title: &str, section: &str, is_lead: bool) -> Pick {
|
||||
let mut a = article(id, title, 900);
|
||||
a.content_html = format!("<p>{title} opens with a specific, concrete claim.</p>");
|
||||
Pick {
|
||||
article: a,
|
||||
section: section.into(),
|
||||
position: 1,
|
||||
is_lead,
|
||||
summary: None,
|
||||
llm: None,
|
||||
discussion: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn lineup() -> Lineup {
|
||||
Lineup {
|
||||
date: "2026-08-15".parse().expect("date"),
|
||||
picks: vec![
|
||||
pick(1, "Migrating 40TB off Postgres", "Top Stories", true),
|
||||
pick(2, "The MBTA slow-zone dataset", "Boston & Local", false),
|
||||
],
|
||||
section_order: vec!["Top Stories".into(), "Boston & Local".into()],
|
||||
}
|
||||
}
|
||||
|
||||
fn client(backend: Arc<MockBackend>, limit: f64) -> LlmClient {
|
||||
LlmClient::with_backend(
|
||||
"deepseek-v4-flash",
|
||||
"SYSTEM".into(),
|
||||
UsageMeter::new(&DeepseekConfig::default(), limit),
|
||||
backend,
|
||||
)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn summary_prompt_carries_headline_and_truncated_body() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push(
|
||||
r#"{"summary": "A team moves 40TB of relational data off Postgres and documents every rollback."}"#,
|
||||
TokenUsage::default(),
|
||||
);
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
let body = format!("<p>{}</p>", "word ".repeat(20_000));
|
||||
let summary = summarize_article(&llm, "Migrating 40TB", &body, 0.8)
|
||||
.await
|
||||
.expect("summary");
|
||||
assert!(summary.starts_with("A team moves 40TB"));
|
||||
|
||||
let prompt = &backend.prompts()[0].user;
|
||||
assert!(prompt.starts_with(SUMMARY_INSTRUCTIONS));
|
||||
assert!(prompt.contains("HEADLINE: Migrating 40TB"));
|
||||
assert!(prompt.contains("(truncated for length)"));
|
||||
// ~5k tokens ≈ 20k characters of body, not the full 100k.
|
||||
assert!(prompt.len() < 26_000, "prompt was {} bytes", prompt.len());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn front_page_parses_and_filters_unknown_sections() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push(FRONT_PAGE_FIXTURE, TokenUsage::default());
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
let lineup = lineup();
|
||||
let summaries = BTreeMap::from([
|
||||
(1, "A migration story with numbers.".to_string()),
|
||||
(2, "Transit data, charted.".to_string()),
|
||||
]);
|
||||
|
||||
let response = front_page(&llm, &lineup, &summaries, 0.8)
|
||||
.await
|
||||
.expect("front page");
|
||||
assert!(response.from_the_editor.split_whitespace().count() > 40);
|
||||
assert_eq!(response.section_intros.len(), 2);
|
||||
assert!(response.section_intros.contains_key("Top Stories"));
|
||||
assert!(
|
||||
!response.section_intros.contains_key("Niche Corner"),
|
||||
"intros for absent sections are dropped"
|
||||
);
|
||||
|
||||
let prompt = &backend.prompts()[0].user;
|
||||
assert!(prompt.starts_with(FRONT_PAGE_INSTRUCTIONS));
|
||||
assert!(prompt.contains("## Top Stories"));
|
||||
assert!(prompt.contains("[LEAD STORY]"));
|
||||
assert!(prompt.contains("abstract: A migration story with numbers."));
|
||||
assert!(prompt.contains("2026-08-15"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn full_stage_c_produces_summaries_intros_and_front_page() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push(r#"{"summary": "First abstract."}"#, TokenUsage::default());
|
||||
backend.push(r#"{"summary": "Second abstract."}"#, TokenUsage::default());
|
||||
backend.push(FRONT_PAGE_FIXTURE, TokenUsage::default());
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
|
||||
let editorial = run(&llm, &lineup(), 0.8).await;
|
||||
assert_eq!(
|
||||
backend.calls(),
|
||||
3,
|
||||
"one call per article plus the front page"
|
||||
);
|
||||
assert_eq!(editorial.summaries.len(), 2);
|
||||
assert_eq!(editorial.summaries[&1], "First abstract.");
|
||||
assert!(editorial.front_page_html.starts_with("<p>"));
|
||||
assert!(editorial.front_page_html.contains("</p>"));
|
||||
assert!(!editorial.front_page_html.contains("<script"));
|
||||
assert_eq!(editorial.section_intros.len(), 2);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn budget_exhaustion_degrades_to_excerpts() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
// The first summary alone blows a $0.05 ceiling.
|
||||
backend.push(
|
||||
r#"{"summary": "The one summary we could afford."}"#,
|
||||
TokenUsage {
|
||||
input_tokens: 1_000_000,
|
||||
cached_tokens: 0,
|
||||
output_tokens: 0,
|
||||
},
|
||||
);
|
||||
let llm = client(Arc::clone(&backend), 0.05);
|
||||
|
||||
let editorial = run(&llm, &lineup(), 0.8).await;
|
||||
assert_eq!(backend.calls(), 1, "no further calls after the ceiling");
|
||||
assert!(llm.meter.budget_exceeded());
|
||||
assert_eq!(
|
||||
editorial.summaries.len(),
|
||||
2,
|
||||
"every pick still has a summary"
|
||||
);
|
||||
assert_eq!(editorial.summaries[&1], "The one summary we could afford.");
|
||||
assert!(
|
||||
editorial.summaries[&2].contains("opens with a specific"),
|
||||
"second summary fell back to the excerpt: {}",
|
||||
editorial.summaries[&2]
|
||||
);
|
||||
// The front page degraded to the plain version.
|
||||
assert!(editorial.front_page_html.contains("2 articles"));
|
||||
assert!(editorial.section_intros.is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_failed_summary_call_is_not_fatal() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push_error("400 bad request");
|
||||
backend.push(r#"{"summary": "Second abstract."}"#, TokenUsage::default());
|
||||
backend.push_error("500 front page exploded");
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
|
||||
let editorial = run(&llm, &lineup(), 0.8).await;
|
||||
assert_eq!(editorial.summaries.len(), 2);
|
||||
assert!(editorial.summaries[&1].contains("opens with a specific"));
|
||||
assert_eq!(editorial.summaries[&2], "Second abstract.");
|
||||
assert!(editorial.front_page_html.contains("Leading today"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fallback_editorial_covers_every_pick() {
|
||||
let lineup = lineup();
|
||||
let editorial = fallback_editorial(&lineup);
|
||||
assert_eq!(editorial.summaries.len(), lineup.picks.len());
|
||||
assert!(editorial.section_intros.is_empty());
|
||||
assert!(editorial.front_page_html.contains("2 articles"));
|
||||
assert!(
|
||||
editorial
|
||||
.front_page_html
|
||||
.contains("Top Stories, Boston & Local")
|
||||
);
|
||||
assert!(editorial.front_page_html.starts_with("<p>"));
|
||||
|
||||
// An empty lineup is still a valid editorial.
|
||||
let empty = Lineup {
|
||||
date: "2026-08-15".parse().expect("date"),
|
||||
picks: vec![],
|
||||
section_order: vec![],
|
||||
};
|
||||
let editorial = fallback_editorial(&empty);
|
||||
assert!(editorial.summaries.is_empty());
|
||||
assert!(editorial.front_page_html.contains("0 articles"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn excerpt_summary_handles_empty_bodies() {
|
||||
let mut p = pick(9, "No body here", "Top Stories", false);
|
||||
p.article.content_html = String::new();
|
||||
assert!(excerpt_summary(&p).contains("No preview text"));
|
||||
assert_eq!(summary_to_html("a <b> c"), "<p>a <b> c</p>");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,708 @@
|
||||
//! DeepSeek client and token/cost accounting (spec §3.6).
|
||||
//!
|
||||
//! The OpenAI-compatible chat-completions endpoint at `https://api.deepseek.com/v1`.
|
||||
//! DeepSeek prefix-caches automatically, so the (identical, long) taste-profile
|
||||
//! system prompt must come first in every request: cached input is $0.0028/M vs
|
||||
//! $0.14/M.
|
||||
//!
|
||||
//! **Why not `async-openai`** (spec §2 crate table): the published crate exposes
|
||||
//! neither `Client` nor `types::chat` under any feature combination we could get
|
||||
//! to build here, and it would drag in a second HTTP stack besides the shared
|
||||
//! `reqwest` client (notes §4). [`DeepseekBackend`] therefore speaks the same
|
||||
//! OpenAI-compatible wire protocol directly — about 80 lines, no new dependency,
|
||||
//! and the request/response shapes are pinned by this module's tests. The
|
||||
//! dependency was dropped from `Cargo.toml`; swapping a vendor SDK back in later
|
||||
//! is a single [`ChatBackend`] impl and nothing else moves.
|
||||
//!
|
||||
//! Every call in the project goes through [`LlmClient`], which
|
||||
//!
|
||||
//! 1. always sends [`LlmClient::system_prompt`] as the **first** message, byte for
|
||||
//! byte identical across requests (that is what makes the prefix cache hit),
|
||||
//! 2. folds the response's token usage into a shared [`UsageMeter`], and
|
||||
//! 3. refuses further work once `max_daily_usd` has been spent (§3.6 guardrail).
|
||||
//!
|
||||
//! The network is reached through a [`ChatBackend`] so tests can inject canned
|
||||
//! responses ([`MockBackend`]) without touching the wire (notes §6).
|
||||
|
||||
use std::future::Future;
|
||||
use std::pin::Pin;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use serde::Deserialize;
|
||||
use serde_json::json;
|
||||
|
||||
use crate::config::DeepseekConfig;
|
||||
use crate::http::RetryPolicy;
|
||||
use crate::types::TokenUsage;
|
||||
|
||||
/// `response_format` value used for every structured call (§3.6).
|
||||
pub const JSON_OBJECT: &str = "json_object";
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum LlmError {
|
||||
#[error("deepseek api key is not configured (set DAILY_EPUB_DEEPSEEK__API_KEY)")]
|
||||
MissingApiKey,
|
||||
#[error("deepseek request failed: {0}")]
|
||||
Api(String),
|
||||
/// A 5xx/429/network failure: worth retrying (crate table "retry").
|
||||
#[error("deepseek request failed (transient): {0}")]
|
||||
Transient(String),
|
||||
#[error("deepseek returned an empty completion")]
|
||||
EmptyResponse,
|
||||
#[error("deepseek returned unparseable JSON: {0}")]
|
||||
Json(#[from] serde_json::Error),
|
||||
/// The `max_daily_usd` ceiling was reached: callers must skip remaining
|
||||
/// editorial calls and fall back to feed excerpts, loudly (§3.6).
|
||||
#[error("daily cost ceiling of ${limit:.2} reached (spent ${spent:.4})")]
|
||||
BudgetExceeded { spent: f64, limit: f64 },
|
||||
}
|
||||
|
||||
impl LlmError {
|
||||
/// True for failures the [`RetryPolicy`] should retry.
|
||||
pub fn is_transient(&self) -> bool {
|
||||
matches!(self, LlmError::Transient(_))
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Usage metering (§3.6 cost guardrail, notes §5)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Shared token/cost accumulator enforcing `max_daily_usd` (notes §5).
|
||||
///
|
||||
/// Cloning shares the counters: one meter per run, cloned into every stage.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct UsageMeter {
|
||||
inner: Arc<Mutex<TokenUsage>>,
|
||||
/// Sticky: once the ceiling is crossed the run stays degraded (§3.6).
|
||||
exceeded: Arc<AtomicBool>,
|
||||
limit_usd: f64,
|
||||
price_input: f64,
|
||||
price_cached: f64,
|
||||
price_output: f64,
|
||||
}
|
||||
|
||||
impl UsageMeter {
|
||||
pub fn new(cfg: &DeepseekConfig, limit_usd: f64) -> Self {
|
||||
Self {
|
||||
inner: Arc::new(Mutex::new(TokenUsage::default())),
|
||||
exceeded: Arc::new(AtomicBool::new(false)),
|
||||
limit_usd,
|
||||
price_input: cfg.price_input_per_mtok,
|
||||
price_cached: cfg.price_cached_input_per_mtok,
|
||||
price_output: cfg.price_output_per_mtok,
|
||||
}
|
||||
}
|
||||
|
||||
/// Seed the meter with spend already recorded for the day (§3.6): the
|
||||
/// guardrail is a *daily* ceiling, not a per-run one.
|
||||
pub fn preload_cost(&self, spent_usd: f64) {
|
||||
if spent_usd > 0.0 && self.limit_usd > 0.0 && spent_usd >= self.limit_usd {
|
||||
self.trip("prior spend for today already exceeds the ceiling");
|
||||
}
|
||||
}
|
||||
|
||||
/// Fold one response's usage in and return the running total.
|
||||
pub fn record(&self, usage: TokenUsage) -> TokenUsage {
|
||||
let total = match self.inner.lock() {
|
||||
Ok(mut guard) => {
|
||||
guard.add(usage);
|
||||
*guard
|
||||
}
|
||||
// A poisoned mutex must not abort a run: accounting is advisory.
|
||||
Err(poisoned) => {
|
||||
let mut guard = poisoned.into_inner();
|
||||
guard.add(usage);
|
||||
*guard
|
||||
}
|
||||
};
|
||||
let cost = self.cost_of(total);
|
||||
tracing::debug!(
|
||||
input = usage.input_tokens,
|
||||
cached = usage.cached_tokens,
|
||||
output = usage.output_tokens,
|
||||
total_cost_usd = cost,
|
||||
"recorded llm usage"
|
||||
);
|
||||
if self.limit_usd > 0.0 && cost > self.limit_usd && !self.exceeded.load(Ordering::SeqCst) {
|
||||
self.trip("token spend crossed the ceiling");
|
||||
}
|
||||
total
|
||||
}
|
||||
|
||||
fn trip(&self, why: &str) {
|
||||
self.exceeded.store(true, Ordering::SeqCst);
|
||||
tracing::error!(
|
||||
spent_usd = self.cost_usd(),
|
||||
limit_usd = self.limit_usd,
|
||||
"LLM budget exceeded ({why}): remaining editorial calls will be skipped \
|
||||
and feed excerpts used instead"
|
||||
);
|
||||
}
|
||||
|
||||
pub fn total(&self) -> TokenUsage {
|
||||
match self.inner.lock() {
|
||||
Ok(guard) => *guard,
|
||||
Err(poisoned) => *poisoned.into_inner(),
|
||||
}
|
||||
}
|
||||
|
||||
fn cost_of(&self, usage: TokenUsage) -> f64 {
|
||||
usage.cost_usd(self.price_input, self.price_cached, self.price_output)
|
||||
}
|
||||
|
||||
pub fn cost_usd(&self) -> f64 {
|
||||
self.cost_of(self.total())
|
||||
}
|
||||
|
||||
pub fn limit_usd(&self) -> f64 {
|
||||
self.limit_usd
|
||||
}
|
||||
|
||||
/// True once the ceiling has been crossed — editorial stages check this and
|
||||
/// silently degrade to excerpts (§3.6).
|
||||
pub fn budget_exceeded(&self) -> bool {
|
||||
self.exceeded.load(Ordering::SeqCst)
|
||||
}
|
||||
|
||||
/// `Err(BudgetExceeded)` once the run has spent more than `max_daily_usd` (§3.6).
|
||||
pub fn check_budget(&self) -> Result<(), LlmError> {
|
||||
if self.budget_exceeded() {
|
||||
return Err(LlmError::BudgetExceeded {
|
||||
spent: self.cost_usd(),
|
||||
limit: self.limit_usd,
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Backend abstraction (notes §6: no network in tests)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One chat completion request. The system prompt is an [`Arc`] so that the
|
||||
/// identical bytes are reused for every call (DeepSeek prefix caching, §3.6).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ChatRequest {
|
||||
pub model: String,
|
||||
pub system: Arc<String>,
|
||||
pub user: String,
|
||||
pub temperature: f32,
|
||||
/// Ask for `response_format: {"type": "json_object"}` (§3.6).
|
||||
pub json: bool,
|
||||
}
|
||||
|
||||
/// One chat completion response, reduced to what the pipeline needs.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct ChatCompletion {
|
||||
pub content: String,
|
||||
pub usage: TokenUsage,
|
||||
}
|
||||
|
||||
type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
|
||||
|
||||
/// The seam between [`LlmClient`] and the network (notes §6).
|
||||
pub trait ChatBackend: std::fmt::Debug + Send + Sync {
|
||||
fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result<ChatCompletion, LlmError>>;
|
||||
}
|
||||
|
||||
/// LLM calls are slow; the shared 10s HTTP timeout would kill them (notes §4).
|
||||
const LLM_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(180);
|
||||
|
||||
/// The real thing: the OpenAI-compatible endpoint at `deepseek.base_url` (§3.6).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DeepseekBackend {
|
||||
http: reqwest::Client,
|
||||
endpoint: String,
|
||||
api_key: String,
|
||||
}
|
||||
|
||||
impl DeepseekBackend {
|
||||
pub fn new(cfg: &DeepseekConfig) -> Result<Self, LlmError> {
|
||||
let api_key = cfg
|
||||
.api_key
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|k| !k.is_empty())
|
||||
.ok_or(LlmError::MissingApiKey)?
|
||||
.to_string();
|
||||
let http = crate::http::build_client(LLM_TIMEOUT)
|
||||
.map_err(|e| LlmError::Api(format!("building the deepseek http client: {e}")))?;
|
||||
Ok(Self {
|
||||
http,
|
||||
endpoint: format!("{}/chat/completions", cfg.base_url.trim_end_matches('/')),
|
||||
api_key,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl ChatBackend for DeepseekBackend {
|
||||
fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result<ChatCompletion, LlmError>> {
|
||||
Box::pin(async move {
|
||||
let mut body = json!({
|
||||
"model": req.model,
|
||||
"messages": [
|
||||
// FIRST and byte-identical across every request: prefix cache (§3.6).
|
||||
{"role": "system", "content": req.system.as_str()},
|
||||
{"role": "user", "content": req.user},
|
||||
],
|
||||
"temperature": req.temperature,
|
||||
"stream": false,
|
||||
});
|
||||
if req.json
|
||||
&& let Some(obj) = body.as_object_mut()
|
||||
{
|
||||
obj.insert("response_format".into(), json!({"type": JSON_OBJECT}));
|
||||
}
|
||||
|
||||
let response = self
|
||||
.http
|
||||
.post(&self.endpoint)
|
||||
.bearer_auth(&self.api_key)
|
||||
.json(&body)
|
||||
.send()
|
||||
.await
|
||||
.map_err(classify_reqwest_error)?;
|
||||
|
||||
let status = response.status();
|
||||
if !status.is_success() {
|
||||
let detail = response.text().await.unwrap_or_default();
|
||||
let detail = detail.chars().take(500).collect::<String>();
|
||||
let msg = format!("{status}: {detail}");
|
||||
return Err(if status.is_server_error() || status.as_u16() == 429 {
|
||||
LlmError::Transient(msg)
|
||||
} else {
|
||||
LlmError::Api(msg)
|
||||
});
|
||||
}
|
||||
|
||||
let parsed: ApiResponse = response.json().await.map_err(|e| {
|
||||
LlmError::Api(format!("decoding the deepseek chat completion: {e}"))
|
||||
})?;
|
||||
let content = parsed
|
||||
.choices
|
||||
.into_iter()
|
||||
.next()
|
||||
.and_then(|c| c.message.content)
|
||||
.filter(|c| !c.trim().is_empty())
|
||||
.ok_or(LlmError::EmptyResponse)?;
|
||||
let usage = parsed.usage.map(usage_from_api).unwrap_or_default();
|
||||
Ok(ChatCompletion { content, usage })
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// The slice of the chat-completions response we consume.
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct ApiResponse {
|
||||
#[serde(default)]
|
||||
choices: Vec<ApiChoice>,
|
||||
#[serde(default)]
|
||||
usage: Option<ApiUsage>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct ApiChoice {
|
||||
message: ApiMessage,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct ApiMessage {
|
||||
#[serde(default)]
|
||||
content: Option<String>,
|
||||
}
|
||||
|
||||
/// DeepSeek reports cache hits both OpenAI-style (`prompt_tokens_details`) and
|
||||
/// natively (`prompt_cache_hit_tokens`); we accept either (§3.6 pricing).
|
||||
#[derive(Debug, Default, Deserialize)]
|
||||
struct ApiUsage {
|
||||
#[serde(default)]
|
||||
prompt_tokens: i64,
|
||||
#[serde(default)]
|
||||
completion_tokens: i64,
|
||||
#[serde(default)]
|
||||
prompt_cache_hit_tokens: Option<i64>,
|
||||
#[serde(default)]
|
||||
prompt_tokens_details: Option<ApiPromptTokensDetails>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Deserialize)]
|
||||
struct ApiPromptTokensDetails {
|
||||
#[serde(default)]
|
||||
cached_tokens: Option<i64>,
|
||||
}
|
||||
|
||||
/// Split `prompt_tokens` into cache-miss and cache-hit halves (§3.6 pricing).
|
||||
fn usage_from_api(u: ApiUsage) -> TokenUsage {
|
||||
let cached = u
|
||||
.prompt_tokens_details
|
||||
.as_ref()
|
||||
.and_then(|d| d.cached_tokens)
|
||||
.or(u.prompt_cache_hit_tokens)
|
||||
.unwrap_or(0)
|
||||
.max(0);
|
||||
let prompt = u.prompt_tokens.max(0);
|
||||
let cached = cached.min(prompt);
|
||||
TokenUsage {
|
||||
input_tokens: prompt - cached,
|
||||
cached_tokens: cached,
|
||||
output_tokens: u.completion_tokens.max(0),
|
||||
}
|
||||
}
|
||||
|
||||
fn classify_reqwest_error(err: reqwest::Error) -> LlmError {
|
||||
if crate::http::is_retryable(&err) {
|
||||
LlmError::Transient(err.to_string())
|
||||
} else {
|
||||
LlmError::Api(err.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Client
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Every LLM call in the project goes through this client (notes §5).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct LlmClient {
|
||||
/// The taste profile, sent as the first (cacheable) system message (§3.6).
|
||||
pub system_prompt: Arc<String>,
|
||||
pub model: String,
|
||||
pub meter: UsageMeter,
|
||||
backend: Arc<dyn ChatBackend>,
|
||||
retry: RetryPolicy,
|
||||
}
|
||||
|
||||
impl LlmClient {
|
||||
/// Build against the configured base URL; fails without an API key.
|
||||
pub fn new(
|
||||
cfg: &DeepseekConfig,
|
||||
system_prompt: String,
|
||||
meter: UsageMeter,
|
||||
) -> Result<Self, LlmError> {
|
||||
let backend = DeepseekBackend::new(cfg)?;
|
||||
tracing::debug!(
|
||||
base_url = %cfg.base_url,
|
||||
model = %cfg.model,
|
||||
system_prompt_chars = system_prompt.len(),
|
||||
"deepseek client ready"
|
||||
);
|
||||
Ok(Self::with_backend(
|
||||
&cfg.model,
|
||||
system_prompt,
|
||||
meter,
|
||||
Arc::new(backend),
|
||||
))
|
||||
}
|
||||
|
||||
/// Construct around an arbitrary backend — the seam used by tests (notes §6).
|
||||
pub fn with_backend(
|
||||
model: &str,
|
||||
system_prompt: String,
|
||||
meter: UsageMeter,
|
||||
backend: Arc<dyn ChatBackend>,
|
||||
) -> Self {
|
||||
Self {
|
||||
system_prompt: Arc::new(system_prompt),
|
||||
model: model.to_string(),
|
||||
meter,
|
||||
backend,
|
||||
retry: RetryPolicy::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Raw completion: budget check → retry loop → usage accounting.
|
||||
pub async fn complete(
|
||||
&self,
|
||||
user_prompt: &str,
|
||||
temperature: f32,
|
||||
json: bool,
|
||||
) -> Result<String, LlmError> {
|
||||
self.meter.check_budget()?;
|
||||
let req = ChatRequest {
|
||||
model: self.model.clone(),
|
||||
system: Arc::clone(&self.system_prompt),
|
||||
user: user_prompt.to_string(),
|
||||
temperature,
|
||||
json,
|
||||
};
|
||||
let completion = self
|
||||
.retry
|
||||
.run("deepseek chat completion", LlmError::is_transient, || {
|
||||
self.backend.complete(req.clone())
|
||||
})
|
||||
.await?;
|
||||
self.meter.record(completion.usage);
|
||||
Ok(completion.content)
|
||||
}
|
||||
|
||||
/// One chat completion returning parsed JSON of type `T`, with the system
|
||||
/// prompt first and `response_format: json_object` (§3.6).
|
||||
pub async fn complete_json<T: serde::de::DeserializeOwned>(
|
||||
&self,
|
||||
user_prompt: &str,
|
||||
temperature: f32,
|
||||
) -> Result<T, LlmError> {
|
||||
let raw = self.complete(user_prompt, temperature, true).await?;
|
||||
let cleaned = strip_code_fence(&raw);
|
||||
match serde_json::from_str::<T>(cleaned) {
|
||||
Ok(v) => Ok(v),
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
error = %e,
|
||||
preview = %cleaned.chars().take(400).collect::<String>(),
|
||||
"deepseek returned malformed JSON"
|
||||
);
|
||||
Err(LlmError::Json(e))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One plain-text completion (used for the front page / intros) (§3.6).
|
||||
pub async fn complete_text(
|
||||
&self,
|
||||
user_prompt: &str,
|
||||
temperature: f32,
|
||||
) -> Result<String, LlmError> {
|
||||
self.complete(user_prompt, temperature, false).await
|
||||
}
|
||||
}
|
||||
|
||||
/// Models occasionally wrap JSON in ```` ```json ```` fences despite `json_object`.
|
||||
pub fn strip_code_fence(raw: &str) -> &str {
|
||||
let trimmed = raw.trim();
|
||||
let Some(rest) = trimmed.strip_prefix("```") else {
|
||||
return trimmed;
|
||||
};
|
||||
let rest = rest.strip_prefix("json").unwrap_or(rest);
|
||||
rest.trim_start_matches(['\n', '\r'])
|
||||
.trim_end()
|
||||
.trim_end_matches("```")
|
||||
.trim()
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test backend
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Canned-response backend for tests: pops scripted replies in order (notes §6).
|
||||
#[derive(Debug, Default)]
|
||||
pub struct MockBackend {
|
||||
scripted: Mutex<std::collections::VecDeque<Result<ChatCompletion, String>>>,
|
||||
/// Every prompt the code under test sent, in order.
|
||||
pub seen: Mutex<Vec<ChatRequest>>,
|
||||
}
|
||||
|
||||
impl MockBackend {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Queue a successful reply carrying `usage` tokens.
|
||||
pub fn push(&self, content: impl Into<String>, usage: TokenUsage) {
|
||||
if let Ok(mut q) = self.scripted.lock() {
|
||||
q.push_back(Ok(ChatCompletion {
|
||||
content: content.into(),
|
||||
usage,
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
/// Queue a permanent (non-retryable) failure.
|
||||
pub fn push_error(&self, message: impl Into<String>) {
|
||||
if let Ok(mut q) = self.scripted.lock() {
|
||||
q.push_back(Err(message.into()));
|
||||
}
|
||||
}
|
||||
|
||||
pub fn calls(&self) -> usize {
|
||||
self.seen.lock().map(|s| s.len()).unwrap_or(0)
|
||||
}
|
||||
|
||||
pub fn prompts(&self) -> Vec<ChatRequest> {
|
||||
self.seen.lock().map(|s| s.clone()).unwrap_or_default()
|
||||
}
|
||||
}
|
||||
|
||||
impl ChatBackend for MockBackend {
|
||||
fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result<ChatCompletion, LlmError>> {
|
||||
Box::pin(async move {
|
||||
let next = self.scripted.lock().ok().and_then(|mut q| q.pop_front());
|
||||
if let Ok(mut seen) = self.seen.lock() {
|
||||
seen.push(req);
|
||||
}
|
||||
match next {
|
||||
Some(Ok(c)) => Ok(c),
|
||||
Some(Err(msg)) => Err(LlmError::Api(msg)),
|
||||
None => Err(LlmError::Api("mock backend ran out of responses".into())),
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn cfg() -> DeepseekConfig {
|
||||
DeepseekConfig::default()
|
||||
}
|
||||
|
||||
pub(crate) fn tokens(input: i64, cached: i64, output: i64) -> TokenUsage {
|
||||
TokenUsage {
|
||||
input_tokens: input,
|
||||
cached_tokens: cached,
|
||||
output_tokens: output,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn meter_accumulates_and_prices() {
|
||||
let meter = UsageMeter::new(&cfg(), 2.0);
|
||||
meter.record(tokens(1_000_000, 0, 0));
|
||||
meter.record(tokens(0, 1_000_000, 1_000_000));
|
||||
let total = meter.total();
|
||||
assert_eq!(total.input_tokens, 1_000_000);
|
||||
assert_eq!(total.cached_tokens, 1_000_000);
|
||||
assert_eq!(total.output_tokens, 1_000_000);
|
||||
assert!((meter.cost_usd() - 0.4228).abs() < 1e-9);
|
||||
assert!(!meter.budget_exceeded());
|
||||
assert!(meter.check_budget().is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn meter_trips_the_budget_flag_and_stays_tripped() {
|
||||
// Ceiling of $0.10; 1M cache-miss input tokens costs $0.14.
|
||||
let meter = UsageMeter::new(&cfg(), 0.10);
|
||||
meter.record(tokens(1_000_000, 0, 0));
|
||||
assert!(meter.budget_exceeded());
|
||||
assert!(matches!(
|
||||
meter.check_budget(),
|
||||
Err(LlmError::BudgetExceeded { .. })
|
||||
));
|
||||
// Cloned meters share the flag.
|
||||
assert!(meter.clone().budget_exceeded());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn preloaded_daily_spend_trips_the_flag() {
|
||||
let meter = UsageMeter::new(&cfg(), 1.0);
|
||||
meter.preload_cost(0.5);
|
||||
assert!(!meter.budget_exceeded());
|
||||
meter.preload_cost(1.5);
|
||||
assert!(meter.budget_exceeded());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_split_uses_prompt_token_details() {
|
||||
let u: ApiUsage = serde_json::from_str(
|
||||
r#"{"prompt_tokens": 1000, "completion_tokens": 120, "total_tokens": 1120,
|
||||
"prompt_tokens_details": {"cached_tokens": 800}}"#,
|
||||
)
|
||||
.expect("fixture usage");
|
||||
assert_eq!(usage_from_api(u), tokens(200, 800, 120));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_falls_back_to_deepseek_native_cache_fields() {
|
||||
let u: ApiUsage = serde_json::from_str(
|
||||
r#"{"prompt_tokens": 500, "completion_tokens": 40,
|
||||
"prompt_cache_hit_tokens": 448, "prompt_cache_miss_tokens": 52}"#,
|
||||
)
|
||||
.expect("fixture usage");
|
||||
assert_eq!(usage_from_api(u), tokens(52, 448, 40));
|
||||
// Missing usage is not an error, just zero.
|
||||
let empty: ApiUsage = serde_json::from_str("{}").expect("empty usage");
|
||||
assert_eq!(usage_from_api(empty), TokenUsage::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn code_fences_are_stripped() {
|
||||
assert_eq!(strip_code_fence("{\"a\":1}"), "{\"a\":1}");
|
||||
assert_eq!(strip_code_fence("```json\n{\"a\":1}\n```"), "{\"a\":1}");
|
||||
assert_eq!(strip_code_fence("```\n{\"a\":1}\n```"), "{\"a\":1}");
|
||||
}
|
||||
|
||||
fn client(backend: Arc<MockBackend>, limit: f64) -> LlmClient {
|
||||
LlmClient::with_backend(
|
||||
"deepseek-v4-flash",
|
||||
"SYSTEM PROMPT".into(),
|
||||
UsageMeter::new(&cfg(), limit),
|
||||
backend,
|
||||
)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn json_completion_records_usage_and_sends_system_prompt_first() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push(r#"{"value": 42}"#, tokens(10, 90, 5));
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
|
||||
#[derive(serde::Deserialize)]
|
||||
struct Out {
|
||||
value: i64,
|
||||
}
|
||||
let out: Out = llm
|
||||
.complete_json("score these", 0.3)
|
||||
.await
|
||||
.expect("mock completion");
|
||||
assert_eq!(out.value, 42);
|
||||
assert_eq!(llm.meter.total(), tokens(10, 90, 5));
|
||||
|
||||
let prompts = backend.prompts();
|
||||
assert_eq!(prompts.len(), 1);
|
||||
assert_eq!(prompts[0].system.as_str(), "SYSTEM PROMPT");
|
||||
assert!(prompts[0].json);
|
||||
assert_eq!(prompts[0].user, "score these");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn identical_system_prompt_bytes_across_calls() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push("{}", TokenUsage::default());
|
||||
backend.push("{}", TokenUsage::default());
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
let _: serde_json::Value = llm.complete_json("a", 0.3).await.expect("first");
|
||||
let _: serde_json::Value = llm.complete_json("b", 0.3).await.expect("second");
|
||||
let prompts = backend.prompts();
|
||||
assert_eq!(prompts[0].system.as_bytes(), prompts[1].system.as_bytes());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn calls_are_refused_once_the_budget_is_gone() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push("{}", tokens(1_000_000, 0, 0));
|
||||
let llm = client(Arc::clone(&backend), 0.01);
|
||||
let _: serde_json::Value = llm.complete_json("first", 0.3).await.expect("first call");
|
||||
let err = llm
|
||||
.complete_text("second", 0.3)
|
||||
.await
|
||||
.expect_err("budget must be enforced");
|
||||
assert!(matches!(err, LlmError::BudgetExceeded { .. }));
|
||||
// The refused call never reached the backend.
|
||||
assert_eq!(backend.calls(), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn malformed_json_surfaces_as_json_error() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push("not json at all", TokenUsage::default());
|
||||
let llm = client(backend, 2.0);
|
||||
let out: Result<serde_json::Value, _> = llm.complete_json("x", 0.3).await;
|
||||
assert!(matches!(out, Err(LlmError::Json(_))));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_api_key_is_reported() {
|
||||
let cfg = DeepseekConfig {
|
||||
api_key: Some(" ".into()),
|
||||
..DeepseekConfig::default()
|
||||
};
|
||||
assert!(matches!(
|
||||
DeepseekBackend::new(&cfg),
|
||||
Err(LlmError::MissingApiKey)
|
||||
));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,305 @@
|
||||
//! Curation pipeline: pre-filter → LLM scoring → selection → editorial (spec §3.5, §3.6).
|
||||
//!
|
||||
//! ```text
|
||||
//! ~400 articles ─prefilter─▶ ~120 candidates ─stage A─▶ scored ─stage B─▶ lineup ─stage C─▶ editorial
|
||||
//! ```
|
||||
//!
|
||||
//! [`Curator`] is the thin orchestration layer the `generate` pipeline calls; the
|
||||
//! interesting logic lives in the stage modules. Every stage is safe to run with
|
||||
//! `llm == None` (`--skip-llm`): the prefilter order stands in for selection and
|
||||
//! feed excerpts stand in for summaries (notes §6).
|
||||
|
||||
pub mod editorial;
|
||||
pub mod llm;
|
||||
pub mod prefilter;
|
||||
pub mod profile;
|
||||
pub mod score;
|
||||
pub mod select;
|
||||
|
||||
use jiff::civil::Date;
|
||||
|
||||
use crate::config::Config;
|
||||
use crate::db::Db;
|
||||
use crate::types::{Article, Editorial, Lineup, ScoredArticle};
|
||||
|
||||
/// Runs the three curation stages against one day's articles (§3.5, §3.6).
|
||||
pub struct Curator {
|
||||
pub config: Config,
|
||||
pub db: Db,
|
||||
pub llm: Option<llm::LlmClient>,
|
||||
}
|
||||
|
||||
impl Curator {
|
||||
/// `llm == None` corresponds to `--skip-llm`: prefilter order is used for
|
||||
/// selection and feed excerpts stand in for summaries (notes §6).
|
||||
pub fn new(config: Config, db: Db, llm: Option<llm::LlmClient>) -> Self {
|
||||
Self { config, db, llm }
|
||||
}
|
||||
|
||||
/// Heuristic pre-filter: 300–500 articles → `prefilter_keep` (§3.5).
|
||||
///
|
||||
/// Also persists each candidate's `prefilter_score` for the day so that a
|
||||
/// re-run of the same date is idempotent (notes §12).
|
||||
pub async fn prefilter(
|
||||
&self,
|
||||
articles: Vec<Article>,
|
||||
date: Date,
|
||||
) -> anyhow::Result<Vec<ScoredArticle>> {
|
||||
let span = tracing::info_span!("prefilter", articles = articles.len());
|
||||
let _guard = span.enter();
|
||||
|
||||
let ctx = prefilter::PrefilterContext::load(&self.db, date).await?;
|
||||
let candidates = prefilter::run(articles, &ctx, &self.config);
|
||||
for candidate in &candidates {
|
||||
if candidate.article.id == 0 {
|
||||
continue; // not persisted yet (dry run over synthetic articles)
|
||||
}
|
||||
if let Err(e) = self
|
||||
.db
|
||||
.upsert_score(
|
||||
candidate.article.id,
|
||||
date,
|
||||
Some(candidate.prefilter_score),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(article_id = candidate.article.id, error = %e,
|
||||
"could not persist the prefilter score");
|
||||
}
|
||||
}
|
||||
Ok(candidates)
|
||||
}
|
||||
|
||||
/// Stage A: batched LLM scoring of the surviving candidates (§3.6).
|
||||
///
|
||||
/// A no-op under `--skip-llm`. Scores are persisted per `(article, date)`.
|
||||
pub async fn score(&self, candidates: &mut [ScoredArticle], date: Date) -> anyhow::Result<()> {
|
||||
let Some(llm) = self.llm.as_ref() else {
|
||||
tracing::info!("--skip-llm: stage A scoring skipped");
|
||||
return Ok(());
|
||||
};
|
||||
let span = tracing::info_span!("llm_score", candidates = candidates.len());
|
||||
let _guard = span.enter();
|
||||
|
||||
let scored = score::score_all(
|
||||
llm,
|
||||
candidates,
|
||||
self.config.deepseek.score_batch_size,
|
||||
&self.config.curation.sections,
|
||||
self.config.deepseek.score_temperature,
|
||||
)
|
||||
.await?;
|
||||
tracing::info!(scored, total = candidates.len(), "stage A complete");
|
||||
|
||||
for candidate in candidates.iter() {
|
||||
if candidate.article.id == 0 {
|
||||
continue;
|
||||
}
|
||||
if let Some(llm_score) = candidate.llm.as_ref()
|
||||
&& let Err(e) = self
|
||||
.db
|
||||
.upsert_score(candidate.article.id, date, None, Some(llm_score))
|
||||
.await
|
||||
{
|
||||
tracing::warn!(article_id = candidate.article.id, error = %e,
|
||||
"could not persist the llm score");
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Stage B: single-call lineup selection into sections (§3.6).
|
||||
pub async fn select(
|
||||
&self,
|
||||
candidates: Vec<ScoredArticle>,
|
||||
date: Date,
|
||||
) -> anyhow::Result<Lineup> {
|
||||
let sections = &self.config.curation.sections;
|
||||
let target = self.config.target_article_count;
|
||||
let Some(llm) = self.llm.as_ref() else {
|
||||
tracing::info!("--skip-llm: selecting by prefilter order");
|
||||
return Ok(select::select_without_llm(
|
||||
candidates, sections, target, date,
|
||||
));
|
||||
};
|
||||
let span = tracing::info_span!("llm_select", candidates = candidates.len());
|
||||
let _guard = span.enter();
|
||||
|
||||
match select::select(llm, candidates.clone(), sections, target, date).await {
|
||||
Ok(lineup) => Ok(lineup),
|
||||
Err(e) => {
|
||||
tracing::error!(error = %e,
|
||||
"stage B selection failed; falling back to prefilter order");
|
||||
Ok(select::select_without_llm(
|
||||
candidates, sections, target, date,
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Stage C: per-article summaries, section intros and the front page (§3.6).
|
||||
///
|
||||
/// Never fails the run: a budget trip or an API error degrades to excerpts.
|
||||
pub async fn editorial(&self, lineup: &Lineup) -> anyhow::Result<Editorial> {
|
||||
let Some(llm) = self.llm.as_ref() else {
|
||||
tracing::info!("--skip-llm: using feed excerpts as summaries");
|
||||
return Ok(editorial::fallback_editorial(lineup));
|
||||
};
|
||||
let span = tracing::info_span!("llm_editorial", picks = lineup.picks.len());
|
||||
let _guard = span.enter();
|
||||
Ok(editorial::run(llm, lineup, self.config.deepseek.editorial_temperature).await)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Small text helpers shared by the prompt builders
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Crude token estimate: DeepSeek averages ~4 characters per token for English
|
||||
/// prose. Only used to size prompt budgets (§3.6 stage C).
|
||||
pub fn approx_tokens(text: &str) -> usize {
|
||||
text.len().div_ceil(4)
|
||||
}
|
||||
|
||||
/// Strip markup and collapse whitespace, so article bodies can go into prompts
|
||||
/// as plain text (cheaper and less confusing for the model than raw HTML).
|
||||
pub fn html_to_text(html: &str) -> String {
|
||||
/// Does `tail` open the named element, i.e. `<name` or `</name`?
|
||||
fn opens(tail: &str, name: &str) -> bool {
|
||||
let bytes = tail.as_bytes();
|
||||
bytes.len() > name.len() && bytes[1..=name.len()].eq_ignore_ascii_case(name.as_bytes())
|
||||
}
|
||||
|
||||
let mut out = String::with_capacity(html.len());
|
||||
let mut rest = html;
|
||||
while let Some(ch) = rest.chars().next() {
|
||||
if ch != '<' {
|
||||
out.push(ch);
|
||||
rest = &rest[ch.len_utf8()..];
|
||||
continue;
|
||||
}
|
||||
// Drop <script>/<style> bodies wholesale rather than reading them aloud.
|
||||
for (name, close) in [("script", "</script"), ("style", "</style")] {
|
||||
if opens(rest, name) {
|
||||
rest = match rest[1..].find(close) {
|
||||
Some(idx) => &rest[1 + idx + close.len()..],
|
||||
None => "",
|
||||
};
|
||||
break;
|
||||
}
|
||||
}
|
||||
// A tag becomes a word boundary.
|
||||
rest = match rest.find('>') {
|
||||
Some(idx) => &rest[idx + 1..],
|
||||
None => "",
|
||||
};
|
||||
out.push(' ');
|
||||
}
|
||||
|
||||
let decoded = out
|
||||
.replace(" ", " ")
|
||||
.replace("&", "&")
|
||||
.replace("<", "<")
|
||||
.replace(">", ">")
|
||||
.replace(""", "\"")
|
||||
.replace("'", "'")
|
||||
.replace("’", "'")
|
||||
.replace("—", "—");
|
||||
decoded.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||
}
|
||||
|
||||
/// First `max_words` words of `text`, with an ellipsis when truncated.
|
||||
pub fn truncate_words(text: &str, max_words: usize) -> String {
|
||||
let mut words = text.split_whitespace();
|
||||
let head: Vec<&str> = words.by_ref().take(max_words).collect();
|
||||
let mut out = head.join(" ");
|
||||
if words.next().is_some() {
|
||||
out.push('…');
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Truncate to roughly `max_tokens` tokens on a word boundary (§3.6 stage C).
|
||||
pub fn truncate_tokens(text: &str, max_tokens: usize) -> String {
|
||||
let max_chars = max_tokens.saturating_mul(4);
|
||||
if text.len() <= max_chars {
|
||||
return text.to_string();
|
||||
}
|
||||
let mut cut = max_chars.min(text.len());
|
||||
while cut > 0 && !text.is_char_boundary(cut) {
|
||||
cut -= 1;
|
||||
}
|
||||
let slice = &text[..cut];
|
||||
let slice = slice
|
||||
.rsplit_once(' ')
|
||||
.map(|(head, _)| head)
|
||||
.unwrap_or(slice);
|
||||
format!("{slice}…")
|
||||
}
|
||||
|
||||
/// Minimal XHTML escaping for text we drop into generated markup (§3.10).
|
||||
pub fn escape_html(text: &str) -> String {
|
||||
let mut out = String::with_capacity(text.len());
|
||||
for ch in text.chars() {
|
||||
match ch {
|
||||
'&' => out.push_str("&"),
|
||||
'<' => out.push_str("<"),
|
||||
'>' => out.push_str(">"),
|
||||
'"' => out.push_str("""),
|
||||
'\'' => out.push_str("'"),
|
||||
c => out.push(c),
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Render plain text (possibly with blank-line paragraphs) as XHTML paragraphs.
|
||||
pub fn text_to_paragraphs(text: &str) -> String {
|
||||
text.split("\n\n")
|
||||
.map(str::trim)
|
||||
.filter(|p| !p.is_empty())
|
||||
.map(|p| format!("<p>{}</p>", escape_html(&p.replace('\n', " "))))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn html_becomes_readable_text() {
|
||||
let html = "<h1>Title</h1><p>First & best.</p><script>alert('x')</script>\
|
||||
<p>Second<br/>line</p><style>p{color:red}</style>";
|
||||
assert_eq!(html_to_text(html), "Title First & best. Second line");
|
||||
assert_eq!(html_to_text(""), "");
|
||||
assert_eq!(html_to_text("no markup at all"), "no markup at all");
|
||||
// Unicode survives byte-wise walking.
|
||||
assert_eq!(html_to_text("<p>café — naïve</p>"), "café — naïve");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn word_and_token_truncation() {
|
||||
assert_eq!(truncate_words("one two three", 5), "one two three");
|
||||
assert_eq!(truncate_words("one two three", 2), "one two…");
|
||||
let long = "word ".repeat(1000);
|
||||
// 10 tokens ≈ 40 characters, cut back to a word boundary, plus the ellipsis.
|
||||
let cut = truncate_tokens(&long, 10);
|
||||
assert!(cut.len() <= 43, "{}", cut.len());
|
||||
assert!(cut.split_whitespace().count() <= 10);
|
||||
assert!(cut.ends_with('…'));
|
||||
assert_eq!(truncate_tokens("short", 10), "short");
|
||||
assert!(approx_tokens("abcd") <= 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn escaping_and_paragraphs() {
|
||||
assert_eq!(escape_html("a<b>&'\""), "a<b>&'"");
|
||||
assert_eq!(
|
||||
text_to_paragraphs("One\nline.\n\nTwo <b>."),
|
||||
"<p>One line.</p>\n<p>Two <b>.</p>"
|
||||
);
|
||||
assert_eq!(text_to_paragraphs(" "), "");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,665 @@
|
||||
//! Heuristic pre-filter: 300–500 articles → ~120 candidates (spec §3.5).
|
||||
//!
|
||||
//! Pure Rust and free: this is what keeps LLM cost flat as feed volume grows.
|
||||
//!
|
||||
//! The 0–100 score is a sum of bounded components so that no single signal can
|
||||
//! dominate, and every component is monotonic in its input:
|
||||
//!
|
||||
//! | component | range | source |
|
||||
//! |---|---|---|
|
||||
//! | long-form word count | 0 … +35 | §3.5 "0 pts <300 words, max at ~2500+" |
|
||||
//! | social proof | 0 … +25 | §3.4 composite, log-scaled again |
|
||||
//! | came via Scour | +8 | §3.5 (already matched a stated interest) |
|
||||
//! | came via HN frontpage | +8 | §3.5 |
|
||||
//! | carried by several feeds | 0 … +8 | §3.2 (multi-source *is* social proof) |
|
||||
//! | feed prior | −12 … +12 | §3.9 beta-smoothed upvote rate, neutral at 0.5 |
|
||||
//! | excerpt only | −20 | §3.5 (penalized, never banned — §7) |
|
||||
//! | roundup/release-notes title | −15 | §3.5 |
|
||||
//! | blocked domain | excluded | §3.5 |
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
|
||||
use crate::config::{Config, CurationConfig};
|
||||
use crate::types::{Article, ArticleId, FeedId, FeedPrior, ScoredArticle, SourceKind};
|
||||
|
||||
/// Title patterns that mark low-effort posts: link roundups, release notes,
|
||||
/// sponsor posts (§3.5).
|
||||
pub const PENALTY_TITLE_PATTERNS: &[&str] = &[
|
||||
"link roundup",
|
||||
"links for",
|
||||
"weekly digest",
|
||||
"release notes",
|
||||
"changelog",
|
||||
"sponsored",
|
||||
"this week in",
|
||||
"linkdump",
|
||||
"link dump",
|
||||
"weekly roundup",
|
||||
"roundup:",
|
||||
"in case you missed it",
|
||||
"what we're reading",
|
||||
"sponsor post",
|
||||
"now available",
|
||||
"is now generally available",
|
||||
"release candidate",
|
||||
"patch notes",
|
||||
"job board",
|
||||
"who's hiring",
|
||||
"newsletter #",
|
||||
"digest #",
|
||||
];
|
||||
|
||||
/// Word count at which the long-form bonus saturates (§3.5).
|
||||
pub const LONGFORM_SATURATION_WORDS: i64 = 2500;
|
||||
/// Below this word count the long-form bonus is zero (§3.5).
|
||||
pub const LONGFORM_FLOOR_WORDS: i64 = 300;
|
||||
/// Articles the LLM scored below this within the last week are not re-scored (§3.5).
|
||||
pub const STALE_LOW_SCORE: f64 = 3.0;
|
||||
/// Lookback for the "don't re-score churn" rule (§3.5).
|
||||
pub const STALE_LOOKBACK_DAYS: i64 = 7;
|
||||
|
||||
/// Maximum contribution of each scoring component (§3.5).
|
||||
pub const MAX_LONGFORM_POINTS: f64 = 35.0;
|
||||
pub const MAX_SOCIAL_POINTS: f64 = 25.0;
|
||||
pub const SCOUR_BONUS: f64 = 8.0;
|
||||
pub const HN_FRONTPAGE_BONUS: f64 = 8.0;
|
||||
pub const MAX_MULTI_SOURCE_POINTS: f64 = 8.0;
|
||||
pub const MAX_FEED_PRIOR_POINTS: f64 = 12.0;
|
||||
pub const EXCERPT_ONLY_PENALTY: f64 = 20.0;
|
||||
pub const ROUNDUP_TITLE_PENALTY: f64 = 15.0;
|
||||
|
||||
/// `composite_social_score` value that earns the full social bonus. Empirically
|
||||
/// ~6.0 is a 1,000-point HN story with 500 comments (§3.4 formula).
|
||||
const SOCIAL_SATURATION: f64 = 6.0;
|
||||
|
||||
/// Everything the pre-filter needs beyond the articles themselves (§3.5, §3.9).
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct PrefilterContext {
|
||||
/// Per-feed Bayesian upvote rate from ratings history (§3.9).
|
||||
pub feed_priors: HashMap<FeedId, FeedPrior>,
|
||||
/// Article ids already published in a previous issue (§3.5).
|
||||
pub already_published: Vec<ArticleId>,
|
||||
/// Article ids the LLM scored < [`STALE_LOW_SCORE`] recently (§3.5).
|
||||
pub recently_rejected: Vec<ArticleId>,
|
||||
}
|
||||
|
||||
impl PrefilterContext {
|
||||
/// Load the history/priors context from SQLite (§3.5 dedup-vs-history, §3.9).
|
||||
///
|
||||
/// `today` anchors the [`STALE_LOOKBACK_DAYS`] window.
|
||||
pub async fn load(
|
||||
db: &crate::db::Db,
|
||||
today: jiff::civil::Date,
|
||||
) -> Result<Self, crate::db::DbError> {
|
||||
let since = today
|
||||
.checked_sub(jiff::Span::new().days(STALE_LOOKBACK_DAYS))
|
||||
.unwrap_or(today);
|
||||
let feed_priors = db
|
||||
.feed_priors()
|
||||
.await?
|
||||
.into_iter()
|
||||
.map(|p| (p.feed_id, p))
|
||||
.collect();
|
||||
let already_published = db.previously_published_ids().await?;
|
||||
let recently_rejected = db.recently_low_scored_ids(STALE_LOW_SCORE, since).await?;
|
||||
tracing::debug!(
|
||||
priors = ?feed_priors_len(&feed_priors),
|
||||
published = already_published.len(),
|
||||
rejected = recently_rejected.len(),
|
||||
"loaded prefilter context"
|
||||
);
|
||||
Ok(Self {
|
||||
feed_priors,
|
||||
already_published,
|
||||
recently_rejected,
|
||||
})
|
||||
}
|
||||
|
||||
fn prior_for(&self, article: &Article) -> f64 {
|
||||
// The cluster's feeds are all candidates; take the most favourable one,
|
||||
// since a story carried by a well-rated feed is a better bet.
|
||||
let mut best = self.feed_priors.get(&article.feed_id).map(FeedPrior::rate);
|
||||
for source in &article.sources {
|
||||
if let Some(p) = self.feed_priors.get(&source.feed_id) {
|
||||
let rate = p.rate();
|
||||
best = Some(best.map_or(rate, |b: f64| b.max(rate)));
|
||||
}
|
||||
}
|
||||
best.unwrap_or(0.5)
|
||||
}
|
||||
}
|
||||
|
||||
fn feed_priors_len(m: &HashMap<FeedId, FeedPrior>) -> usize {
|
||||
m.len()
|
||||
}
|
||||
|
||||
/// True when the article's feed is in `curation.always_include_feeds` (§3.5).
|
||||
///
|
||||
/// Entries are matched either as a Miniflux feed id (any feed in the cluster) or
|
||||
/// as a case-insensitive substring of the article/site URL.
|
||||
///
|
||||
/// Auto-includes are still LLM-scored (for section + summary) but can't be dropped.
|
||||
pub fn is_auto_include(article: &Article, cfg: &CurationConfig) -> bool {
|
||||
if cfg.always_include_feeds.is_empty() {
|
||||
return false;
|
||||
}
|
||||
let url = article.url.to_lowercase();
|
||||
let canonical = article.canonical_url.to_lowercase();
|
||||
cfg.always_include_feeds.iter().any(|raw| {
|
||||
let needle = raw.trim();
|
||||
if needle.is_empty() {
|
||||
return false;
|
||||
}
|
||||
if let Ok(id) = needle.parse::<FeedId>()
|
||||
&& (article.feed_id == id || article.sources.iter().any(|s| s.feed_id == id))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
let needle = needle.to_lowercase();
|
||||
// Bare host or full site URL: compare against both URLs we hold.
|
||||
let needle = needle
|
||||
.trim_start_matches("https://")
|
||||
.trim_start_matches("http://")
|
||||
.trim_end_matches('/');
|
||||
!needle.is_empty() && (url.contains(needle) || canonical.contains(needle))
|
||||
})
|
||||
}
|
||||
|
||||
/// True when the article's host matches `curation.blocked_domains` (§3.5).
|
||||
pub fn is_blocked(article: &Article, cfg: &CurationConfig) -> bool {
|
||||
if cfg.blocked_domains.is_empty() {
|
||||
return false;
|
||||
}
|
||||
let host = host_of(&article.canonical_url)
|
||||
.or_else(|| host_of(&article.url))
|
||||
.unwrap_or_default();
|
||||
if host.is_empty() {
|
||||
return false;
|
||||
}
|
||||
cfg.blocked_domains.iter().any(|raw| {
|
||||
let blocked = raw.trim().trim_start_matches('.').to_lowercase();
|
||||
!blocked.is_empty() && (host == blocked || host.ends_with(&format!(".{blocked}")))
|
||||
})
|
||||
}
|
||||
|
||||
/// Lowercased host of a URL, `www.` stripped.
|
||||
fn host_of(url: &str) -> Option<String> {
|
||||
let rest = url
|
||||
.split_once("://")
|
||||
.map(|(_, rest)| rest)
|
||||
.unwrap_or(url)
|
||||
.split(['/', '?', '#'])
|
||||
.next()?;
|
||||
let host = rest.rsplit_once('@').map(|(_, h)| h).unwrap_or(rest);
|
||||
let host = host.split_once(':').map(|(h, _)| h).unwrap_or(host);
|
||||
let host = host.trim().to_lowercase();
|
||||
if host.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(host.trim_start_matches("www.").to_string())
|
||||
}
|
||||
}
|
||||
|
||||
/// True when the title reads like a link roundup / release note / sponsor post (§3.5).
|
||||
pub fn looks_like_roundup(title: &str) -> bool {
|
||||
let lower = title.to_lowercase();
|
||||
PENALTY_TITLE_PATTERNS
|
||||
.iter()
|
||||
.any(|pattern| lower.contains(pattern))
|
||||
}
|
||||
|
||||
/// Long-form bonus: zero below [`LONGFORM_FLOOR_WORDS`], saturating at
|
||||
/// [`LONGFORM_SATURATION_WORDS`], with a concave curve so that the jump from a
|
||||
/// 400-word note to a 1,200-word piece matters more than 2,000 → 2,500 (§3.5).
|
||||
pub fn longform_points(word_count: i64) -> f64 {
|
||||
let span = (LONGFORM_SATURATION_WORDS - LONGFORM_FLOOR_WORDS) as f64;
|
||||
let over = (word_count - LONGFORM_FLOOR_WORDS).max(0) as f64;
|
||||
MAX_LONGFORM_POINTS * (over / span).min(1.0).powf(0.65)
|
||||
}
|
||||
|
||||
/// Social proof, log-scaled a second time so that a viral story cannot swamp the
|
||||
/// long-form preference (§3.4, §3.5).
|
||||
pub fn social_points(social_score: f64) -> f64 {
|
||||
if social_score <= 0.0 {
|
||||
return 0.0;
|
||||
}
|
||||
MAX_SOCIAL_POINTS * (social_score / SOCIAL_SATURATION).min(1.0).sqrt()
|
||||
}
|
||||
|
||||
/// Score one article 0–100 from word count, social proof, source signals, feed
|
||||
/// prior, and the excerpt/roundup/blocklist penalties (§3.5).
|
||||
pub fn score_article(article: &Article, ctx: &PrefilterContext, cfg: &Config) -> f64 {
|
||||
if is_blocked(article, &cfg.curation) {
|
||||
return 0.0;
|
||||
}
|
||||
let mut score = longform_points(article.word_count);
|
||||
score += social_points(article.social_score());
|
||||
|
||||
if article.came_via(SourceKind::Scour) {
|
||||
score += SCOUR_BONUS;
|
||||
}
|
||||
if article.came_via(SourceKind::HnFrontpage) {
|
||||
score += HN_FRONTPAGE_BONUS;
|
||||
}
|
||||
|
||||
let extra_feeds = article.sources.len().saturating_sub(1) as f64;
|
||||
score += (extra_feeds * 4.0).min(MAX_MULTI_SOURCE_POINTS);
|
||||
|
||||
// Beta-smoothed upvote rate, neutral (0.5) contributing nothing (§3.9).
|
||||
score += (ctx.prior_for(article) - 0.5) * 2.0 * MAX_FEED_PRIOR_POINTS;
|
||||
|
||||
if article.excerpt_only {
|
||||
score -= EXCERPT_ONLY_PENALTY;
|
||||
}
|
||||
if looks_like_roundup(&article.title) {
|
||||
score -= ROUNDUP_TITLE_PENALTY;
|
||||
}
|
||||
|
||||
score.clamp(0.0, 100.0)
|
||||
}
|
||||
|
||||
/// Apply [`score_article`] to everything, drop history duplicates, then keep the
|
||||
/// top `prefilter_keep` plus every auto-include (§3.5).
|
||||
pub fn run(articles: Vec<Article>, ctx: &PrefilterContext, cfg: &Config) -> Vec<ScoredArticle> {
|
||||
let published: HashSet<ArticleId> = ctx.already_published.iter().copied().collect();
|
||||
let rejected: HashSet<ArticleId> = ctx.recently_rejected.iter().copied().collect();
|
||||
|
||||
let total = articles.len();
|
||||
let (mut dropped_history, mut dropped_blocked) = (0usize, 0usize);
|
||||
let mut scored: Vec<ScoredArticle> = Vec::with_capacity(total);
|
||||
|
||||
for article in articles {
|
||||
let auto_include = is_auto_include(&article, &cfg.curation);
|
||||
|
||||
// Never print the same story twice, not even from an always-include feed.
|
||||
if published.contains(&article.id) {
|
||||
dropped_history += 1;
|
||||
continue;
|
||||
}
|
||||
// "Don't re-score churn" (§3.5) — but an always-include feed still gets in.
|
||||
if !auto_include && rejected.contains(&article.id) {
|
||||
dropped_history += 1;
|
||||
continue;
|
||||
}
|
||||
if !auto_include && is_blocked(&article, &cfg.curation) {
|
||||
dropped_blocked += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
let prefilter_score = score_article(&article, ctx, cfg);
|
||||
let social_score = article.social_score();
|
||||
let feed_prior = ctx.prior_for(&article);
|
||||
scored.push(ScoredArticle {
|
||||
article,
|
||||
prefilter_score,
|
||||
social_score,
|
||||
feed_prior,
|
||||
llm: None,
|
||||
auto_include,
|
||||
});
|
||||
}
|
||||
|
||||
// Descending by score; ties broken by word count then id so the order is
|
||||
// deterministic across runs (notes §12).
|
||||
sort_by_prefilter(&mut scored);
|
||||
|
||||
let keep = cfg.prefilter_keep.max(cfg.target_article_count);
|
||||
let kept: Vec<ScoredArticle> = if scored.len() <= keep {
|
||||
scored
|
||||
} else {
|
||||
let (head, tail) = scored.split_at(keep);
|
||||
let mut kept = head.to_vec();
|
||||
// Auto-includes below the cut are pulled back in — they can't be dropped.
|
||||
kept.extend(tail.iter().filter(|s| s.auto_include).cloned());
|
||||
sort_by_prefilter(&mut kept);
|
||||
kept
|
||||
};
|
||||
|
||||
tracing::info!(
|
||||
input = total,
|
||||
kept = kept.len(),
|
||||
auto_includes = kept.iter().filter(|s| s.auto_include).count(),
|
||||
dropped_history,
|
||||
dropped_blocked,
|
||||
"pre-filter complete"
|
||||
);
|
||||
kept
|
||||
}
|
||||
|
||||
/// Deterministic ranking: score desc, then longer, then lowest id (notes §12).
|
||||
pub fn sort_by_prefilter(scored: &mut [ScoredArticle]) {
|
||||
scored.sort_by(|a, b| {
|
||||
b.prefilter_score
|
||||
.partial_cmp(&a.prefilter_score)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
.then_with(|| b.article.word_count.cmp(&a.article.word_count))
|
||||
.then_with(|| a.article.id.cmp(&b.article.id))
|
||||
});
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) mod tests {
|
||||
use super::*;
|
||||
use crate::types::{ExtractMethod, SocialRef, SocialSource, SourceRef};
|
||||
use jiff::Timestamp;
|
||||
|
||||
pub(crate) fn ts() -> Timestamp {
|
||||
"2026-08-15T05:30:00Z"
|
||||
.parse()
|
||||
.expect("static timestamp parses")
|
||||
}
|
||||
|
||||
/// A plain 800-word article from feed 7 with no social proof.
|
||||
pub(crate) fn article(id: ArticleId, title: &str, word_count: i64) -> Article {
|
||||
Article {
|
||||
id,
|
||||
canonical_url: format!("https://example.com/{id}"),
|
||||
title: title.into(),
|
||||
best_entry_id: 1000 + id,
|
||||
content_html: format!("<p>{}</p>", "word ".repeat(word_count.max(0) as usize)),
|
||||
word_count,
|
||||
excerpt_only: false,
|
||||
image_count: 0,
|
||||
sources: vec![SourceRef {
|
||||
entry_id: 1000 + id,
|
||||
feed_id: 7,
|
||||
feed_title: "Some Blog".into(),
|
||||
category: Some("Tech".into()),
|
||||
kind: SourceKind::Feed,
|
||||
}],
|
||||
first_seen: ts(),
|
||||
url: format!("https://example.com/{id}"),
|
||||
author: Some("A. Writer".into()),
|
||||
feed_id: 7,
|
||||
feed_title: "Some Blog".into(),
|
||||
category: Some("Tech".into()),
|
||||
published_at: Some(ts()),
|
||||
comments_url: None,
|
||||
image_urls: vec![],
|
||||
social: vec![],
|
||||
extract_method: ExtractMethod::Readability,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn with_social(mut a: Article, points: i64, comments: i64) -> Article {
|
||||
a.social = vec![SocialRef {
|
||||
article_id: a.id,
|
||||
source: SocialSource::Hn,
|
||||
item_id: Some("1".into()),
|
||||
score: points,
|
||||
num_comments: comments,
|
||||
item_url: Some("https://news.ycombinator.com/item?id=1".into()),
|
||||
fetched_at: ts(),
|
||||
}];
|
||||
a
|
||||
}
|
||||
|
||||
pub(crate) fn via(mut a: Article, kind: SourceKind, feed_id: FeedId) -> Article {
|
||||
a.sources.push(SourceRef {
|
||||
entry_id: a.best_entry_id,
|
||||
feed_id,
|
||||
feed_title: format!("{kind:?} feed"),
|
||||
category: None,
|
||||
kind,
|
||||
});
|
||||
a
|
||||
}
|
||||
|
||||
fn cfg() -> Config {
|
||||
Config {
|
||||
prefilter_keep: 3,
|
||||
target_article_count: 2,
|
||||
..Config::default()
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn longform_curve_is_monotonic_and_bounded() {
|
||||
assert_eq!(longform_points(0), 0.0);
|
||||
assert_eq!(longform_points(LONGFORM_FLOOR_WORDS), 0.0);
|
||||
let mut prev = -1.0;
|
||||
for wc in [0, 100, 299, 300, 500, 900, 1500, 2200, 2500, 9000] {
|
||||
let pts = longform_points(wc);
|
||||
assert!(pts >= prev, "not monotonic at {wc}");
|
||||
assert!(pts <= MAX_LONGFORM_POINTS);
|
||||
prev = pts;
|
||||
}
|
||||
assert!((longform_points(2500) - MAX_LONGFORM_POINTS).abs() < 1e-9);
|
||||
assert!((longform_points(50_000) - MAX_LONGFORM_POINTS).abs() < 1e-9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn social_curve_is_monotonic_and_bounded() {
|
||||
let mut prev = -1.0;
|
||||
for s in [0.0, 0.5, 1.0, 2.0, 4.0, 6.0, 20.0] {
|
||||
let pts = social_points(s);
|
||||
assert!(pts >= prev);
|
||||
assert!(pts <= MAX_SOCIAL_POINTS);
|
||||
prev = pts;
|
||||
}
|
||||
assert_eq!(social_points(0.0), 0.0);
|
||||
assert!((social_points(6.0) - MAX_SOCIAL_POINTS).abs() < 1e-9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn score_rises_with_length_and_social_proof() {
|
||||
let (ctx, cfg) = (PrefilterContext::default(), cfg());
|
||||
let short = score_article(&article(1, "A thought", 200), &ctx, &cfg);
|
||||
let medium = score_article(&article(2, "An essay", 1200), &ctx, &cfg);
|
||||
let long = score_article(&article(3, "A treatise", 3000), &ctx, &cfg);
|
||||
assert!(short < medium, "{short} !< {medium}");
|
||||
assert!(medium < long, "{medium} !< {long}");
|
||||
|
||||
let quiet = score_article(&article(4, "An essay", 1200), &ctx, &cfg);
|
||||
let loud = score_article(
|
||||
&with_social(article(5, "An essay", 1200), 400, 250),
|
||||
&ctx,
|
||||
&cfg,
|
||||
);
|
||||
assert!(loud > quiet);
|
||||
assert!(loud <= 100.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn source_bonuses_and_penalties_apply() {
|
||||
let (ctx, cfg) = (PrefilterContext::default(), cfg());
|
||||
// Long enough that the penalties do not run into the 0 floor.
|
||||
let plain = score_article(&article(1, "Deep dive", 3000), &ctx, &cfg);
|
||||
assert!(plain > EXCERPT_ONLY_PENALTY);
|
||||
|
||||
let scoured = score_article(
|
||||
&via(article(2, "Deep dive", 3000), SourceKind::Scour, 42),
|
||||
&ctx,
|
||||
&cfg,
|
||||
);
|
||||
// Scour bonus + one extra feed in the cluster.
|
||||
assert!(scoured > plain + SCOUR_BONUS - 0.001);
|
||||
|
||||
let mut excerpt = article(3, "Deep dive", 3000);
|
||||
excerpt.excerpt_only = true;
|
||||
assert!(
|
||||
(score_article(&excerpt, &ctx, &cfg) - (plain - EXCERPT_ONLY_PENALTY)).abs() < 1e-9
|
||||
);
|
||||
|
||||
let roundup = article(4, "This Week in Rust #612", 3000);
|
||||
assert!(looks_like_roundup(&roundup.title));
|
||||
assert!(
|
||||
(score_article(&roundup, &ctx, &cfg) - (plain - ROUNDUP_TITLE_PENALTY)).abs() < 1e-9
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn feed_prior_moves_the_score_both_ways() {
|
||||
let cfg = cfg();
|
||||
let mut liked = PrefilterContext::default();
|
||||
liked.feed_priors.insert(
|
||||
7,
|
||||
FeedPrior {
|
||||
feed_id: 7,
|
||||
upvotes: 18,
|
||||
downvotes: 0,
|
||||
included: 18,
|
||||
},
|
||||
);
|
||||
let mut disliked = PrefilterContext::default();
|
||||
disliked.feed_priors.insert(
|
||||
7,
|
||||
FeedPrior {
|
||||
feed_id: 7,
|
||||
upvotes: 0,
|
||||
downvotes: 18,
|
||||
included: 18,
|
||||
},
|
||||
);
|
||||
let a = article(1, "Deep dive", 1200);
|
||||
let neutral = score_article(&a, &PrefilterContext::default(), &cfg);
|
||||
assert!(score_article(&a, &liked, &cfg) > neutral);
|
||||
assert!(score_article(&a, &disliked, &cfg) < neutral);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blocked_domains_and_auto_includes_match_urls_and_ids() {
|
||||
let mut cfg = cfg();
|
||||
cfg.curation.blocked_domains = vec!["spam.example".into()];
|
||||
cfg.curation.always_include_feeds = vec!["99".into(), "tyler.blog".into()];
|
||||
|
||||
let mut blocked = article(1, "Buy now", 1200);
|
||||
blocked.canonical_url = "https://news.spam.example/post".into();
|
||||
blocked.url.clone_from(&blocked.canonical_url);
|
||||
assert!(is_blocked(&blocked, &cfg.curation));
|
||||
assert_eq!(
|
||||
score_article(&blocked, &PrefilterContext::default(), &cfg),
|
||||
0.0
|
||||
);
|
||||
|
||||
let mut by_url = article(2, "A rare post", 900);
|
||||
by_url.url = "https://tyler.blog/2026/rare".into();
|
||||
assert!(is_auto_include(&by_url, &cfg.curation));
|
||||
|
||||
let mut by_id = article(3, "Another rare post", 900);
|
||||
by_id.feed_id = 99;
|
||||
assert!(is_auto_include(&by_id, &cfg.curation));
|
||||
|
||||
assert!(!is_auto_include(&article(4, "Normal", 900), &cfg.curation));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keeps_top_n_plus_auto_includes_and_drops_history() {
|
||||
let mut cfg = cfg();
|
||||
cfg.prefilter_keep = 2;
|
||||
cfg.curation.always_include_feeds = vec!["99".into()];
|
||||
|
||||
let mut auto = article(5, "A short personal note", 120);
|
||||
auto.feed_id = 99;
|
||||
|
||||
let articles = vec![
|
||||
article(1, "Long treatise", 4000),
|
||||
article(2, "Medium essay", 1500),
|
||||
article(3, "Shorter piece", 700),
|
||||
article(4, "Already printed", 5000),
|
||||
auto,
|
||||
article(6, "Rejected yesterday", 3000),
|
||||
];
|
||||
let ctx = PrefilterContext {
|
||||
already_published: vec![4],
|
||||
recently_rejected: vec![6],
|
||||
..PrefilterContext::default()
|
||||
};
|
||||
|
||||
let kept = run(articles, &ctx, &cfg);
|
||||
let ids: Vec<ArticleId> = kept.iter().map(|s| s.article.id).collect();
|
||||
assert!(!ids.contains(&4), "previously published must be dropped");
|
||||
assert!(!ids.contains(&6), "recently rejected must be dropped");
|
||||
assert!(ids.contains(&5), "auto-include survives below the cut");
|
||||
assert!(ids.contains(&1) && ids.contains(&2));
|
||||
assert!(!ids.contains(&3), "cut at prefilter_keep");
|
||||
assert_eq!(kept.len(), 3); // 2 kept + 1 auto-include
|
||||
|
||||
// Sorted by score, descending.
|
||||
for pair in kept.windows(2) {
|
||||
assert!(pair[0].prefilter_score >= pair[1].prefilter_score);
|
||||
}
|
||||
assert!(
|
||||
kept.iter()
|
||||
.find(|s| s.article.id == 5)
|
||||
.is_some_and(|s| s.auto_include)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn auto_include_survives_the_recently_rejected_list_but_not_republication() {
|
||||
let mut cfg = cfg();
|
||||
cfg.curation.always_include_feeds = vec!["99".into()];
|
||||
let mut a = article(1, "Personal note", 200);
|
||||
a.feed_id = 99;
|
||||
let mut b = article(2, "Personal note two", 200);
|
||||
b.feed_id = 99;
|
||||
|
||||
let ctx = PrefilterContext {
|
||||
recently_rejected: vec![1],
|
||||
already_published: vec![2],
|
||||
..PrefilterContext::default()
|
||||
};
|
||||
let kept = run(vec![a, b], &ctx, &cfg);
|
||||
let ids: Vec<ArticleId> = kept.iter().map(|s| s.article.id).collect();
|
||||
assert_eq!(ids, vec![1]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn context_loads_history_from_sqlite() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let db = crate::db::Db::open_and_migrate(&dir.path().join("t.db"))
|
||||
.await
|
||||
.expect("db");
|
||||
let date: jiff::civil::Date = "2026-08-15".parse().expect("date");
|
||||
|
||||
db.upsert_feed_prior(&FeedPrior {
|
||||
feed_id: 7,
|
||||
upvotes: 4,
|
||||
downvotes: 1,
|
||||
included: 5,
|
||||
})
|
||||
.await
|
||||
.expect("prior");
|
||||
sqlx::query(
|
||||
"INSERT INTO articles (id, canonical_url, title, first_seen) VALUES
|
||||
(42, 'https://example.com/42', 'Printed', '2026-08-14T00:00:00Z'),
|
||||
(43, 'https://example.com/43', 'Rejected', '2026-08-14T00:00:00Z'),
|
||||
(44, 'https://example.com/44', 'Ancient', '2020-01-01T00:00:00Z')",
|
||||
)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.expect("articles");
|
||||
db.upsert_issue(
|
||||
"2026-08-14".parse().expect("date"),
|
||||
1,
|
||||
ts(),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.expect("issue");
|
||||
sqlx::query(
|
||||
"INSERT INTO issue_articles (issue_date, article_id, section, position, is_lead)
|
||||
VALUES ('2026-08-14', 42, 'Top Stories', 1, 0)",
|
||||
)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.expect("issue article");
|
||||
sqlx::query(
|
||||
"INSERT INTO scores (article_id, run_date, llm_score) VALUES (43, '2026-08-14', 1.5),
|
||||
(44, '2020-01-01', 1.0)",
|
||||
)
|
||||
.execute(db.pool())
|
||||
.await
|
||||
.expect("scores");
|
||||
|
||||
let ctx = PrefilterContext::load(&db, date).await.expect("context");
|
||||
assert_eq!(ctx.already_published, vec![42]);
|
||||
assert_eq!(ctx.recently_rejected, vec![43], "old rejects age out");
|
||||
assert!((ctx.feed_priors[&7].rate() - 5.0 / 7.0).abs() < 1e-12);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,593 @@
|
||||
//! Stage A — batched LLM scoring (spec §3.6).
|
||||
//!
|
||||
//! Batches of `deepseek.score_batch_size` articles per request. Per article we
|
||||
//! send title, source feed, author, word count, social stats, sources list and a
|
||||
//! ~200-word excerpt; the model returns one JSON object per article.
|
||||
//!
|
||||
//! Parsing is deliberately forgiving: one malformed item must not cost us the
|
||||
//! other eleven, and a failed batch must not fail the run.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fmt::Write as _;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
use super::llm::{LlmClient, LlmError, strip_code_fence};
|
||||
use super::{html_to_text, truncate_words};
|
||||
use crate::types::{ArticleId, LlmScore, ScoredArticle, SourceKind};
|
||||
|
||||
/// Words of article text sent per candidate in stage A (§3.6).
|
||||
pub const EXCERPT_WORDS: usize = 200;
|
||||
|
||||
/// One element of the stage-A JSON response (§3.6).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct ScoreItem {
|
||||
pub id: ArticleId,
|
||||
/// 0–10.
|
||||
pub score: f64,
|
||||
pub category: String,
|
||||
/// ≤ 20 words.
|
||||
#[serde(default)]
|
||||
pub rationale: String,
|
||||
#[serde(default)]
|
||||
pub is_paywalled_guess: bool,
|
||||
}
|
||||
|
||||
impl From<ScoreItem> for LlmScore {
|
||||
fn from(i: ScoreItem) -> Self {
|
||||
LlmScore {
|
||||
score: i.score,
|
||||
category: i.category,
|
||||
rationale: i.rationale,
|
||||
is_paywalled_guess: i.is_paywalled_guess,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Envelope the model is asked to return (`{"articles": [...]}`).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct ScoreResponse {
|
||||
#[serde(default)]
|
||||
pub articles: Vec<ScoreItem>,
|
||||
}
|
||||
|
||||
/// The invariant instruction block for stage A. Everything article-specific goes
|
||||
/// in the per-batch tail so this prefix stays cacheable (§3.6).
|
||||
pub const SCORE_INSTRUCTIONS: &str = "\
|
||||
TASK: score a batch of candidate articles for today's issue of The Daily EPUB.
|
||||
|
||||
Judge each article against the reader profile in your system prompt — not against \
|
||||
a general audience, and not against what is objectively newsworthy.
|
||||
|
||||
Return one object per input article with these fields:
|
||||
\"id\" integer, copied exactly from the input
|
||||
\"score\" number 0-10, the rubric below
|
||||
\"category\" one short label from the palette below
|
||||
\"rationale\" at most 20 words, concrete, no hedging, no restating the title
|
||||
\"is_paywalled_guess\" true when the text looks truncated, teaser-like or paywalled
|
||||
|
||||
SCORING RUBRIC — calibrate hard; a normal day averages about 4, and a 9 should \
|
||||
appear a couple of times a week, not a couple of times a day:
|
||||
9-10 Exceptional. Original reporting, a deep technical dive, or an essay he \
|
||||
will still be thinking about next week. Evident effort and a real point of view.
|
||||
7-8 Strong. A well-made long-form piece squarely in his interests, or an \
|
||||
outstanding piece outside them.
|
||||
5-6 Worth a slot on a thin day. Solid, useful, a little thin or a little \
|
||||
familiar.
|
||||
3-4 Marginal. Competent news-of-the-day, short posts, incremental updates, \
|
||||
good writing about an over-covered story.
|
||||
1-2 Weak. Announcements, changelogs and release notes, link roundups, \
|
||||
listicles, rewrites of a story available at the source, thin AI-industry churn.
|
||||
0 Unusable. Press releases, sponsored content, engagement bait, spam, \
|
||||
pure crypto promotion, or an entry with no readable body.
|
||||
|
||||
CALIBRATION NOTES
|
||||
- Length alone is not quality; padding scores worse than a tight short piece. But \
|
||||
between two equally good pieces, prefer the one with more substance.
|
||||
- Social proof is evidence, not a verdict: hundreds of HN points mean a critical \
|
||||
audience read it; a quiet post from a good blog can still outrank it.
|
||||
- \"came via scour\" means the story already matched one of his standing \
|
||||
interests. \"came via hn_frontpage\" means it cleared HN's front page.
|
||||
- Boston/New England local stories and ultra-niche community news get a genuine \
|
||||
lift — this paper wants them.
|
||||
- Wire-service world/US news should score low here: the World Briefing section \
|
||||
covers that separately.
|
||||
- Excerpt-only or paywalled text is a real cost to the reader; score it lower \
|
||||
unless the piece is clearly excellent.
|
||||
|
||||
Return JSON exactly in this shape, with one entry per input article and nothing \
|
||||
else:
|
||||
{\"articles\": [{\"id\": 123, \"score\": 7.5, \"category\": \"Tech & Engineering\", \
|
||||
\"rationale\": \"first-hand account of migrating 40TB off Postgres\", \
|
||||
\"is_paywalled_guess\": false}]}";
|
||||
|
||||
/// Render the user prompt for one batch (§3.6).
|
||||
pub fn build_batch_prompt(batch: &[ScoredArticle], sections: &[String]) -> String {
|
||||
let mut prompt = String::with_capacity(4096 + batch.len() * 1500);
|
||||
prompt.push_str(SCORE_INSTRUCTIONS);
|
||||
let _ = write!(
|
||||
prompt,
|
||||
"\n\nCATEGORY PALETTE (use one of these exact strings): {}\n\nARTICLES ({} in this batch)\n",
|
||||
sections.join(" | "),
|
||||
batch.len()
|
||||
);
|
||||
for candidate in batch {
|
||||
prompt.push('\n');
|
||||
prompt.push_str(&render_candidate(candidate));
|
||||
}
|
||||
prompt
|
||||
}
|
||||
|
||||
/// One article's block in the stage-A prompt (§3.6).
|
||||
fn render_candidate(candidate: &ScoredArticle) -> String {
|
||||
let a = &candidate.article;
|
||||
let mut block = String::with_capacity(1500);
|
||||
let _ = writeln!(block, "--- id: {}", a.id);
|
||||
let _ = writeln!(block, "title: {}", a.title.trim());
|
||||
let _ = writeln!(
|
||||
block,
|
||||
"feed: {}{}",
|
||||
if a.feed_title.is_empty() {
|
||||
"unknown"
|
||||
} else {
|
||||
a.feed_title.trim()
|
||||
},
|
||||
a.category
|
||||
.as_deref()
|
||||
.filter(|c| !c.is_empty())
|
||||
.map(|c| format!(" (category: {c})"))
|
||||
.unwrap_or_default()
|
||||
);
|
||||
if let Some(author) = a.author.as_deref().filter(|s| !s.trim().is_empty()) {
|
||||
let _ = writeln!(block, "author: {}", author.trim());
|
||||
}
|
||||
let _ = writeln!(
|
||||
block,
|
||||
"length: {} words (~{} min read){}",
|
||||
a.word_count,
|
||||
a.reading_minutes(),
|
||||
if a.excerpt_only {
|
||||
" [EXCERPT ONLY — full text unavailable]"
|
||||
} else {
|
||||
""
|
||||
}
|
||||
);
|
||||
let _ = writeln!(block, "social: {}", social_line(candidate));
|
||||
let _ = writeln!(block, "came via: {}", sources_line(candidate));
|
||||
let excerpt = truncate_words(&html_to_text(&a.content_html), EXCERPT_WORDS);
|
||||
let _ = writeln!(
|
||||
block,
|
||||
"excerpt: {}",
|
||||
if excerpt.is_empty() {
|
||||
"(no body text extracted)"
|
||||
} else {
|
||||
&excerpt
|
||||
}
|
||||
);
|
||||
block
|
||||
}
|
||||
|
||||
fn social_line(candidate: &ScoredArticle) -> String {
|
||||
if candidate.article.social.is_empty() {
|
||||
return "none found".into();
|
||||
}
|
||||
let mut parts: Vec<String> = candidate
|
||||
.article
|
||||
.social
|
||||
.iter()
|
||||
.map(|s| {
|
||||
format!(
|
||||
"{} {} points / {} comments",
|
||||
s.source.display_name(),
|
||||
s.score,
|
||||
s.num_comments
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
parts.push(format!("composite {:.2}", candidate.social_score));
|
||||
parts.join("; ")
|
||||
}
|
||||
|
||||
fn sources_line(candidate: &ScoredArticle) -> String {
|
||||
let mut kinds: Vec<&str> = candidate
|
||||
.article
|
||||
.sources
|
||||
.iter()
|
||||
.map(|s| match s.kind {
|
||||
SourceKind::Scour => "scour",
|
||||
SourceKind::HnFrontpage => "hn_frontpage",
|
||||
SourceKind::Lobsters => "lobsters",
|
||||
SourceKind::Reddit => "reddit",
|
||||
SourceKind::Feed => "feed",
|
||||
})
|
||||
.collect();
|
||||
kinds.sort_unstable();
|
||||
kinds.dedup();
|
||||
if candidate.auto_include {
|
||||
kinds.push("always-include feed (cannot be dropped)");
|
||||
}
|
||||
if kinds.is_empty() {
|
||||
"feed".into()
|
||||
} else {
|
||||
kinds.join(", ")
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Response parsing (§3.6: tolerate anything the model does to us)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Keys the model might wrap the array in, in preference order.
|
||||
const ARRAY_KEYS: &[&str] = &["articles", "scores", "results", "items", "data"];
|
||||
|
||||
/// Parse a stage-A response leniently: missing optional fields default, scores
|
||||
/// are clamped to 0–10, and malformed items are skipped with a warning (§3.6).
|
||||
pub fn parse_score_response(raw: &str) -> Vec<ScoreItem> {
|
||||
let cleaned = strip_code_fence(raw);
|
||||
let value: Value = match serde_json::from_str(cleaned) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "stage A response was not JSON at all");
|
||||
return Vec::new();
|
||||
}
|
||||
};
|
||||
|
||||
let array = match &value {
|
||||
Value::Array(items) => Some(items),
|
||||
Value::Object(map) => ARRAY_KEYS
|
||||
.iter()
|
||||
.find_map(|k| map.get(*k).and_then(Value::as_array))
|
||||
// Some models return {"1234": {...}} or a single bare object.
|
||||
.or_else(|| map.values().find_map(Value::as_array)),
|
||||
_ => None,
|
||||
};
|
||||
let Some(array) = array else {
|
||||
tracing::warn!("stage A response contained no array of scores");
|
||||
return Vec::new();
|
||||
};
|
||||
|
||||
let mut out = Vec::with_capacity(array.len());
|
||||
let mut skipped = 0usize;
|
||||
for item in array {
|
||||
match parse_item(item) {
|
||||
Some(parsed) => out.push(parsed),
|
||||
None => {
|
||||
skipped += 1;
|
||||
tracing::warn!(item = %truncate_debug(item), "skipping malformed stage A item");
|
||||
}
|
||||
}
|
||||
}
|
||||
if skipped > 0 {
|
||||
tracing::warn!(skipped, kept = out.len(), "stage A items were dropped");
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn parse_item(item: &Value) -> Option<ScoreItem> {
|
||||
let obj = item.as_object()?;
|
||||
let id = obj.get("id").and_then(as_i64_lenient)?;
|
||||
let score = obj
|
||||
.get("score")
|
||||
.and_then(as_f64_lenient)
|
||||
.or_else(|| obj.get("rating").and_then(as_f64_lenient))?;
|
||||
Some(ScoreItem {
|
||||
id,
|
||||
score: score.clamp(0.0, 10.0),
|
||||
category: obj
|
||||
.get("category")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default()
|
||||
.trim()
|
||||
.to_string(),
|
||||
rationale: obj
|
||||
.get("rationale")
|
||||
.or_else(|| obj.get("reason"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default()
|
||||
.trim()
|
||||
.to_string(),
|
||||
is_paywalled_guess: obj
|
||||
.get("is_paywalled_guess")
|
||||
.or_else(|| obj.get("paywalled"))
|
||||
.and_then(as_bool_lenient)
|
||||
.unwrap_or(false),
|
||||
})
|
||||
}
|
||||
|
||||
fn as_i64_lenient(v: &Value) -> Option<i64> {
|
||||
v.as_i64()
|
||||
.or_else(|| v.as_f64().map(|f| f as i64))
|
||||
.or_else(|| v.as_str().and_then(|s| s.trim().parse().ok()))
|
||||
}
|
||||
|
||||
fn as_f64_lenient(v: &Value) -> Option<f64> {
|
||||
v.as_f64()
|
||||
.or_else(|| v.as_str().and_then(|s| s.trim().parse().ok()))
|
||||
.filter(|f| f.is_finite())
|
||||
}
|
||||
|
||||
fn as_bool_lenient(v: &Value) -> Option<bool> {
|
||||
v.as_bool().or_else(|| match v.as_str()?.trim() {
|
||||
"true" | "yes" => Some(true),
|
||||
"false" | "no" => Some(false),
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
|
||||
fn truncate_debug(v: &Value) -> String {
|
||||
v.to_string().chars().take(160).collect()
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Stage driver
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Score every candidate, filling in [`ScoredArticle::llm`] (§3.6).
|
||||
///
|
||||
/// Batches that fail are logged and left unscored rather than aborting the run.
|
||||
/// Returns how many candidates came back with a score.
|
||||
pub async fn score_all(
|
||||
llm: &LlmClient,
|
||||
candidates: &mut [ScoredArticle],
|
||||
batch_size: usize,
|
||||
sections: &[String],
|
||||
temperature: f32,
|
||||
) -> Result<usize, LlmError> {
|
||||
if candidates.is_empty() {
|
||||
return Ok(0);
|
||||
}
|
||||
let batch_size = batch_size.max(1);
|
||||
let batches = candidates.len().div_ceil(batch_size);
|
||||
let mut scores: HashMap<ArticleId, LlmScore> = HashMap::with_capacity(candidates.len());
|
||||
|
||||
for (n, batch) in candidates.chunks(batch_size).enumerate() {
|
||||
if let Err(e) = llm.meter.check_budget() {
|
||||
tracing::error!(
|
||||
error = %e,
|
||||
batch = n + 1,
|
||||
of = batches,
|
||||
unscored = candidates.len() - scores.len(),
|
||||
"COST CEILING HIT during stage A scoring — remaining batches skipped; \
|
||||
the lineup will fall back to heuristic ranking for them"
|
||||
);
|
||||
break;
|
||||
}
|
||||
let prompt = build_batch_prompt(batch, sections);
|
||||
tracing::debug!(
|
||||
batch = n + 1,
|
||||
of = batches,
|
||||
articles = batch.len(),
|
||||
approx_tokens = super::approx_tokens(&prompt),
|
||||
"stage A request"
|
||||
);
|
||||
match llm.complete(&prompt, temperature, true).await {
|
||||
Ok(raw) => {
|
||||
let items = parse_score_response(&raw);
|
||||
if items.is_empty() {
|
||||
tracing::warn!(
|
||||
batch = n + 1,
|
||||
of = batches,
|
||||
"stage A batch returned no scores"
|
||||
);
|
||||
}
|
||||
for item in items {
|
||||
scores.insert(item.id, item.into());
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(batch = n + 1, of = batches, error = %e,
|
||||
"stage A batch failed; its articles stay unscored");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut applied = 0usize;
|
||||
for candidate in candidates.iter_mut() {
|
||||
if let Some(score) = scores.remove(&candidate.article.id) {
|
||||
candidate.llm = Some(score);
|
||||
applied += 1;
|
||||
}
|
||||
}
|
||||
if !scores.is_empty() {
|
||||
tracing::warn!(
|
||||
unknown_ids = scores.len(),
|
||||
"stage A returned scores for ids that were not in the batch"
|
||||
);
|
||||
}
|
||||
Ok(applied)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::config::DeepseekConfig;
|
||||
use crate::curate::llm::{MockBackend, UsageMeter};
|
||||
use crate::curate::prefilter::tests::{article, via, with_social};
|
||||
use crate::types::TokenUsage;
|
||||
use std::sync::Arc;
|
||||
|
||||
const BATCH_FIXTURE: &str = include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/tests/fixtures/deepseek_score_batch.json"
|
||||
));
|
||||
const MESSY_FIXTURE: &str = include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/tests/fixtures/deepseek_score_batch_messy.json"
|
||||
));
|
||||
|
||||
fn sections() -> Vec<String> {
|
||||
crate::config::CurationConfig::default().sections
|
||||
}
|
||||
|
||||
fn candidate(id: i64, title: &str, words: i64) -> ScoredArticle {
|
||||
ScoredArticle {
|
||||
article: article(id, title, words),
|
||||
prefilter_score: 50.0,
|
||||
social_score: 0.0,
|
||||
feed_prior: 0.5,
|
||||
llm: None,
|
||||
auto_include: false,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn batch_prompt_carries_every_documented_signal() {
|
||||
let mut c = candidate(12, "Migrating 40TB off Postgres", 3200);
|
||||
c.article = via(
|
||||
with_social(c.article, 342, 210),
|
||||
SourceKind::HnFrontpage,
|
||||
9001,
|
||||
);
|
||||
c.social_score = c.article.social_score();
|
||||
c.auto_include = true;
|
||||
let prompt = build_batch_prompt(&[c], §ions());
|
||||
|
||||
assert!(prompt.starts_with(SCORE_INSTRUCTIONS));
|
||||
assert!(prompt.contains("--- id: 12"));
|
||||
assert!(prompt.contains("title: Migrating 40TB off Postgres"));
|
||||
assert!(prompt.contains("feed: Some Blog (category: Tech)"));
|
||||
assert!(prompt.contains("author: A. Writer"));
|
||||
assert!(prompt.contains("length: 3200 words"));
|
||||
assert!(prompt.contains("HN 342 points / 210 comments"));
|
||||
assert!(prompt.contains("hn_frontpage"));
|
||||
assert!(prompt.contains("always-include feed"));
|
||||
assert!(prompt.contains("excerpt: word word"));
|
||||
assert!(prompt.contains("Tech & Engineering"));
|
||||
// The excerpt is capped.
|
||||
let excerpt_line = prompt
|
||||
.lines()
|
||||
.find(|l| l.starts_with("excerpt:"))
|
||||
.expect("excerpt line");
|
||||
assert!(excerpt_line.split_whitespace().count() <= EXCERPT_WORDS + 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_a_realistic_deepseek_batch() {
|
||||
let items = parse_score_response(BATCH_FIXTURE);
|
||||
assert_eq!(items.len(), 4);
|
||||
assert_eq!(items[0].id, 101);
|
||||
assert!((items[0].score - 8.5).abs() < 1e-9);
|
||||
assert_eq!(items[0].category, "Tech & Engineering");
|
||||
assert!(items[0].rationale.split_whitespace().count() <= 20);
|
||||
assert!(!items[0].is_paywalled_guess);
|
||||
assert!(items[3].is_paywalled_guess);
|
||||
let score: LlmScore = items[0].clone().into();
|
||||
assert_eq!(score.category, "Tech & Engineering");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parsing_survives_everything_a_model_might_do() {
|
||||
let items = parse_score_response(MESSY_FIXTURE);
|
||||
let ids: Vec<ArticleId> = items.iter().map(|i| i.id).collect();
|
||||
// 201 fine; 202 string score clamped; 203 missing rationale/category;
|
||||
// 204 out-of-range clamped; the two malformed entries are dropped.
|
||||
assert_eq!(ids, vec![201, 202, 203, 204]);
|
||||
assert!((items[1].score - 6.0).abs() < 1e-9);
|
||||
assert_eq!(items[2].rationale, "");
|
||||
assert_eq!(items[2].category, "");
|
||||
assert!(
|
||||
(items[3].score - 10.0).abs() < 1e-9,
|
||||
"clamped to the 0-10 range"
|
||||
);
|
||||
assert!(items.iter().all(|i| (0.0..=10.0).contains(&i.score)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parsing_tolerates_fences_arrays_and_junk() {
|
||||
assert_eq!(
|
||||
parse_score_response("```json\n{\"articles\":[{\"id\":1,\"score\":5}]}\n```").len(),
|
||||
1
|
||||
);
|
||||
assert_eq!(parse_score_response("[{\"id\": 2, \"score\": 3}]").len(), 1);
|
||||
assert_eq!(
|
||||
parse_score_response("{\"results\":[{\"id\":3,\"score\":\"4.5\"}]}")[0].score,
|
||||
4.5
|
||||
);
|
||||
assert!(parse_score_response("I'm sorry, I can't do that").is_empty());
|
||||
assert!(parse_score_response("{\"articles\": {}}").is_empty());
|
||||
}
|
||||
|
||||
fn client(backend: Arc<MockBackend>, limit_usd: f64) -> LlmClient {
|
||||
LlmClient::with_backend(
|
||||
"deepseek-v4-flash",
|
||||
"SYSTEM".into(),
|
||||
UsageMeter::new(&DeepseekConfig::default(), limit_usd),
|
||||
backend,
|
||||
)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn scores_are_applied_batch_by_batch() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push(
|
||||
r#"{"articles":[{"id":1,"score":8,"category":"Tech & Engineering","rationale":"good"},
|
||||
{"id":2,"score":2,"category":"Niche Corner","rationale":"thin"}]}"#,
|
||||
TokenUsage::default(),
|
||||
);
|
||||
backend.push(
|
||||
r#"{"articles":[{"id":3,"score":6.5,"category":"Culture & Essays","rationale":"solid"}]}"#,
|
||||
TokenUsage::default(),
|
||||
);
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
|
||||
let mut candidates = vec![
|
||||
candidate(1, "One", 1000),
|
||||
candidate(2, "Two", 1000),
|
||||
candidate(3, "Three", 1000),
|
||||
];
|
||||
let scored = score_all(&llm, &mut candidates, 2, §ions(), 0.3)
|
||||
.await
|
||||
.expect("scoring");
|
||||
assert_eq!(scored, 3);
|
||||
assert_eq!(backend.calls(), 2, "batched by score_batch_size");
|
||||
assert_eq!(candidates[0].llm.as_ref().map(|l| l.score), Some(8.0));
|
||||
assert_eq!(candidates[2].llm.as_ref().map(|l| l.score), Some(6.5));
|
||||
// combined_score now reflects the LLM verdict.
|
||||
assert!(candidates[0].combined_score() > candidates[1].combined_score());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_failed_batch_does_not_sink_the_run() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
backend.push_error("500 upstream exploded");
|
||||
backend.push(
|
||||
r#"{"articles":[{"id":2,"score":7,"category":"Top Stories","rationale":"ok"}]}"#,
|
||||
TokenUsage::default(),
|
||||
);
|
||||
let llm = client(Arc::clone(&backend), 2.0);
|
||||
let mut candidates = vec![candidate(1, "One", 900), candidate(2, "Two", 900)];
|
||||
let scored = score_all(&llm, &mut candidates, 1, §ions(), 0.3)
|
||||
.await
|
||||
.expect("scoring must not abort");
|
||||
assert_eq!(scored, 1);
|
||||
assert!(candidates[0].llm.is_none());
|
||||
assert!(candidates[1].llm.is_some());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn scoring_stops_when_the_budget_is_gone() {
|
||||
let backend = Arc::new(MockBackend::new());
|
||||
// First batch alone blows a $0.05 ceiling ($0.14 per 1M input tokens).
|
||||
backend.push(
|
||||
r#"{"articles":[{"id":1,"score":9,"category":"Top Stories","rationale":"great"}]}"#,
|
||||
TokenUsage {
|
||||
input_tokens: 1_000_000,
|
||||
cached_tokens: 0,
|
||||
output_tokens: 0,
|
||||
},
|
||||
);
|
||||
backend.push(
|
||||
r#"{"articles":[{"id":2,"score":9,"category":"Top Stories","rationale":"great"}]}"#,
|
||||
TokenUsage::default(),
|
||||
);
|
||||
let llm = client(Arc::clone(&backend), 0.05);
|
||||
let mut candidates = vec![candidate(1, "One", 900), candidate(2, "Two", 900)];
|
||||
let scored = score_all(&llm, &mut candidates, 1, §ions(), 0.3)
|
||||
.await
|
||||
.expect("scoring");
|
||||
assert_eq!(scored, 1, "only the first batch ran");
|
||||
assert_eq!(backend.calls(), 1);
|
||||
assert!(llm.meter.budget_exceeded());
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user