Stop routine excerpt fallbacks from degrading every run
Every issue since 2026-08-15 landed as `degraded`, because the extraction stage warned on any fetch failure at all and a few always fail: the worst of the 21 published days was 15% (33 of 220 articles), the median under 11%. A status every run carries says nothing, so warn only past a 30% share -- roughly twice the worst day seen -- and log the rest. The exact count was already in `counts.excerpt_only` either way. `rebuild()` also stored the bumped profile version before reading the interests OPML, so a rebuild that failed on a missing file would mark the profile fresh for another week having never rewritten its text. Read the prompt inputs first, ahead of both the model call and the writes, so a failure stays due and costs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0164rQrMUZkf7uV1VGYuCFy4
This commit is contained in:
+22
-4
@@ -346,6 +346,11 @@ struct StageContext<'a> {
|
||||
rescore: bool,
|
||||
}
|
||||
|
||||
/// Share of the day's articles that must fall back to a feed excerpt before the
|
||||
/// run is degraded over it. Single-digit percentages are routine; the exact count
|
||||
/// is always in `counts.excerpt_only`.
|
||||
const EXCERPT_FALLBACK_WARN_SHARE: f64 = 0.30;
|
||||
|
||||
async fn run_stages(
|
||||
ctx: &StageContext<'_>,
|
||||
window_start: Timestamp,
|
||||
@@ -412,11 +417,24 @@ async fn run_stages(
|
||||
let extract_stats = extractor.extract_all(&mut articles).await;
|
||||
report.counts.extracted = (extract_stats.from_miniflux + extract_stats.from_readability) as i64;
|
||||
report.counts.excerpt_only = extract_stats.excerpt_only as i64;
|
||||
// A handful of fetch failures is the normal state of the open web, so only a
|
||||
// day well past the usual rate is worth degrading the run over.
|
||||
if extract_stats.fetch_failures > 0 {
|
||||
report.warn(format!(
|
||||
"{} articles fell back to a feed excerpt",
|
||||
extract_stats.fetch_failures
|
||||
));
|
||||
let share = extract_stats.fetch_failures as f64 / articles.len().max(1) as f64;
|
||||
if share >= EXCERPT_FALLBACK_WARN_SHARE {
|
||||
report.warn(format!(
|
||||
"{} of {} articles ({:.0}%) fell back to a feed excerpt",
|
||||
extract_stats.fetch_failures,
|
||||
articles.len(),
|
||||
share * 100.0
|
||||
));
|
||||
} else {
|
||||
tracing::info!(
|
||||
failures = extract_stats.fetch_failures,
|
||||
articles = articles.len(),
|
||||
"articles fell back to a feed excerpt"
|
||||
);
|
||||
}
|
||||
}
|
||||
report.timings.record("extract", elapsed_ms(stage));
|
||||
|
||||
|
||||
Reference in New Issue
Block a user