Bisect provider-rejected batches in triage and deep assessment

DeepSeek's content filter rejects a whole request (400 "Content Exists
Risk") when one article trips it, which cost the other articles in the
batch their assessment and retried them every run. New curate/batch.rs
runs both stages through a bisecting runner: a rejected batch is split
until the offending article is isolated, that article is retried once on
the editor provider when it is a different one, and a still-rejected
article is recorded as a provider_rejected assessment row so it is not
retried for assessment_reuse_days. Cache reuse accepts rows from either
configured model. Each stage logs reused/requested/rejected counts, the
curation: line shows rejections when non-zero, explain prints them, and
the llm_assess span reports the deep-set size.

Implemented by a Claude agent from an orchestrator brief; verified
fmt/clippy(-W dead_code)/test green (354 lib tests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
This commit is contained in:
2026-09-02 20:44:28 +00:00
co-authored by Claude Fable 5.1
parent 7a842b61b6
commit af2a3ceb49
14 changed files with 1711 additions and 127 deletions
+13
View File
@@ -685,5 +685,18 @@ From spec §7, plus what implementation turned up:
beginning/middle/end sample, separates editorial quality from reader fit, and
records descriptive facets. Utility is normalized over the deep set; embedding
leader clusters cap near-duplicates before the 60-item editor shortlist.
- **DeepSeek's content filter rejects whole batches.** A `400 Content Exists
Risk` refuses the entire request when any one article in it trips the input
filter, without saying which. Triage and deep assessment therefore bisect a
rejected batch (halves, then quarters, down to single articles) so the other
articles keep their assessment. A single article the bulk provider still
refuses is retried once on the editor provider with the same prompt when that
is a different provider; if that also fails, an `article_assessments` row with
`kind = 'provider_rejected'` and a NULL score is written so the article is not
sent again for `assessment_reuse_days`, and it is ranked on its other signals.
See them with `explain` (`triage: rejected by provider — deepseek: …`), the
`triage:` / `assess:` log lines (`… 3 rejected (2 recovered on gemini)`), the
` · N rejected` suffix on the `curation:` line, or
`sqlite3 /var/lib/daily-epub/daily-epub.db "select stage, count(*) from article_assessments where kind = 'provider_rejected' group by stage"`.
- **One reader, one issue per day.** There is no multi-user support and no
weekly/retrospective edition (spec §6).