thallada/the-daily-epub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bisect provider-rejected batches in triage and deep assessment

Browse Source 
DeepSeek's content filter rejects a whole request (400 "Content Exists
Risk") when one article trips it, which cost the other articles in the
batch their assessment and retried them every run. New curate/batch.rs
runs both stages through a bisecting runner: a rejected batch is split
until the offending article is isolated, that article is retried once on
the editor provider when it is a different one, and a still-rejected
article is recorded as a provider_rejected assessment row so it is not
retried for assessment_reuse_days. Cache reuse accepts rows from either
configured model. Each stage logs reused/requested/rejected counts, the
curation: line shows rejections when non-zero, explain prints them, and
the llm_assess span reports the deep-set size.

Implemented by a Claude agent from an orchestrator brief; verified
fmt/clippy(-W dead_code)/test green (354 lib tests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
This commit is contained in:
thallada
2026-09-02 20:44:28 +00:00
co-authored by Claude Fable 5.1
parent 7a842b61b6
commit af2a3ceb49
14 changed files with 1711 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/2026-08-15-implementation-notes.md
+10
View File
@@ -211,3 +211,13 @@ implementer needs that are easy to get wrong:
concurrency (`triage`, `assess`) is the bulk provider's `max_concurrent_requests`; the
summaries fan out at the summary provider's. `Models { bulk, editor, summaries }` in the
colophon and Behind the paper stay model ids taken from the built clients.
- **Provider rejections** (2026-09-02): `curate::batch` bisects a triage or deep batch that
fails with a non-transient error (`Api`, `Refusal`, `EmptyResponse`, or a response that
parses to zero items) down to single articles; a rejected single is retried once on the
editor client when it is another provider. An article both refuse gets an
`article_assessments` row with `kind = 'provider_rejected'`, `score`/`fit` NULL,
`rationale = "<provider>: <message ≤ 200 chars>"`, the bulk `model` and the stage's
`prompt_version`; the cache loaders skip it (leaving the assessment absent) while it is within
`assessment_reuse_days`, `--rescore` ignores it, and `admit::hygiene` never treats it as a low
score. Because a recovered article's row carries the editor's model, reuse accepts rows whose
`model` is either configured model (`triage::reusable_models`).
docs/runbooks/curation-v2-migration.md
+6 -1
View File
@@ -257,7 +257,12 @@ sudo systemd-run --quiet --wait --pty --collect --uid=daily-epub --gid=daily-epu
```
produces the same date's paper with Gemini as the editor. Triage and deep assessments are cached
for three days, so the second run costs only the editor, summaries and the Brief. Compare the two
per article for three days, so the second run re-requests only articles that have no row yet:
ones ingested since the first run, and ones whose earlier batch was rejected by the provider's
content filter (those are bisected, retried once on the editor provider, and then recorded as
`provider_rejected` so they are not retried daily). The `triage:` and `assess:` log lines say how
many were reused versus requested. The bulk-side cost of the second run is therefore small; the
editor, summaries and the Brief are the real spend. Compare the two
lineups, the `why` lines and the Brief side by side, and the `providers:` cost line. To switch
for good, set `editor = "gemini"` in `[llm]` (and `summary_model` stays `editor`, so summaries
move with it). The same trick works for the bulk role: `DAILY_EPUB_LLM__BULK=gemini`.