From 99d1338890682f85b21e2f5deeb119f4644cd1a3 Mon Sep 17 00:00:00 2001 From: Tyler Hallada Date: Wed, 2 Sep 2026 18:44:57 +0000 Subject: [PATCH] Provider-agnostic LLM registry, config check, migration runbook [llm] assigns the bulk and editor roles by name over a [providers.*] registry (kind = openai | anthropic, per-provider model, effort, daily ceiling and price table); DeepseekBackend becomes OpenAiCompatibleBackend (reasoning_effort passthrough), AnthropicBackend builds from the same ProviderConfig, meters and provider_costs are keyed by provider name. Gemini 3.8 Flash is declared via Google's OpenAI-compatible endpoint so switching the editor is one line (or DAILY_EPUB_LLM__EDITOR=gemini for an A/B dry run). Stale [deepseek]/[anthropic] tables, the top-level max_daily_usd and the old key env vars fail loudly. daily-epub config check validates and prints the resolved roles, models, key presence and paths without opening the database. docs/runbooks/curation-v2-migration.md walks the server upgrade from v1. Registry implemented by a Claude agent from an orchestrator brief; verified fmt/clippy(-W dead_code)/test green. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe --- README.md | 153 ++- config.example.toml | 74 +- docs/plans/2026-08-15-implementation-notes.md | 62 +- docs/plans/2026-09-02-curation-v2-progress.md | 10 +- docs/runbooks/curation-v2-migration.md | 278 ++++++ src/config.rs | 913 +++++++++++++++--- src/curate/assess.rs | 6 +- src/curate/editor.rs | 10 +- src/curate/editorial.rs | 33 +- src/curate/llm.rs | 796 ++++++++++----- src/curate/mod.rs | 12 +- src/curate/profile/mod.rs | 4 +- src/curate/telemetry.rs | 5 +- src/curate/triage.rs | 8 +- src/main.rs | 62 +- src/pipeline.rs | 232 +++-- src/world.rs | 7 +- tests/config_check.rs | 69 ++ tests/e2e_pipeline.rs | 22 +- 19 files changed, 2136 insertions(+), 620 deletions(-) create mode 100644 docs/runbooks/curation-v2-migration.md create mode 100644 tests/config_check.rs diff --git a/README.md b/README.md index 9cda068..2bca22f 100644 --- a/README.md +++ b/README.md @@ -21,9 +21,12 @@ it all cost. Steady-state cost is roughly **$1/day**: $0.05–0.30 in DeepSeek tokens plus ~$0.50–0.80 for the Claude editor and a few cents of Voyage AI embeddings, each -with its own per-UTC-day ceiling (`max_daily_usd`, `anthropic.max_daily_usd` and +provider with its own per-UTC-day ceiling (`providers..max_daily_usd` and `voyage.max_daily_usd`). Those ceilings are runaway guards, not accounting — set -hard spend limits in the providers' dashboards as the real backstop. +hard spend limits in the providers' dashboards as the real backstop. The two +LLM roles — *bulk* (triage, assessment, fallbacks) and *editor* — are assigned +by name to entries of a provider registry, so swapping DeepSeek or Claude for +Gemini (or anything OpenAI-compatible) is a config line plus an API key. - Full design: [`docs/plans/2026-08-15-the-daily-epub.md`](docs/plans/2026-08-15-the-daily-epub.md) - Implementation decisions: [`docs/plans/2026-08-15-implementation-notes.md`](docs/plans/2026-08-15-implementation-notes.md) @@ -34,9 +37,9 @@ hard spend limits in the providers' dashboards as the real backstop. ``` Miniflux ingest ─▶ dedupe ─▶ extraction ─▶ persist ─▶ social enrichment - ─▶ hygiene ─▶ embeddings (Voyage) + cheap signals ─▶ triage (DeepSeek) - ─▶ union admission ─▶ deep assessment (DeepSeek) ─▶ utility + diversity - ─▶ editor (Claude) ─▶ comments ─▶ editorial (Claude) + ─▶ hygiene ─▶ embeddings (Voyage) + cheap signals ─▶ triage (bulk LLM) + ─▶ union admission ─▶ deep assessment (bulk LLM) ─▶ utility + diversity + ─▶ editor (editor LLM) ─▶ comments ─▶ editorial (editor LLM) ─▶ world briefing ─▶ EPUB (standard + X4) ─▶ XTC ─▶ publish ─▶ report ``` @@ -66,12 +69,16 @@ per-provider usage) is stored on the `runs` row and in `issues.report_json`. are fatal — without them there is no issue, and the `runs` row records why. Social lookups, comment fetching, the world briefing, images and the XTC conversion are best-effort: they log, add a warning (run status `degraded`) and -the run continues. Every LLM stage *degrades*: a Claude call that fails, is -refused, or is over its daily ceiling is retried with the same prompt on -DeepSeek; if DeepSeek is missing, dead or over budget too, the run takes the -`--skip-llm` shape (admission uses cheap signals and feed excerpts stand in for -summaries) instead of losing the day's issue. Anthropic's server-side refusal -fallback (`fallbacks = "default"`) is enabled on every editor request. +the run continues. Every LLM stage *degrades*: an editor call that fails, is +refused, or is over its provider's daily ceiling is retried with the same +prompt on the bulk provider; if the bulk provider is missing, dead or over +budget too, the run takes the `--skip-llm` shape (admission uses cheap signals +and feed excerpts stand in for summaries) instead of losing the day's issue. +Which provider plays which role is the `[llm]` table (`bulk = "deepseek"`, +`editor = "anthropic"` by default); a role whose key is absent is simply +unavailable, and `daily-epub config check` shows the resolved assignment before +a run. Anthropic's server-side refusal fallback (`fallbacks = "default"`) is +enabled on every request to an `anthropic`-kind provider. --- @@ -81,8 +88,8 @@ fallback (`fallbacks = "default"`) is enabled on every editor request. |---|---|---| | Rust (2024 edition toolchain) | building | `cargo build --release` | | **Miniflux** with an API key | the only content source | Settings → API Keys. The client is read-only and never mutates read state. | -| **DeepSeek API key** | triage and deep assessment, and the fallback for every editor call | . Optional: `--skip-llm` runs the whole pipeline without it. | -| **Anthropic API key** | the editor: selection, summaries, The Brief, the weekly profile rebuild | . Optional: without it every editor call runs on DeepSeek. Set a dashboard spend limit; `anthropic.max_daily_usd` is only a runaway guard. | +| A key for the **bulk** provider (DeepSeek by default) | triage and deep assessment, and the fallback for every editor call | . `DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY`. Optional: `--skip-llm` runs the whole pipeline without it. | +| A key for the **editor** provider (Anthropic by default) | the editor: selection, summaries, The Brief, the weekly profile rebuild | . `DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY`. Optional: without it every editor call runs on the bulk provider. Set a dashboard spend limit; `providers.anthropic.max_daily_usd` is only a runaway guard. Any other `[providers.*]` entry (Gemini is shipped) can take either role — see *Switching providers*. | | **Voyage AI API key** | article and interest embeddings behind the learned ranking signals | . Optional: without it (or with `--skip-embeddings`) the run uses cached vectors only and the learned signals are absent, never a penalty. | | A 32+ byte random secret | signs the article rating links | `openssl rand -hex 32` | | **BookOrbit** library + watched folder | *optional* — a richer library UI on top of the same folder | Delivery does not need it: `daily-epub serve` has its own OPDS catalog over `publish.epub_dir`. If you do run it, create a dedicated "The Daily EPUB" library, enable *Watch folders*, and point `publish.epub_dir` at it. | @@ -119,6 +126,7 @@ daily-epub features backfill [--days 30] [--rated-only] [--all] [--yes] daily-epub features prune # stale embeddings, old telemetry and assessments daily-epub backfill-social [--days 7] # re-poll social scores for recent articles daily-epub db migrate # run migrations (also automatic on every start) +daily-epub config check # validate the config, print the resolved roles, keys and paths ``` `--dry-run` does everything except deliver: it still ingests, persists entries and @@ -160,6 +168,15 @@ than `curation.ranking.embedding_retention_days`, and `candidate_runs` rows and `article_assessments` older than `curation.ranking.telemetry_retention_days`. `generate` runs the same sweep once after publishing, best effort. +`config check` loads and validates the configuration exactly as `generate` +would and prints one fact per line: the config path, the database, profile, +interests and publish paths with `exists`/`MISSING`, each `[llm]` role with its +provider name, kind, model, effort, ceiling and whether its key is present, the +Voyage line likewise, and `editorial.summary_model`. Lines that need attention +start with `!`. It exits non-zero only on a validation error — a missing key or +file is a warning, since the run degrades rather than fails — and never opens +the database or takes the lock, so it is safe to run next to a live `generate`. + --- ## Configuration @@ -171,12 +188,43 @@ Start from [`config.example.toml`](config.example.toml). Load order, later wins: 3. `DAILY_EPUB_*` environment variables Nested keys use a **double underscore**: `[miniflux] api_key` becomes -`DAILY_EPUB_MINIFLUX__API_KEY`. Top-level keys are just uppercased: -`DAILY_EPUB_LOOKBACK_HOURS=30`. As a convenience, plain **`DAILY_EPUB_SECRET`** -is accepted as an alias for `server.hmac_secret` (the explicit key wins if both -are set). +`DAILY_EPUB_MINIFLUX__API_KEY`, and a provider's key is +`DAILY_EPUB_PROVIDERS____API_KEY` with the `[providers.]` table name +upper-cased — the shipped registry reads `DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY`, +`DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY` and +`DAILY_EPUB_PROVIDERS__GEMINI__API_KEY` (provider names are therefore lowercase +`a-z0-9_`). Top-level keys are just uppercased: `DAILY_EPUB_LOOKBACK_HOURS=30`. +As a convenience, plain **`DAILY_EPUB_SECRET`** is accepted as an alias for +`server.hmac_secret` (the explicit key wins if both are set). -Secrets belong in the environment file, never in the TOML. +Secrets belong in the environment file, never in the TOML. Stale configuration +fails at startup rather than silently curating without a provider: a +`[deepseek]` or `[anthropic]` table, a top-level `max_daily_usd`, a batch-size +or temperature key outside `[llm]`, or a `DAILY_EPUB_DEEPSEEK__*` / +`DAILY_EPUB_ANTHROPIC__*` environment variable is an error naming the new key. + +### Switching providers + +The roles are names, the providers are tables. To run the editor on Gemini: + +```toml +[llm] +editor = "gemini" # [providers.gemini] is already declared in config.example.toml +``` + +and put `DAILY_EPUB_PROVIDERS__GEMINI__API_KEY=…` in the env file. A one-off +A/B without touching the file, since every key is also an env var: + +```sh +DAILY_EPUB_LLM__EDITOR=gemini daily-epub generate --dry-run --date 2026-09-02 +``` + +The same works for `bulk`. Both roles may name one provider (they then share +one client and one `max_daily_usd`), `editor = ""` runs everything on bulk, and +a new endpoint is a new `[providers.]` table: `kind = "openai"` for any +OpenAI-compatible chat-completions API (DeepSeek, Gemini, OpenAI, a local +server), `kind = "anthropic"` for the Messages API. `daily-epub config check` +prints what resolved. ### Reference @@ -187,7 +235,6 @@ Secrets belong in the environment file, never in the TOML. | `target_article_count` | `20` | Soft target the editor aims for. There is no minimum: a nine-pick issue is published as nine. | | `retention_days` | `21` | EPUBs older than this are deleted from `publish.epub_dir`. SQLite history is kept forever. | | `xtc_retention_count` | `5` | How many XTC issues to keep in `publish.xtc_dir`. Counted, not dated: each `.xtch` is ~80–100 MB, so the binding constraint is disk, not age. | -| `max_daily_usd` | `2.0` | Ceiling on DeepSeek spend per **UTC day** of the run's start, not per run — a re-run inherits what earlier runs that day already spent (`runs.provider_costs_json`). Tripping it skips remaining DeepSeek calls; in-flight requests finish and the paper still publishes. | | `world_briefing` | `true` | Include the Wikipedia Current Events section. | | `database_path` | `/var/lib/daily-epub/daily-epub.db` | SQLite file; parent dirs are created. | | `out_dir` | `/var/lib/daily-epub/out` | Where `generate` writes artifacts before publishing. | @@ -196,28 +243,23 @@ Secrets belong in the environment file, never in the TOML. | `miniflux.base_url` | `http://127.0.0.1:8082` | Miniflux root (no `/v1`). | | `miniflux.api_key` | — | **`DAILY_EPUB_MINIFLUX__API_KEY`**. Required. | | `miniflux.page_limit` | `250` | Entries per page; Miniflux caps this at 250. | -| `deepseek.base_url` | `https://api.deepseek.com/v1` | OpenAI-compatible endpoint. | -| `deepseek.model` | `deepseek-v4-flash` | Verified 2026-08-15 (DeepSeek-V4-Flash-0731). | -| `deepseek.api_key` | — | **`DAILY_EPUB_DEEPSEEK__API_KEY`**. Absent ⇒ the run curates heuristically. | -| `deepseek.deep_batch_size` | `8` | Articles per close-reading assessment request. The removed `score_batch_size` key is a startup error. | -| `deepseek.triage_batch_size` | `25` | Articles per first-pass triage request. | -| `deepseek.max_concurrent_requests` | `4` | Triage and deep-assessment batches in flight at once; the budget is checked before each is spawned. | -| `deepseek.score_temperature` | `0.3` | Scoring temperature. | -| `deepseek.editorial_temperature` | `0.8` | Summaries and The Brief, only when DeepSeek is the fallback editor. | -| `deepseek.price_input_per_mtok` | `0.14` | USD per 1M cache-miss input tokens (cost guardrail arithmetic). | -| `deepseek.price_cached_input_per_mtok` | `0.0028` | USD per 1M prefix-cache-hit input tokens. | -| `deepseek.price_output_per_mtok` | `0.28` | USD per 1M output tokens. | -| `anthropic.enabled` | `true` | `false` runs every editor call on DeepSeek. | -| `anthropic.base_url` | `https://api.anthropic.com` | Messages API root. | -| `anthropic.model` | `claude-opus-5` | The editor. Requests carry `output_config.effort`, a cached system block, and `fallbacks = "default"` with the `server-side-fallback-2026-07-01` beta so a classifier refusal is re-routed server-side. | -| `anthropic.api_key` | — | **`DAILY_EPUB_ANTHROPIC__API_KEY`**. Absent ⇒ editor calls fall back to DeepSeek. | -| `anthropic.effort` | `high` | `low`, `medium`, `high`, `xhigh` or `max`. | -| `anthropic.price_input_per_mtok` | `5.0` | USD per 1M uncached input tokens. | -| `anthropic.price_cache_write_per_mtok` | `6.25` | USD per 1M tokens written to the prompt cache. | -| `anthropic.price_cache_read_per_mtok` | `0.5` | USD per 1M cache-read input tokens. | -| `anthropic.price_output_per_mtok` | `25.0` | USD per 1M output tokens. | -| `anthropic.max_daily_usd` | `3.0` | Claude ceiling per UTC day; tripping it moves the remaining editor work to DeepSeek. | -| `anthropic.max_concurrent_requests` | `4` | Reserved for the parallel editor stages. | +| `llm.bulk` | `deepseek` | The `[providers.*]` name that runs triage, deep assessment and every fallback. `""` ⇒ no bulk provider (those stages are skipped). | +| `llm.editor` | `anthropic` | The provider that assembles the lineup, writes the summaries and The Brief and rebuilds the profile. `""` or absent ⇒ everything runs on `bulk`. Naming the same provider as `bulk` shares one client and one ceiling. | +| `llm.triage_batch_size` | `25` | Articles per first-pass triage request. | +| `llm.deep_batch_size` | `8` | Articles per close-reading assessment request. The removed `score_batch_size` key is a startup error. | +| `llm.score_temperature` | `0.3` | Scoring temperature, sent only to `openai`-kind providers. | +| `llm.editorial_temperature` | `0.8` | Summaries and The Brief on an `openai`-kind provider. | +| `providers..kind` | — | `openai` (chat completions at `{base_url}/chat/completions`, bearer key) or `anthropic` (Messages API: `output_config.effort`, a cached system block, `fallbacks = "default"` with the `server-side-fallback-2026-07-01` beta). Shipped entries: `deepseek`, `anthropic`, `gemini`. | +| `providers..base_url` | — | Endpoint root. DeepSeek `https://api.deepseek.com/v1`; Anthropic `https://api.anthropic.com`; Gemini `https://generativelanguage.googleapis.com/v1beta/openai`. | +| `providers..model` | — | `deepseek-v4-flash` (verified 2026-08-15), `claude-opus-5`, `gemini-3.8-flash` (verified 2026-09-02). | +| `providers..api_key` | — | **`DAILY_EPUB_PROVIDERS____API_KEY`**, environment only. Absent ⇒ that role is unavailable and degrades (bulk ⇒ heuristic curation, editor ⇒ bulk). | +| `providers..effort` | `high` (anthropic, gemini) | `anthropic`: `low`, `medium`, `high`, `xhigh` or `max` → `output_config.effort`. `openai`: passed through as `reasoning_effort` (Gemini takes `minimal`–`high`); omit it for models without one (DeepSeek). | +| `providers..max_daily_usd` | `2.0` / `3.0` / `3.0` | That provider's ceiling per **UTC day** of the run's start, not per run — a re-run inherits what earlier runs that day already spent on it (`runs.provider_costs_json`). Tripping it skips that provider's remaining calls; in-flight requests finish and the paper still publishes. `0` disables the guard. | +| `providers..max_concurrent_requests` | `4` | Triage and deep-assessment batches in flight on the bulk provider; summaries in flight on the summary provider. | +| `providers..price_input_per_mtok` | `0.14` / `5.0` / `0.75` | USD per 1M cache-miss input tokens (cost guardrail arithmetic). | +| `providers..price_cache_read_per_mtok` | `0.0028` / `0.5` / `0.075` | USD per 1M cache-hit input tokens. | +| `providers..price_cache_write_per_mtok` | `0.0` / `6.25` / `0.0` | USD per 1M tokens written to the prompt cache (implicit caches charge nothing). | +| `providers..price_output_per_mtok` | `0.28` / `25.0` / `3.75` | USD per 1M output tokens, thinking tokens included where the provider bills them as output. | | `voyage.enabled` | `true` | Embed articles and interests with Voyage AI. `false` ⇒ cached vectors only. | | `voyage.base_url` | `https://api.voyageai.com/v1` | `POST {base_url}/embeddings`. | | `voyage.model` | `voyage-4-lite` | Embedding model; changing it invalidates the cache. | @@ -239,7 +281,7 @@ Secrets belong in the environment file, never in the TOML. | `curation.recent_rejection_days` | `7` | Churn window for recent low triage/deep assessments. | | `curation.recent_rejection_floor` | `3.0` | Scores below this floor are excluded during the churn window (except auto-includes). | | `curation.ranking.*` | see below | Every weight, quota, gate and threshold of the personalized ranker. | -| `editorial.summary_model` | `editor` | `editor` (Claude) or `bulk` (DeepSeek) for the per-article summaries. | +| `editorial.summary_model` | `editor` | Which `[llm]` role writes the per-article summaries: `editor` (with per-article bulk fallback) or `bulk`. | | `editorial.summary_input_tokens` | `3000` | Article text offered to the summary prompt. | | `publish.epub_dir` | `/srv/bookorbit/libraries/daily-epub` | Both EPUB editions land here by atomic copy, and this is the directory the OPDS feed lists. The editions are distinguished by a `(X4)` tag in **both** the filename and `dc:title` — libraries and OPDS clients list books by title, so the filename alone would make them look identical. Point a BookOrbit watched folder at it if you want its UI too. **Renamed from `bookorbit_dir`**; the old key is a hard config error. | | `publish.xtc_dir` | `/var/lib/daily-epub/xtc` | XTC artifacts. **Not** listed in the OPDS feed — CrossPoint cannot acquire them — but downloadable at `/files/xtc/` for sideloading. | @@ -298,15 +340,20 @@ sudo install -m0640 -o daily-epub -g daily-epub config.example.toml /etc/daily-e sudo -e /etc/daily-epub/config.toml # set publish dirs, xtc args, public_url sudo tee /etc/daily-epub/env >/dev/null <] entry a role uses, named after the table +# (upper-cased). Any may be left unset: that role is then unavailable and the +# run degrades (editor → bulk, bulk → heuristic curation, Voyage → cached +# vectors). The pre-registry DAILY_EPUB_DEEPSEEK__API_KEY / +# DAILY_EPUB_ANTHROPIC__API_KEY names are a startup error, not a silent no-op. +sudo -u daily-epub bash -c 'set -a; . /etc/daily-epub/env; set +a; + daily-epub --config /etc/daily-epub/config.toml config check' # roles, keys present?, paths # publish dirs must exist and be writable by the service user sudo install -d -o daily-epub -g daily-epub /var/lib/daily-epub/xtc @@ -465,6 +512,7 @@ Condensed from spec §5. Run it in this order the first time. ```sh # 1. Config and schema +daily-epub --config /etc/daily-epub/config.toml config check # roles → providers, keys present? daily-epub --config /etc/daily-epub/config.toml db migrate # 2. Ingest only, no keys spent: does Miniflux answer, and with how much? @@ -477,7 +525,7 @@ epubcheck "./out/The Daily EPUB - $(date +%F).epub" # expect zero errors # open the standard edition in Calibre / KOReader: cover, The Brief, # In This Issue, sections, discussions, Behind the paper, colophon; TOC depth 2 -# 4. Now with DeepSeek and Claude, still not publishing +# 4. Now with the bulk and editor providers, still not publishing daily-epub generate --dry-run --out ./out --max-articles 6 # → check the lineup is sane (at most 6 picks, each with a "why" line) and the # printed per-provider cost is well under $1 @@ -622,10 +670,11 @@ From spec §7, plus what implementation turned up: stack, so `curate/llm.rs` speaks the OpenAI-compatible wire protocol over the shared `reqwest` client instead, behind a `ChatBackend` trait. The dependency was removed. -- **Two chat providers are wired**, DeepSeek (bulk) and Anthropic (editor), - each a `ChatBackend` impl with its own `UsageMeter` and price table. A third - means another impl. Voyage AI embeddings sit behind the analogous - `EmbeddingBackend` trait in `curate/embedding.rs`. +- **Two wire protocols are implemented**, `openai` (chat completions) and + `anthropic` (Messages API), each a `ChatBackend` impl. Providers are config + entries over those two kinds, each with its own `UsageMeter`, price table and + ceiling; a protocol that is neither means another impl. Voyage AI embeddings + sit behind the analogous `EmbeddingBackend` trait in `curate/embedding.rs`. - **Triage and union admission replace the heuristic gate.** Every eligible article gets interest, rated-neighbour, feed-affinity, social and heuristic signals, then DeepSeek reads its opening (up to `triage_max`). The deep set is diff --git a/config.example.toml b/config.example.toml index b423329..b95d071 100644 --- a/config.example.toml +++ b/config.example.toml @@ -3,18 +3,19 @@ # Load order (later wins): built-in defaults ← this file ← `DAILY_EPUB_*` env vars. # Nested keys use a double underscore in env vars, e.g. # DAILY_EPUB_MINIFLUX__API_KEY=... -# DAILY_EPUB_DEEPSEEK__API_KEY=... -# DAILY_EPUB_ANTHROPIC__API_KEY=... +# DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY=... +# DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY=... +# DAILY_EPUB_PROVIDERS__GEMINI__API_KEY=... # DAILY_EPUB_VOYAGE__API_KEY=... # DAILY_EPUB_SERVER__HMAC_SECRET=... # DAILY_EPUB_LOOKBACK_HOURS=30 +# DAILY_EPUB_LLM__EDITOR=gemini # one-off role override, no file edit timezone = "America/New_York" lookback_hours = 26 target_article_count = 20 retention_days = 21 # EPUBs, by age xtc_retention_count = 5 # XTC issues, by count (~80-100 MB each) -max_daily_usd = 2.0 # DeepSeek ceiling per UTC day; [anthropic] and [voyage] have their own world_briefing = true # SQLite database file. Parent directories are created on demand. @@ -32,37 +33,66 @@ base_url = "http://127.0.0.1:8082" # api_key via DAILY_EPUB_MINIFLUX__API_KEY env page_limit = 250 -[deepseek] +# The two LLM roles, each assigned to a provider declared in [providers.*] +# below. Switching the editor to Gemini is `editor = "gemini"` plus its key in +# the env file; nothing else changes. An empty editor ("") runs everything on +# the bulk provider. Every editor call degrades to bulk when the editor's key +# is missing, its daily ceiling is hit, or the API refuses/fails. +[llm] +bulk = "deepseek" # triage, deep assessment, and every fallback +editor = "anthropic" # lineup, summaries, The Brief, the weekly profile rebuild +triage_batch_size = 25 # articles per first-pass triage request +deep_batch_size = 8 # articles per close-reading assessment request +score_temperature = 0.3 # sent only by providers that take a temperature (kind = "openai") +editorial_temperature = 0.8 # summaries and The Brief on an openai-kind provider + +# The provider registry. Any number of entries; a role above names one by its +# table name. Keys never live here: DAILY_EPUB_PROVIDERS____API_KEY. +# `max_daily_usd` is a per-provider runaway guard per UTC day (0 = none), not +# accounting — set hard spend limits in each provider's dashboard as well. +# Prices are USD per 1M tokens and only feed the guard's arithmetic. +[providers.deepseek] +kind = "openai" # openai | anthropic base_url = "https://api.deepseek.com/v1" model = "deepseek-v4-flash" # DeepSeek-V4-Flash-0731 (confirmed 2026-08-15) -# api_key via DAILY_EPUB_DEEPSEEK__API_KEY env -deep_batch_size = 8 # articles per close-reading assessment request -triage_batch_size = 25 # articles per first-pass triage request +# api_key via DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY env +max_daily_usd = 2.0 max_concurrent_requests = 4 # triage and deep-assessment batches in flight -score_temperature = 0.3 -editorial_temperature = 0.8 # used only when DeepSeek is the fallback editor -# USD per 1M tokens, used for the cost guardrail. price_input_per_mtok = 0.14 -price_cached_input_per_mtok = 0.0028 +price_cache_read_per_mtok = 0.0028 # prefix-cache hits +price_cache_write_per_mtok = 0.0 # DeepSeek caches implicitly, no write charge price_output_per_mtok = 0.28 -# Claude is the editor: selection, summaries, The Brief and the weekly profile -# rebuild. Every call degrades to DeepSeek when the key is missing, the daily -# ceiling is hit, or the API refuses/fails. Server-side refusal fallback -# (`fallbacks = "default"`) is always on. Set a spend limit in the Anthropic -# dashboard too: `max_daily_usd` is a runaway guard, not accounting. -[anthropic] -enabled = true +# Claude Opus 5 over the Messages API. Requests carry `output_config.effort`, +# a cached system block, and `fallbacks = "default"` so a classifier refusal is +# re-routed server-side; a refusal that still comes back degrades to bulk. +[providers.anthropic] +kind = "anthropic" base_url = "https://api.anthropic.com" model = "claude-opus-5" -# api_key via DAILY_EPUB_ANTHROPIC__API_KEY env -effort = "high" # low | medium | high | xhigh | max +# api_key via DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY env +effort = "high" # low | medium | high | xhigh | max → output_config.effort +max_daily_usd = 3.0 +max_concurrent_requests = 4 # summaries in flight when this is the summary provider price_input_per_mtok = 5.0 -price_cache_write_per_mtok = 6.25 price_cache_read_per_mtok = 0.5 +price_cache_write_per_mtok = 6.25 price_output_per_mtok = 25.0 + +# Gemini 3.8 Flash through Google's OpenAI-compatible endpoint (beta, verified +# 2026-09-02). Declared but unreferenced until a role names it. +[providers.gemini] +kind = "openai" +base_url = "https://generativelanguage.googleapis.com/v1beta/openai" +model = "gemini-3.8-flash" +# api_key via DAILY_EPUB_PROVIDERS__GEMINI__API_KEY env +effort = "high" # minimal | low | medium | high → reasoning_effort max_daily_usd = 3.0 max_concurrent_requests = 4 +price_input_per_mtok = 0.75 # promotional through 2026-12-31; $1.50 from 2027-01-01 +price_cache_read_per_mtok = 0.075 # implicit cache hits; $0.15 from 2027-01-01 +price_cache_write_per_mtok = 0.0 +price_output_per_mtok = 3.75 # includes thinking tokens; $7.50 from 2027-01-01 # Voyage AI embeddings behind the interest and rated-neighbour signals. Set # `enabled = false` (or leave the key unset) and the paper still builds: the @@ -154,7 +184,7 @@ per_cluster_cap = 2 utility_protected = 10 [editorial] -summary_model = "editor" # editor (Claude) | bulk (DeepSeek) +summary_model = "editor" # editor | bulk — which [llm] role writes the summaries summary_input_tokens = 3000 # article text offered per summary [publish] diff --git a/docs/plans/2026-08-15-implementation-notes.md b/docs/plans/2026-08-15-implementation-notes.md index 8f4f669..6f72995 100644 --- a/docs/plans/2026-08-15-implementation-notes.md +++ b/docs/plans/2026-08-15-implementation-notes.md @@ -87,7 +87,23 @@ This file records implementation-time decisions and verified external facts. Fol `stop_reason: "refusal"` on HTTP 200, which the code treats as an error that degrades the call to DeepSeek. Usage fields: `input_tokens` (uncached remainder), `cache_creation_input_tokens`, `cache_read_input_tokens`, `output_tokens`. Timeout 300 s; - retry 429/5xx/network, never 400. Key only from `DAILY_EPUB_ANTHROPIC__API_KEY`. + retry 429/5xx/network, never 400. Key only from `DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY` + (the provider registry below; the pre-registry `DAILY_EPUB_ANTHROPIC__API_KEY` is a startup + error). +- **Gemini 3.8 Flash over the OpenAI-compatible endpoint** (beta, verified 2026-09-02): + `POST https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` with + `Authorization: Bearer `, the standard `messages` / `temperature` / + `response_format: {"type": "json_object"}` body. Model id `gemini-3.8-flash`. Reasoning depth + is the OpenAI `reasoning_effort` field, which Google maps onto Gemini 3.x's `thinking_level` + (`minimal | low | medium | high`; `none` is not accepted by 3.x models). Usage: implicit + cache hits are reported in `prompt_tokens_details.cached_tokens` (the same field DeepSeek + now fills), and `completion_tokens` already includes the thinking tokens that + `completion_tokens_details.reasoning_tokens` breaks out — so output is priced from + `completion_tokens` alone, never the sum. Prices per 1M tokens (promotional through + 2026-12-31): **$0.75 input, $0.075 cache read, $3.75 output** (thinking included); from + 2027-01-01 **$1.50 / $0.15 / $7.50**. No cache-write charge. Key only from + `DAILY_EPUB_PROVIDERS__GEMINI__API_KEY`. Shipped as `[providers.gemini]`, unreferenced until + a role names it. - **Voyage AI embeddings** (verified 2026-09-02): `POST https://api.voyageai.com/v1/embeddings` with `Authorization: Bearer `; body `{input: [...], model: "voyage-4-lite", input_type: "document" | "query", truncation: true, output_dimension: 512, output_dtype: "float"}`. Up @@ -109,13 +125,25 @@ This file records implementation-time decisions and verified external facts. Fol `the-daily-epub/1.0 (personal rss digest; contact tyler@hallada.net)`), passed by clone. 5. **LLM**: a hand-rolled `reqwest` client, not `async-openai` (the published crate exposes neither `Client` nor `CreateChatCompletionRequest` at the pinned version). Every LLM call - goes through `curate/llm.rs`: `LlmClient { system_prompt, model, meter, backend, retry }` - over the `ChatBackend` trait, with `DeepseekBackend` (OpenAI-compatible chat completions, - `response_format: json_object`) and `AnthropicBackend` (Messages API, facts above). The - pipeline holds `Llms { bulk, editor }`; `editor_or_bulk()` degrades to DeepSeek when the - Claude client is missing or its meter is tripped. One `UsageMeter` per provider - (DeepSeek, Anthropic, Voyage) with its own price table and `max_daily_usd`. The system - prompt is sent first and byte-identical within a run so both providers' prefix caches hit. + goes through `curate/llm.rs`: `LlmClient { provider, system_prompt, model, effort, + max_concurrent_requests, meter, backend, retry }` over the `ChatBackend` trait, with two + wire protocols — `OpenAiCompatibleBackend` (`{base_url}/chat/completions`, bearer key, + `response_format: json_object`, `reasoning_effort` when the provider has an `effort`) and + `AnthropicBackend` (Messages API, facts above). **Providers are config, not code**: the + `[providers.]` registry (`kind = openai | anthropic`, `base_url`, `model`, `effort`, + `max_daily_usd`, `max_concurrent_requests`, `price_*`) is a `BTreeMap`, and `[llm] bulk = ""` / `editor = ""` assign the two roles by + name (`editor = ""` means everything runs on bulk; both roles on one provider share one + client and one ceiling). `LlmClient::for_provider(name, &cfg, ..)` dispatches on `kind`; + `Llms::from_config(&config, prompt, &meters)` builds the roles; `editor_or_bulk()` degrades + to bulk when the editor client is missing or its meter is tripped. One `UsageMeter` per + *referenced* provider (`llm::provider_meters`), keyed by provider name — the same key used + for `runs.provider_costs_json`, the UTC-day spend preload and the log lines — plus Voyage's + own. `LlmClient.provider` is the config name, never the kind. The system prompt is sent + first and byte-identical within a run so every provider's prefix cache hits. Keys come only + from `DAILY_EPUB_PROVIDERS____API_KEY` (figment lower-cases the path, so provider + names are `[a-z0-9_]+`); `daily-epub config check` prints the resolved roles without + opening the database. 6. **Testing**: unit tests inline per module; integration tests in `tests/` over fixture JSON in `tests/fixtures/`. Never hit the network in tests: `MockBackend` (`ChatBackend`) and the embedding mock (`EmbeddingBackend`) stand in for all three providers. `--skip-llm` makes @@ -168,6 +196,18 @@ implementer needs that are easy to get wrong: articles older than `embedding_retention_days` (120) and `candidate_runs` rows plus `article_assessments` older than `telemetry_retention_days` (180). `features prune` runs it on demand; `generate` runs it once after publishing, best effort. -- **Keys**: `DAILY_EPUB_ANTHROPIC__API_KEY` and `DAILY_EPUB_VOYAGE__API_KEY` map onto - `AnthropicConfig.api_key` / `VoyageConfig.api_key` through figment; the fields exist only - for that mapping and are never documented in TOML, logged, or stored. +- **Keys**: `DAILY_EPUB_PROVIDERS____API_KEY` and `DAILY_EPUB_VOYAGE__API_KEY` map onto + `ProviderConfig.api_key` / `VoyageConfig.api_key` through figment; the fields exist only + for that mapping and are never documented in TOML, logged, or stored + (`Config::providers_redacted()` is what reaches `runs.config_json`). +- **Provider registry** (2026-09-02, after step 7): the `[deepseek]` and `[anthropic]` tables + and the top-level `max_daily_usd` are gone. `[llm]` holds the role names and the role-level + knobs (`triage_batch_size`, `deep_batch_size`, `score_temperature`, + `editorial_temperature`); `[providers.deepseek]`, `[providers.anthropic]` and + `[providers.gemini]` ship in `config.example.toml` and are `Config::default()` key for key. + Stale shapes fail at load, naming the new key: a `[deepseek]`/`[anthropic]` header, a + top-level `max_daily_usd`, any of the four role keys outside `[llm]`, or a + `DAILY_EPUB_DEEPSEEK__*` / `DAILY_EPUB_ANTHROPIC__*` environment variable. Batching + concurrency (`triage`, `assess`) is the bulk provider's `max_concurrent_requests`; the + summaries fan out at the summary provider's. `Models { bulk, editor, summaries }` in the + colophon and Behind the paper stay model ids taken from the built clients. diff --git a/docs/plans/2026-09-02-curation-v2-progress.md b/docs/plans/2026-09-02-curation-v2-progress.md index 7eacb61..74950ba 100644 --- a/docs/plans/2026-09-02-curation-v2-progress.md +++ b/docs/plans/2026-09-02-curation-v2-progress.md @@ -18,6 +18,13 @@ brief was handed to an implementation agent as `cat 00-preamble.md stepN.md`. | 6. Paper telemetry, stats, lock | **done** (Claude agent), reviewed | `d261cd4` | | 7. Cleanup + implementation notes | **done** (Claude agent), reviewed | `d403c51` | +**Post-plan addition (same day):** a provider-agnostic LLM registry — `[llm]` roles (`bulk`, +`editor`) over named `[providers.*]` entries of `kind = openai | anthropic`, keys from +`DAILY_EPUB_PROVIDERS____API_KEY`, a `gemini` entry (Gemini 3.8 Flash via Google's +OpenAI-compatible endpoint) declared but unreferenced, and a `daily-epub config check` subcommand. +The old `[deepseek]`/`[anthropic]` tables, top-level `max_daily_usd` and the old key env vars +fail loudly. The server upgrade is written up in `docs/runbooks/curation-v2-migration.md`. + All seven plan steps are implemented. `cargo fmt --check`, `cargo clippy --all-targets` (including `-W dead_code`) and `cargo test` (329 lib tests + 6 bin tests + the 7 integration suites) are green at HEAD. @@ -124,7 +131,8 @@ Step 7: ## Next session: what remains -1. `git checkout curation-v2 && cargo test` (expect green). +1. `git checkout curation-v2 && cargo test` (expect green). Follow + `docs/runbooks/curation-v2-migration.md` on the server. 2. Optional: a Codex `review --background --scope branch --base main` pass over the whole branch. 3. `git merge --no-ff curation-v2` into `main`, build, deploy, and do the operator to-dos above. 4. After a week of real runs: read `stats`, tune `[curation.ranking]` from what `explain` shows, diff --git a/docs/runbooks/curation-v2-migration.md b/docs/runbooks/curation-v2-migration.md new file mode 100644 index 0000000..2e5de4c --- /dev/null +++ b/docs/runbooks/curation-v2-migration.md @@ -0,0 +1,278 @@ +# Runbook — upgrading the server from v1 to curation v2 + +**Written:** 2026-09-02, for the `curation-v2` branch (all seven plan steps plus the provider +registry). Every command below is meant to be run on the server, as `root` via `sudo`, unless it +says otherwise. Expect the whole thing to take about an hour, most of it waiting on the embedding +backfill and one dry run. + +What changes for the operator, in one paragraph: the binary is replaced; the SQLite schema gains +tables and drops `ratings`, `feed_priors` and `scores` (the migration copies your ratings first); +`config.toml` loses a few keys and gains the `[llm]` / `[providers.*]` registry plus a +`profile_path`; the env file gains two API keys and renames the DeepSeek one; a hand-maintained +`profile.md` is installed next to the OPML; the systemd units are unchanged. + +## 0. Before touching the server + +On the dev box: + +```sh +git checkout curation-v2 +cargo test # expect green +cargo build --release # or build on the server, as you do today +``` + +The crate uses `edition = "2024"` and let-chains, so the server's toolchain must be current +(`rustup update stable`). Read `data/profile.md` once; it is the reader profile you will be +editing by hand from now on, and the first thing to tune when the paper feels off. + +## 1. Freeze the timer, back everything up + +```sh +sudo systemctl stop daily-epub-generate.timer daily-epub-generate.service +sudo systemctl stop daily-epub.service # the server also runs migrations on start + +sudo install -d -m0750 -o daily-epub -g daily-epub /var/lib/daily-epub/backup +sudo -u daily-epub sqlite3 /var/lib/daily-epub/daily-epub.db \ + ".backup '/var/lib/daily-epub/backup/daily-epub-pre-v2-$(date +%F).db'" +sudo cp -a /etc/daily-epub/config.toml /etc/daily-epub/config.toml.v1 +sudo cp -a /etc/daily-epub/env /etc/daily-epub/env.v1 +sudo cp -a /usr/local/bin/daily-epub /usr/local/bin/daily-epub.v1 +``` + +`.backup` is the safe way to copy a WAL-mode database; a plain `cp` while anything is open can +miss the WAL. The migration is one-way (`ratings` is dropped after being copied), so this backup +is the rollback path. + +## 2. Install the new binary + +```sh +sudo install -m0755 target/release/daily-epub /usr/local/bin/ +daily-epub --version +daily-epub --help # confirm `config`, `stats`, `explain`, `ratings`, `features` exist +``` + +Do **not** start any service yet: every subcommand except `config check` opens the database and +applies pending migrations, and you want the config right first. + +The units in `systemd/` did not change. If you want to be sure your installed copies match: + +```sh +diff systemd/daily-epub-generate.service /etc/systemd/system/daily-epub-generate.service +diff systemd/daily-epub.service /etc/systemd/system/daily-epub.service +``` + +## 3. Edit `config.toml` in place + +Everything you do not mention keeps its documented default, so the edit is small. Open it with +`sudo -e /etc/daily-epub/config.toml` and apply this checklist. + +**Remove** (each one now fails loudly at startup, naming its replacement): + +| Old key | Why | +|---|---| +| `prefilter_keep = …` (top level) | replaced by `curation.ranking.deep_keep` (default 120) | +| `max_daily_usd = …` (top level) | now per provider: `providers.deepseek.max_daily_usd` | +| the whole `[deepseek]` table | becomes `[providers.deepseek]` + `[llm]` (see below) | +| any `[anthropic]` table (only if you added one from an interim build) | becomes `[providers.anthropic]` | + +**Add** near the top, next to `interests_opml`: + +```toml +profile_path = "/var/lib/daily-epub/data/profile.md" +``` + +Use an absolute path. The default is `data/profile.md` *relative to the working directory*, which +under the unit is `/var/lib/daily-epub`, so the default would resolve to the same place, but an +explicit path survives running one-off commands from another directory. Point +`interests_opml` at an absolute path too if it is still relative. + +**Add** the LLM registry. Carry over the `base_url`, `model` and `price_*` values from your old +`[deepseek]` table if you had changed them; the values shown are the defaults. + +```toml +[llm] +bulk = "deepseek" # triage, deep assessment, and the fallback for every editor call +editor = "anthropic" # lineup, summaries, the Brief, the weekly profile rebuild +triage_batch_size = 25 +deep_batch_size = 8 +score_temperature = 0.3 +editorial_temperature = 0.8 + +[providers.deepseek] +kind = "openai" +base_url = "https://api.deepseek.com/v1" +model = "deepseek-v4-flash" +max_daily_usd = 2.0 +max_concurrent_requests = 4 +price_input_per_mtok = 0.14 +price_cache_read_per_mtok = 0.0028 +price_cache_write_per_mtok = 0.0 +price_output_per_mtok = 0.28 + +[providers.anthropic] +kind = "anthropic" +base_url = "https://api.anthropic.com" +model = "claude-opus-5" +effort = "high" +max_daily_usd = 3.0 +max_concurrent_requests = 4 +price_input_per_mtok = 5.0 +price_cache_read_per_mtok = 0.5 +price_cache_write_per_mtok = 6.25 +price_output_per_mtok = 25.0 +``` + +Optional, for the Gemini comparison (section 9): + +```toml +[providers.gemini] +kind = "openai" # Gemini's OpenAI-compatible endpoint +base_url = "https://generativelanguage.googleapis.com/v1beta/openai" +model = "gemini-3.8-flash" +effort = "high" +max_daily_usd = 3.0 +max_concurrent_requests = 4 +price_input_per_mtok = 0.75 # $1.50 from 2027-01-01 +price_cache_read_per_mtok = 0.075 # $0.15 from 2027-01-01 +price_cache_write_per_mtok = 0.0 +price_output_per_mtok = 3.75 # includes thinking tokens; $7.50 from 2027-01-01 +``` + +**Leave alone** `[miniflux]`, `[server]`, `[publish]`, `[xtc]`, `[world]`, `[curation] +sections`, `always_include_feeds`, `blocked_domains`. `[voyage]`, `[editorial]`, +`[curation.feedback]` and `[curation.ranking]` all have sensible defaults; copy a section from +`config.example.toml` only when you want to change a value in it. + +## 4. Edit the env file + +```sh +sudo -e /etc/daily-epub/env +``` + +| Variable | Action | +|---|---| +| `DAILY_EPUB_DEEPSEEK__API_KEY` | **rename** to `DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY` (the old name is rejected at startup so it cannot silently disable the bulk model) | +| `DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY` | add | +| `DAILY_EPUB_VOYAGE__API_KEY` | add | +| `DAILY_EPUB_PROVIDERS__GEMINI__API_KEY` | add only if you configured `[providers.gemini]` | +| `DAILY_EPUB_MINIFLUX__API_KEY`, `DAILY_EPUB_SERVER__HMAC_SECRET` | unchanged | + +Keep it `0600 daily-epub:daily-epub`. Then set hard spend limits in the DeepSeek, Anthropic and +Voyage dashboards: the in-app `max_daily_usd` meters are runaway guards, not accounting. + +## 5. Install the reader profile + +```sh +sudo install -d -m0750 -o daily-epub -g daily-epub /var/lib/daily-epub/data +sudo install -m0640 -o daily-epub -g daily-epub data/profile.md /var/lib/daily-epub/data/profile.md +# if the OPML is not already there: +sudo install -m0640 -o daily-epub -g daily-epub data/scour-interests.opml /var/lib/daily-epub/data/ +``` + +If the file is missing the run does not fail; it logs a warning and uses the OPML interests only, +which is a much worse prompt. `config check` in the next step tells you whether it was found. + +## 6. Check the config as the service user + +The units run as `daily-epub` with the env file loaded, so check the same way. This helper runs +one command in that identity, with the env file and the working directory the unit uses, and no +other hardening: + +```sh +de() { sudo systemd-run --quiet --wait --pty --collect \ + --uid=daily-epub --gid=daily-epub \ + -p WorkingDirectory=/var/lib/daily-epub -p EnvironmentFile=/etc/daily-epub/env \ + /usr/local/bin/daily-epub --config /etc/daily-epub/config.toml "$@"; } + +de config check +``` + +Expected: every line is a fact, no line starts with `!`. Fix anything marked `MISSING` (a key +name, a path) before continuing. A stale key in the TOML is reported as an error naming its +replacement; go back to section 3. + +## 7. Migrate the schema + +```sh +de db migrate +sudo -u daily-epub sqlite3 /var/lib/daily-epub/daily-epub.db '.tables' +# expect rating_events, article_embeddings, interest_embeddings, article_assessments, +# candidate_runs; no ratings, feed_priors or scores +sudo -u daily-epub sqlite3 /var/lib/daily-epub/daily-epub.db \ + "select label, count(*) from rating_events group by label;" +# your old votes: up → loved, down → not_for_me, source = 'migration' +``` + +## 8. Warm the embedding cache, then a dry run + +```sh +de features backfill --rated-only # the learned set; prints a token estimate first +de features backfill --days 30 # recent articles, so day one is not all cache misses +de generate --dry-run --out /var/lib/daily-epub/out-check +``` + +The dry run makes real DeepSeek, Claude and Voyage calls but publishes nothing and writes no +`issues` row. Read the printed lineup and the four report lines (`curation:`, `admission:`, +`preference:`, `providers:`); the cost should be well under $1. Then read the EPUB it wrote +(Calibre or KOReader): The Brief, the `Why it's here` line under each headline, and the new +Behind-the-paper chapter before the colophon. Finally: + +```sh +de explain --date "$(date +%F)" --near-misses +``` + +Remove `/var/lib/daily-epub/out-check` when done. + +## 9. Go live + +```sh +sudo systemctl daemon-reload +sudo systemctl start daily-epub.service +sudo systemctl enable --now daily-epub-generate.timer +sudo systemctl list-timers daily-epub-generate.timer +``` + +If you want today's paper regenerated by the new pipeline now rather than tomorrow at 05:30: +`sudo systemctl start daily-epub-generate`. A same-date rerun replaces today's issue and is a +new run id in telemetry. + +Afterwards: + +```sh +journalctl -u daily-epub-generate -n 60 --no-pager # the four-line info block near the end +de stats --days 14 +``` + +## 10. Comparing editors (Claude Opus 5 vs Gemini 3.8 Flash) + +With `[providers.gemini]` and its key in place, an A/B needs no config edit: environment variables +override the TOML, so + +```sh +sudo systemd-run --quiet --wait --pty --collect --uid=daily-epub --gid=daily-epub \ + -p WorkingDirectory=/var/lib/daily-epub -p EnvironmentFile=/etc/daily-epub/env \ + -E DAILY_EPUB_LLM__EDITOR=gemini \ + /usr/local/bin/daily-epub --config /etc/daily-epub/config.toml \ + generate --dry-run --date "$(date +%F)" --out /var/lib/daily-epub/out-gemini +``` + +produces the same date's paper with Gemini as the editor. Triage and deep assessments are cached +for three days, so the second run costs only the editor, summaries and the Brief. Compare the two +lineups, the `why` lines and the Brief side by side, and the `providers:` cost line. To switch +for good, set `editor = "gemini"` in `[llm]` (and `summary_model` stays `editor`, so summaries +move with it). The same trick works for the bulk role: `DAILY_EPUB_LLM__BULK=gemini`. + +## Rollback + +```sh +sudo systemctl stop daily-epub-generate.timer daily-epub.service +sudo install -m0755 /usr/local/bin/daily-epub.v1 /usr/local/bin/daily-epub +sudo cp -a /etc/daily-epub/config.toml.v1 /etc/daily-epub/config.toml +sudo cp -a /etc/daily-epub/env.v1 /etc/daily-epub/env +sudo -u daily-epub cp /var/lib/daily-epub/backup/daily-epub-pre-v2-.db /var/lib/daily-epub/daily-epub.db +sudo rm -f /var/lib/daily-epub/daily-epub.db-wal /var/lib/daily-epub/daily-epub.db-shm +sudo systemctl start daily-epub.service daily-epub-generate.timer +``` + +The database restore is required, not optional: the v1 binary expects `ratings`, `feed_priors` +and `scores`, which v2's migrations drop. diff --git a/src/config.rs b/src/config.rs index 344f709..a2f80d8 100644 --- a/src/config.rs +++ b/src/config.rs @@ -4,6 +4,7 @@ //! else `./config.toml` if present) ← `DAILY_EPUB_*` environment variables, where //! nesting is expressed with a double underscore (`DAILY_EPUB_MINIFLUX__API_KEY`). +use std::collections::BTreeMap; use std::path::{Path, PathBuf}; use figment::Figment; @@ -57,9 +58,6 @@ pub struct Config { /// Counted, not dated, because an XTCH issue is ~80–100 MB of pre-rendered /// page bitmaps: the constraint is disk, not age. pub xtc_retention_count: u32, - /// DeepSeek spend ceiling per UTC day (§5); `[anthropic]` and `[voyage]` - /// carry their own. - pub max_daily_usd: f64, /// Include the Wikipedia Current Events section (§3.8). pub world_briefing: bool, @@ -73,8 +71,11 @@ pub struct Config { pub profile_path: PathBuf, pub miniflux: MinifluxConfig, - pub deepseek: DeepseekConfig, - pub anthropic: AnthropicConfig, + /// Which named provider plays each LLM role, plus the role-level knobs. + pub llm: LlmConfig, + /// The provider registry: `[providers.]`, referenced by name from + /// `[llm]`. Keys arrive only through `DAILY_EPUB_PROVIDERS____API_KEY`. + pub providers: BTreeMap, pub voyage: VoyageConfig, pub curation: CurationConfig, pub editorial: EditorialConfig, @@ -91,15 +92,14 @@ impl Default for Config { target_article_count: 20, retention_days: 21, xtc_retention_count: 5, - max_daily_usd: 2.0, world_briefing: true, database_path: PathBuf::from("/var/lib/daily-epub/daily-epub.db"), out_dir: PathBuf::from("/var/lib/daily-epub/out"), interests_opml: PathBuf::from("data/scour-interests.opml"), profile_path: PathBuf::from("data/profile.md"), miniflux: MinifluxConfig::default(), - deepseek: DeepseekConfig::default(), - anthropic: AnthropicConfig::default(), + llm: LlmConfig::default(), + providers: default_providers(), voyage: VoyageConfig::default(), curation: CurationConfig::default(), editorial: EditorialConfig::default(), @@ -131,81 +131,211 @@ impl Default for MinifluxConfig { } } -/// `[deepseek]` — bulk LLM endpoint, model and pricing (§4.1, notes "verified facts"). +/// `[llm]` — the role assignments and the role-level knobs (§4). +/// +/// `bulk` runs triage, deep assessment and every fallback; `editor` runs the +/// lineup, summaries, the Brief and the profile rebuild. Both name an entry of +/// `[providers.*]`; an empty `editor` means "everything runs on bulk". #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[serde(deny_unknown_fields, default)] -pub struct DeepseekConfig { - pub base_url: String, - pub model: String, - /// Supply via `DAILY_EPUB_DEEPSEEK__API_KEY`. - pub api_key: Option, - /// Articles per deep-assessment request (§12.1). - pub deep_batch_size: usize, +pub struct LlmConfig { + pub bulk: String, + pub editor: String, /// Articles per first-pass triage request (§10). pub triage_batch_size: usize, - pub max_concurrent_requests: usize, + /// Articles per close-reading assessment request (§12.1). + pub deep_batch_size: usize, + /// Sent only by providers that accept a temperature (`kind = "openai"`). pub score_temperature: f32, pub editorial_temperature: f32, +} + +impl Default for LlmConfig { + fn default() -> Self { + Self { + bulk: "deepseek".into(), + editor: "anthropic".into(), + triage_batch_size: 25, + deep_batch_size: 8, + score_temperature: 0.3, + editorial_temperature: 0.8, + } + } +} + +impl LlmConfig { + /// The bulk provider's name, `None` when `bulk = ""`. + pub fn bulk_name(&self) -> Option<&str> { + Some(self.bulk.trim()).filter(|name| !name.is_empty()) + } + + /// The editor provider's name, `None` when `editor` is empty or absent. + pub fn editor_name(&self) -> Option<&str> { + Some(self.editor.trim()).filter(|name| !name.is_empty()) + } + + /// `(role, provider name)` for every assigned role, bulk first. + pub fn roles(&self) -> Vec<(&'static str, &str)> { + let mut roles = Vec::with_capacity(2); + if let Some(name) = self.bulk_name() { + roles.push(("bulk", name)); + } + if let Some(name) = self.editor_name() { + roles.push(("editor", name)); + } + roles + } +} + +/// The wire protocol a provider speaks (`[providers.] kind`). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum ProviderKind { + /// OpenAI-compatible `POST {base_url}/chat/completions` (DeepSeek, Gemini's + /// compatibility endpoint, OpenAI itself). + OpenAi, + /// The Anthropic Messages API. + Anthropic, +} + +impl ProviderKind { + pub fn as_str(self) -> &'static str { + match self { + ProviderKind::OpenAi => "openai", + ProviderKind::Anthropic => "anthropic", + } + } +} + +/// `[providers.]` — one chat-completion provider: endpoint, model, +/// reasoning effort, its own daily ceiling and its price table (§4, §5). +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(deny_unknown_fields, default)] +pub struct ProviderConfig { + pub kind: ProviderKind, + pub base_url: String, + pub model: String, + /// Supply only via `DAILY_EPUB_PROVIDERS____API_KEY`. + pub api_key: Option, + /// `kind = "anthropic"`: `output_config.effort` (`low | medium | high | + /// xhigh | max`). `kind = "openai"`: passed through as `reasoning_effort`. + pub effort: Option, + /// Spend ceiling per UTC day; `0` disables the guard. + pub max_daily_usd: f64, + pub max_concurrent_requests: usize, /// USD per 1M cache-miss input tokens. pub price_input_per_mtok: f64, - /// USD per 1M prefix-cache-hit input tokens. - pub price_cached_input_per_mtok: f64, - /// USD per 1M output tokens. + /// USD per 1M cache-hit input tokens. + pub price_cache_read_per_mtok: f64, + /// USD per 1M tokens written to the prompt cache (0 where caching is implicit). + pub price_cache_write_per_mtok: f64, + /// USD per 1M output tokens (thinking tokens included where billed as output). pub price_output_per_mtok: f64, } -impl Default for DeepseekConfig { +impl Default for ProviderConfig { fn default() -> Self { Self { + kind: ProviderKind::OpenAi, + base_url: String::new(), + model: String::new(), + api_key: None, + effort: None, + max_daily_usd: 0.0, + max_concurrent_requests: 4, + price_input_per_mtok: 0.0, + price_cache_read_per_mtok: 0.0, + price_cache_write_per_mtok: 0.0, + price_output_per_mtok: 0.0, + } + } +} + +impl ProviderConfig { + /// DeepSeek V4 Flash over its OpenAI-compatible endpoint (verified 2026-08-15). + pub fn deepseek() -> Self { + Self { + kind: ProviderKind::OpenAi, base_url: "https://api.deepseek.com/v1".into(), model: "deepseek-v4-flash".into(), api_key: None, - deep_batch_size: 8, - triage_batch_size: 25, + effort: None, + max_daily_usd: 2.0, max_concurrent_requests: 4, - score_temperature: 0.3, - editorial_temperature: 0.8, price_input_per_mtok: 0.14, - price_cached_input_per_mtok: 0.0028, + price_cache_read_per_mtok: 0.0028, + price_cache_write_per_mtok: 0.0, price_output_per_mtok: 0.28, } } -} -/// `[anthropic]` — Claude editor, editorial and profile settings (§4.2). -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -#[serde(deny_unknown_fields, default)] -pub struct AnthropicConfig { - pub enabled: bool, - pub base_url: String, - pub model: String, - /// Supply only via `DAILY_EPUB_ANTHROPIC__API_KEY`. - pub api_key: Option, - pub effort: String, - pub price_input_per_mtok: f64, - pub price_cache_write_per_mtok: f64, - pub price_cache_read_per_mtok: f64, - pub price_output_per_mtok: f64, - pub max_daily_usd: f64, - pub max_concurrent_requests: usize, -} - -impl Default for AnthropicConfig { - fn default() -> Self { + /// Claude Opus 5 over the Messages API (verified 2026-09-02). + pub fn anthropic() -> Self { Self { - enabled: true, + kind: ProviderKind::Anthropic, base_url: "https://api.anthropic.com".into(), model: "claude-opus-5".into(), api_key: None, - effort: "high".into(), - price_input_per_mtok: 5.0, - price_cache_write_per_mtok: 6.25, - price_cache_read_per_mtok: 0.5, - price_output_per_mtok: 25.0, + effort: Some("high".into()), max_daily_usd: 3.0, max_concurrent_requests: 4, + price_input_per_mtok: 5.0, + price_cache_read_per_mtok: 0.5, + price_cache_write_per_mtok: 6.25, + price_output_per_mtok: 25.0, } } + + /// Gemini 3.8 Flash over Google's OpenAI-compatible endpoint (verified + /// 2026-09-02; promotional prices through 2026-12-31). + pub fn gemini() -> Self { + Self { + kind: ProviderKind::OpenAi, + base_url: "https://generativelanguage.googleapis.com/v1beta/openai".into(), + model: "gemini-3.8-flash".into(), + api_key: None, + effort: Some("high".into()), + max_daily_usd: 3.0, + max_concurrent_requests: 4, + price_input_per_mtok: 0.75, + price_cache_read_per_mtok: 0.075, + price_cache_write_per_mtok: 0.0, + price_output_per_mtok: 3.75, + } + } + + /// The only place a key may come from: `DAILY_EPUB_PROVIDERS____API_KEY`. + pub fn api_key_env_var(name: &str) -> String { + format!( + "{ENV_PREFIX}PROVIDERS{ENV_SPLIT}{}{ENV_SPLIT}API_KEY", + name.to_uppercase() + ) + } + + /// The key, trimmed, when one is configured. + pub fn api_key(&self) -> Option<&str> { + self.api_key + .as_deref() + .map(str::trim) + .filter(|key| !key.is_empty()) + } + + /// A copy safe to log or persist: the key is stripped. + pub fn redacted(&self) -> Self { + Self { + api_key: None, + ..self.clone() + } + } +} + +/// The three providers `config.example.toml` documents. +pub fn default_providers() -> BTreeMap { + BTreeMap::from([ + ("deepseek".to_string(), ProviderConfig::deepseek()), + ("anthropic".to_string(), ProviderConfig::anthropic()), + ("gemini".to_string(), ProviderConfig::gemini()), + ]) } /// Which provider writes per-article summaries (§14.1). @@ -584,6 +714,18 @@ impl Default for ServerConfig { } } +/// Config keys that moved from `[deepseek]` to `[llm]`; anywhere else they are +/// a stale-configuration error. +const LLM_ROLE_KEYS: &[&str] = &[ + "deep_batch_size", + "triage_batch_size", + "score_temperature", + "editorial_temperature", +]; + +/// The pre-registry provider tables; each is now `[providers.]`. +const STALE_PROVIDER_TABLES: &[&str] = &["deepseek", "anthropic"]; + impl Config { /// Build the figment layer stack. `path` is required to exist when explicit. fn figment(path: Option<&Path>, require_file: bool) -> Result { @@ -599,14 +741,22 @@ impl Config { Ok(fig.merge(Env::prefixed(ENV_PREFIX).split(ENV_SPLIT))) } + /// The file `load` reads: the explicit `--config` path, else `./config.toml` + /// when it exists, else `None` (built-in defaults plus the environment). + pub fn resolve_path(explicit: Option<&Path>) -> Option { + match explicit { + Some(path) => Some(path.to_path_buf()), + None => Some(PathBuf::from(DEFAULT_CONFIG_FILE)).filter(|path| path.exists()), + } + } + /// Load config for the CLI: explicit `--config` path, else `./config.toml` /// when it exists, then `DAILY_EPUB_*` env overrides (§3.14). pub fn load(explicit: Option<&Path>) -> Result { - if std::env::var_os("DAILY_EPUB_DEEPSEEK__SCORE_BATCH_SIZE").is_some() { - return Err(ConfigError::Invalid( - "DAILY_EPUB_DEEPSEEK__SCORE_BATCH_SIZE was removed; use DAILY_EPUB_DEEPSEEK__DEEP_BATCH_SIZE" - .into(), - )); + if let Some(message) = + stale_env_error(std::env::vars_os().filter_map(|(key, _)| key.into_string().ok())) + { + return Err(ConfigError::Invalid(message)); } let (path, require) = match explicit { Some(p) => (Some(p.to_path_buf()), true), @@ -616,27 +766,8 @@ impl Config { let raw = std::fs::read_to_string(path).map_err(|error| { ConfigError::Invalid(format!("could not inspect {}: {error}", path.display())) })?; - if raw.lines().any(|line| { - let line = line.trim_start(); - !line.starts_with('#') - && line - .strip_prefix("prefilter_keep") - .is_some_and(|tail| tail.trim_start().starts_with('=')) - }) { - return Err(ConfigError::Invalid( - "prefilter_keep was removed; use curation.ranking.deep_keep".into(), - )); - } - if raw.lines().any(|line| { - let line = line.trim_start(); - !line.starts_with('#') - && line - .strip_prefix("score_batch_size") - .is_some_and(|tail| tail.trim_start().starts_with('=')) - }) { - return Err(ConfigError::Invalid( - "deepseek.score_batch_size was removed; use deepseek.deep_batch_size".into(), - )); + if let Some(message) = stale_toml_error(&raw) { + return Err(ConfigError::Invalid(message)); } } let mut config: Config = Self::figment(path.as_deref(), require)?.extract()?; @@ -651,6 +782,153 @@ impl Config { Ok(config) } + /// The provider an `[llm]` role names, with its name. + fn role_provider<'a>(&'a self, name: Option<&'a str>) -> Option<(&'a str, &'a ProviderConfig)> { + let name = name?; + self.providers.get(name).map(|provider| (name, provider)) + } + + /// `(name, provider)` for `llm.bulk`, `None` when no bulk provider is set. + pub fn bulk_provider(&self) -> Option<(&str, &ProviderConfig)> { + self.role_provider(self.llm.bulk_name()) + } + + /// `(name, provider)` for `llm.editor`, `None` when there is no editor. + pub fn editor_provider(&self) -> Option<(&str, &ProviderConfig)> { + self.role_provider(self.llm.editor_name()) + } + + /// Every provider some role references, bulk first, each once. + pub fn referenced_providers(&self) -> Vec<(&str, &ProviderConfig)> { + let mut seen = Vec::new(); + for (_, name) in self.llm.roles() { + if seen.iter().any(|(seen, _)| *seen == name) { + continue; + } + if let Some(provider) = self.providers.get(name) { + seen.push((name, provider)); + } + } + seen + } + + /// The registry with every key stripped, for logs and `runs.config_json`. + pub fn providers_redacted(&self) -> BTreeMap { + self.providers + .iter() + .map(|(name, provider)| (name.clone(), provider.redacted())) + .collect() + } + + /// `config check`: one fact per line about the resolved configuration. + /// + /// Lines that need the operator's attention (a missing key or file) start + /// with `! `; nothing here opens the database, takes the lock or touches + /// the network, and no key is ever printed. + pub fn check_report(&self, path: Option<&Path>) -> Vec { + fn exists(path: &Path) -> &'static str { + if path.exists() { "exists" } else { "MISSING" } + } + fn file_line(label: &str, path: &Path) -> String { + let prefix = if path.exists() { "" } else { "! " }; + format!("{prefix}{label}: {} ({})", path.display(), exists(path)) + } + fn provider_line(label: &str, name: &str, provider: &ProviderConfig) -> String { + let key = if provider.api_key().is_some() { + "key present".to_string() + } else { + format!( + "key MISSING (set {})", + ProviderConfig::api_key_env_var(name) + ) + }; + let prefix = if provider.api_key().is_some() { + "" + } else { + "! " + }; + format!( + "{prefix}{label}: {name} · {} · {} · effort {} · max_daily_usd ${:.2} · {key}", + provider.kind.as_str(), + provider.model, + provider.effort.as_deref().unwrap_or("-"), + provider.max_daily_usd, + ) + } + + let mut lines = Vec::new(); + lines.push(match path { + Some(path) => format!("config: {}", path.display()), + None => "config: built-in defaults (no config.toml; environment only)".into(), + }); + lines.push(file_line("database_path", &self.database_path)); + lines.push(file_line("profile_path", &self.profile_path)); + lines.push(file_line("interests_opml", &self.interests_opml)); + for (role, name) in self.llm.roles() { + match self.providers.get(name) { + Some(provider) => lines.push(provider_line(&format!("llm.{role}"), name, provider)), + None => lines.push(format!("! llm.{role}: {name} is not a [providers.*] entry")), + } + } + if self.llm.bulk_name().is_none() { + lines.push("! llm.bulk: none (triage and deep assessment are skipped)".into()); + } + if self.llm.editor_name().is_none() { + lines.push("llm.editor: none (editor work runs on the bulk provider)".into()); + } + let referenced = self.referenced_providers(); + for (name, provider) in &self.providers { + if referenced.iter().any(|(used, _)| used == name) { + continue; + } + let key = if provider.api_key().is_some() { + "key present" + } else { + "key absent" + }; + lines.push(format!( + "providers.{name}: unreferenced · {} · {} · {key}", + provider.kind.as_str(), + provider.model + )); + } + let voyage_key = self + .voyage + .api_key + .as_deref() + .is_some_and(|key| !key.trim().is_empty()); + lines.push(format!( + "{}voyage: {} · {} · max_daily_usd ${:.2} · {}", + if voyage_key || !self.voyage.enabled { + "" + } else { + "! " + }, + self.voyage.model, + if self.voyage.enabled { + "enabled" + } else { + "disabled" + }, + self.voyage.max_daily_usd, + if voyage_key { + "key present".to_string() + } else { + format!("key MISSING (set {ENV_PREFIX}VOYAGE{ENV_SPLIT}API_KEY)") + } + )); + lines.push(format!( + "editorial.summary_model: {}", + match self.editorial.summary_model { + SummaryModel::Editor => "editor", + SummaryModel::Bulk => "bulk", + } + )); + lines.push(file_line("publish.epub_dir", &self.publish.epub_dir)); + lines.push(file_line("publish.xtc_dir", &self.publish.xtc_dir)); + lines + } + /// Cheap sanity checks so misconfiguration fails at startup, not mid-run. pub fn validate(&self) -> Result<(), ConfigError> { if self.lookback_hours == 0 { @@ -666,24 +944,14 @@ impl Config { "curation.max_article_count must be >= target_article_count".into(), )); } - if self.deepseek.deep_batch_size == 0 { + if self.llm.deep_batch_size == 0 { return Err(ConfigError::Invalid( - "deepseek.deep_batch_size must be >= 1".into(), + "llm.deep_batch_size must be >= 1".into(), )); } - if self.deepseek.triage_batch_size == 0 { + if self.llm.triage_batch_size == 0 { return Err(ConfigError::Invalid( - "deepseek.triage_batch_size must be >= 1".into(), - )); - } - if self.deepseek.max_concurrent_requests == 0 { - return Err(ConfigError::Invalid( - "deepseek.max_concurrent_requests must be >= 1".into(), - )); - } - if self.anthropic.max_concurrent_requests == 0 { - return Err(ConfigError::Invalid( - "anthropic.max_concurrent_requests must be >= 1".into(), + "llm.triage_batch_size must be >= 1".into(), )); } if self.editorial.summary_input_tokens == 0 { @@ -691,13 +959,15 @@ impl Config { "editorial.summary_input_tokens must be >= 1".into(), )); } - if !matches!( - self.anthropic.effort.as_str(), - "low" | "medium" | "high" | "xhigh" | "max" - ) { - return Err(ConfigError::Invalid( - "anthropic.effort must be one of low, medium, high, xhigh, max".into(), - )); + for (role, name) in self.llm.roles() { + if !self.providers.contains_key(name) { + return Err(ConfigError::Invalid(format!( + "llm.{role} = {name:?} names no [providers.{name}] entry" + ))); + } + } + for (name, provider) in &self.providers { + provider.validate(name)?; } let ranking = &self.curation.ranking; if ranking.deep_keep < ranking.shortlist_keep @@ -779,6 +1049,160 @@ impl Config { } } +impl ProviderConfig { + /// Field-level checks for one `[providers.]` entry. + fn validate(&self, name: &str) -> Result<(), ConfigError> { + let invalid = |message: String| ConfigError::Invalid(message); + if name.is_empty() + || !name + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '_') + { + return Err(invalid(format!( + "provider name {name:?} must be lowercase ascii letters, digits or '_' \ + so that {} can reach it", + ProviderConfig::api_key_env_var(name) + ))); + } + if self.base_url.trim().is_empty() { + return Err(invalid(format!( + "providers.{name}.base_url must not be empty" + ))); + } + if self.model.trim().is_empty() { + return Err(invalid(format!("providers.{name}.model must not be empty"))); + } + if self.max_concurrent_requests == 0 { + return Err(invalid(format!( + "providers.{name}.max_concurrent_requests must be >= 1" + ))); + } + if !self.max_daily_usd.is_finite() || self.max_daily_usd < 0.0 { + return Err(invalid(format!( + "providers.{name}.max_daily_usd must be >= 0" + ))); + } + for (key, price) in [ + ("price_input_per_mtok", self.price_input_per_mtok), + ("price_cache_read_per_mtok", self.price_cache_read_per_mtok), + ( + "price_cache_write_per_mtok", + self.price_cache_write_per_mtok, + ), + ("price_output_per_mtok", self.price_output_per_mtok), + ] { + if !price.is_finite() || price < 0.0 { + return Err(invalid(format!("providers.{name}.{key} must be >= 0"))); + } + } + match (self.kind, self.effort.as_deref().map(str::trim)) { + (ProviderKind::Anthropic, Some(effort)) + if !matches!(effort, "low" | "medium" | "high" | "xhigh" | "max") => + { + return Err(invalid(format!( + "providers.{name}.effort must be one of low, medium, high, xhigh, max" + ))); + } + (ProviderKind::OpenAi, Some("")) => { + return Err(invalid(format!( + "providers.{name}.effort must be a reasoning_effort value or absent" + ))); + } + _ => {} + } + Ok(()) + } +} + +/// The stale-configuration checks over the raw TOML text, following the +/// `prefilter_keep` precedent: silently ignoring a `[deepseek]` table would +/// leave the bulk provider unconfigured and publish a heuristic paper. +fn stale_toml_error(raw: &str) -> Option { + let mut section: Option = None; + for line in raw.lines() { + let line = line.trim_start(); + if line.starts_with('#') || line.is_empty() { + continue; + } + if let Some(header) = line.strip_prefix('[') { + let name = header + .trim_start_matches('[') + .split(']') + .next() + .unwrap_or_default() + .trim() + .to_string(); + if STALE_PROVIDER_TABLES.contains(&name.as_str()) { + return Some(format!( + "the [{name}] table was replaced by [providers.{name}] (kind, base_url, \ + model, effort, max_daily_usd, max_concurrent_requests, price_*) and \ + [llm] (bulk, editor, triage_batch_size, deep_batch_size, \ + score_temperature, editorial_temperature); keys move to \ + {}", + ProviderConfig::api_key_env_var(&name) + )); + } + section = Some(name); + continue; + } + let Some((key, _)) = line.split_once('=') else { + continue; + }; + let key = key.trim(); + if key == "prefilter_keep" { + return Some("prefilter_keep was removed; use curation.ranking.deep_keep".into()); + } + if key == "score_batch_size" { + return Some("deepseek.score_batch_size was removed; use llm.deep_batch_size".into()); + } + if key == "max_daily_usd" && section.is_none() { + return Some( + "the top-level max_daily_usd was removed; every provider carries its own \ + providers..max_daily_usd" + .into(), + ); + } + if LLM_ROLE_KEYS.contains(&key) && section.as_deref() != Some("llm") { + return Some(format!( + "{key} moved to the [llm] table (it was {}.{key})", + section.as_deref().unwrap_or("top-level") + )); + } + } + None +} + +/// A `DAILY_EPUB_DEEPSEEK__*` or `DAILY_EPUB_ANTHROPIC__*` variable in the +/// environment, naming the variable that replaced it. A silently unavailable +/// bulk provider would publish a heuristic paper, so this fails config load. +fn stale_env_error(vars: impl IntoIterator) -> Option { + for var in vars { + let Some(rest) = var.strip_prefix(ENV_PREFIX) else { + continue; + }; + for table in STALE_PROVIDER_TABLES { + let Some(key) = rest.strip_prefix(&format!("{}{ENV_SPLIT}", table.to_uppercase())) + else { + continue; + }; + let replacement = if key == "SCORE_BATCH_SIZE" { + format!("{ENV_PREFIX}LLM{ENV_SPLIT}DEEP_BATCH_SIZE") + } else if LLM_ROLE_KEYS.contains(&key.to_lowercase().as_str()) { + format!("{ENV_PREFIX}LLM{ENV_SPLIT}{key}") + } else { + format!( + "{ENV_PREFIX}PROVIDERS{ENV_SPLIT}{}{ENV_SPLIT}{key}", + table.to_uppercase() + ) + }; + return Some(format!( + "{var} is stale: the [{table}] table became [providers.{table}]; set {replacement}" + )); + } + } + None +} + #[cfg(test)] mod tests { use super::*; @@ -792,11 +1216,16 @@ mod tests { assert_eq!(c.target_article_count, 20); assert_eq!(c.curation.ranking.deep_keep, 120); assert_eq!(c.retention_days, 21); - assert_eq!(c.max_daily_usd, 2.0); assert!(c.world_briefing); - assert_eq!(c.deepseek.model, "deepseek-v4-flash"); - assert_eq!(c.deepseek.triage_batch_size, 25); - assert_eq!(c.deepseek.deep_batch_size, 8); + assert_eq!(c.llm.bulk, "deepseek"); + assert_eq!(c.llm.editor, "anthropic"); + assert_eq!(c.llm.triage_batch_size, 25); + assert_eq!(c.llm.deep_batch_size, 8); + assert_eq!(c.providers["deepseek"].model, "deepseek-v4-flash"); + assert_eq!(c.providers["deepseek"].max_daily_usd, 2.0); + assert_eq!(c.providers["anthropic"].kind, ProviderKind::Anthropic); + assert_eq!(c.providers["gemini"].kind, ProviderKind::OpenAi); + assert_eq!(c.providers.len(), 3); assert_eq!(c.curation.recent_rejection_days, 7); assert_eq!(c.curation.recent_rejection_floor, 3.0); assert_eq!(c.profile_path, PathBuf::from("data/profile.md")); @@ -835,10 +1264,21 @@ mod tests { jail.set_env("DAILY_EPUB_SERVER__HMAC_SECRET", "hunter2"); jail.set_env("DAILY_EPUB_VOYAGE__API_KEY", "voyage-key"); jail.set_env("DAILY_EPUB_VOYAGE__ENABLED", "false"); + jail.set_env("DAILY_EPUB_PROVIDERS__GEMINI__API_KEY", "gemini-key"); + jail.set_env("DAILY_EPUB_LLM__EDITOR", "gemini"); let c = Config::load(None).map_err(|e| figment::Error::from(e.to_string()))?; assert_eq!(c.voyage.api_key.as_deref(), Some("voyage-key")); assert!(!c.voyage.enabled); + // The registry is reachable through the same double-underscore path. + assert_eq!(c.providers["gemini"].api_key.as_deref(), Some("gemini-key")); + assert!(c.providers["deepseek"].api_key.is_none()); + assert_eq!(c.llm.editor, "gemini"); + assert_eq!( + c.editor_provider() + .map(|(name, p)| (name, p.model.as_str())), + Some(("gemini", "gemini-3.8-flash")) + ); // from file assert_eq!(c.lookback_hours, 30); assert!(!c.world_briefing); @@ -905,6 +1345,223 @@ mod tests { assert!(error.to_string().contains("deep_batch_size"), "{error}"); } + /// The pre-registry `[deepseek]` / `[anthropic]` tables and the role keys + /// that lived in `[deepseek]` fail loudly, naming their new homes. + #[test] + fn stale_provider_tables_and_role_keys_fail_loudly() { + let dir = tempfile::tempdir().unwrap(); + for (body, needles) in [ + ( + "[deepseek]\nmodel = \"deepseek-v4-flash\"\n", + vec![ + "[providers.deepseek]", + "[llm]", + "DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY", + ], + ), + ( + "[anthropic]\nenabled = true\n", + vec!["[providers.anthropic]", "[llm]"], + ), + ( + "[providers.deepseek]\ntriage_batch_size = 25\n", + vec!["triage_batch_size", "[llm]"], + ), + ("editorial_temperature = 0.8\n", vec!["[llm]"]), + ( + "max_daily_usd = 2.0\n", + vec!["providers..max_daily_usd"], + ), + ] { + let path = dir.path().join("config.toml"); + std::fs::write(&path, body).unwrap(); + let error = Config::load(Some(&path)).expect_err(body); + let message = error.to_string(); + for needle in needles { + assert!(message.contains(needle), "{body}: {message}"); + } + } + // The same keys inside `[llm]`, and a provider's own ceiling, are fine. + let path = dir.path().join("config.toml"); + std::fs::write( + &path, + "[llm]\ntriage_batch_size = 10\n\n[providers.deepseek]\nmax_daily_usd = 1.5\n", + ) + .unwrap(); + let c = Config::load(Some(&path)).expect("valid registry config"); + assert_eq!(c.llm.triage_batch_size, 10); + assert_eq!(c.providers["deepseek"].max_daily_usd, 1.5); + assert!(stale_toml_error("# [deepseek] in a comment\n").is_none()); + } + + #[test] + fn stale_env_vars_name_their_replacement() { + let error = + stale_env_error(["DAILY_EPUB_DEEPSEEK__API_KEY".to_string()]).expect("stale key var"); + assert!( + error.contains("DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY"), + "{error}" + ); + let error = + stale_env_error(["DAILY_EPUB_ANTHROPIC__API_KEY".to_string()]).expect("stale key var"); + assert!( + error.contains("DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY"), + "{error}" + ); + let error = stale_env_error(["DAILY_EPUB_DEEPSEEK__SCORE_BATCH_SIZE".to_string()]) + .expect("stale batch var"); + assert!(error.contains("DAILY_EPUB_LLM__DEEP_BATCH_SIZE"), "{error}"); + let error = stale_env_error(["DAILY_EPUB_DEEPSEEK__TRIAGE_BATCH_SIZE".to_string()]) + .expect("stale role var"); + assert!( + error.contains("DAILY_EPUB_LLM__TRIAGE_BATCH_SIZE"), + "{error}" + ); + assert!( + stale_env_error([ + "DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY".to_string(), + "DAILY_EPUB_VOYAGE__API_KEY".to_string(), + "PATH".to_string(), + ]) + .is_none() + ); + } + + #[test] + fn registry_validation_rejects_bad_roles_kinds_and_efforts() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("config.toml"); + + std::fs::write(&path, "[providers.mistral]\nkind = \"mistral\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("unknown kind"); + assert!(error.to_string().contains("mistral"), "{error}"); + + std::fs::write(&path, "[llm]\neditor = \"gemini2\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("undefined provider"); + assert!(error.to_string().contains("providers.gemini2"), "{error}"); + + std::fs::write(&path, "[llm]\nbulk = \"nope\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("undefined bulk provider"); + assert!(error.to_string().contains("llm.bulk"), "{error}"); + + std::fs::write(&path, "[llm]\neditor = \"\"\n").unwrap(); + let c = Config::load(Some(&path)).expect("no editor is allowed"); + assert!(c.editor_provider().is_none()); + assert_eq!(c.llm.roles(), vec![("bulk", "deepseek")]); + assert_eq!(c.referenced_providers().len(), 1); + + std::fs::write(&path, "[llm]\nbulk = \"\"\neditor = \"\"\n").unwrap(); + let c = Config::load(Some(&path)).expect("no providers at all is allowed"); + assert!(c.bulk_provider().is_none()); + assert!(c.referenced_providers().is_empty()); + + std::fs::write( + &path, + "[llm]\nbulk = \"anthropic\"\neditor = \"anthropic\"\n", + ) + .unwrap(); + let c = Config::load(Some(&path)).expect("one provider for both roles"); + assert_eq!(c.referenced_providers().len(), 1); + + std::fs::write(&path, "[providers.anthropic]\neffort = \"turbo\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("anthropic effort is an enum"); + assert!(error.to_string().contains("effort"), "{error}"); + for effort in ["low", "medium", "high", "xhigh", "max"] { + std::fs::write( + &path, + format!("[providers.anthropic]\neffort = \"{effort}\"\n"), + ) + .unwrap(); + assert!(Config::load(Some(&path)).is_ok(), "{effort}"); + } + std::fs::write(&path, "[providers.gemini]\neffort = \"minimal\"\n").unwrap(); + assert!( + Config::load(Some(&path)).is_ok(), + "openai effort is free-form" + ); + + std::fs::write(&path, "[providers.gemini]\nmodel = \"\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("empty model"); + assert!( + error.to_string().contains("providers.gemini.model"), + "{error}" + ); + std::fs::write(&path, "[providers.gemini]\nprice_output_per_mtok = -1\n").unwrap(); + assert!(Config::load(Some(&path)).is_err(), "negative price"); + std::fs::write(&path, "[providers.gemini]\nmax_concurrent_requests = 0\n").unwrap(); + assert!(Config::load(Some(&path)).is_err(), "zero concurrency"); + std::fs::write(&path, "[providers.gemini]\nnot_a_key = 1\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("unknown provider key"); + assert!(error.to_string().contains("not_a_key"), "{error}"); + std::fs::write(&path, "[providers.Gemini]\nmodel = \"x\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("upper-case name"); + assert!(error.to_string().contains("lowercase"), "{error}"); + std::fs::write(&path, "[providers.local]\nmodel = \"llama\"\n").unwrap(); + let error = Config::load(Some(&path)).expect_err("empty base_url"); + assert!( + error.to_string().contains("providers.local.base_url"), + "{error}" + ); + + // A brand-new provider only needs the endpoint and model; it is + // reachable as soon as a role names it. + std::fs::write( + &path, + "[llm]\neditor = \"local\"\n\n[providers.local]\nbase_url = \"http://127.0.0.1:11434/v1\"\nmodel = \"llama\"\n", + ) + .unwrap(); + let c = Config::load(Some(&path)).expect("new provider"); + let (name, local) = c.editor_provider().expect("editor"); + assert_eq!(name, "local"); + assert_eq!(local.kind, ProviderKind::OpenAi); + assert_eq!(local.max_daily_usd, 0.0, "no ceiling unless configured"); + assert_eq!( + ProviderConfig::api_key_env_var("local"), + "DAILY_EPUB_PROVIDERS__LOCAL__API_KEY" + ); + } + + #[test] + fn check_report_lists_every_fact_and_flags_missing_keys() { + let dir = tempfile::tempdir().unwrap(); + let mut c = Config { + profile_path: dir.path().join("profile.md"), + database_path: dir.path().join("missing.db"), + ..Config::default() + }; + std::fs::write(&c.profile_path, "# profile\n").unwrap(); + c.providers.get_mut("deepseek").unwrap().api_key = Some("sk-secret".into()); + let lines = c.check_report(Some(Path::new("/etc/daily-epub/config.toml"))); + let text = lines.join("\n"); + assert!( + !text.contains("sk-secret"), + "keys are never printed:\n{text}" + ); + for needle in [ + "config: /etc/daily-epub/config.toml", + "! database_path: ", + "(MISSING)", + "profile_path: ", + "(exists)", + "llm.bulk: deepseek · openai · deepseek-v4-flash · effort - · max_daily_usd $2.00 · key present", + "! llm.editor: anthropic · anthropic · claude-opus-5 · effort high · max_daily_usd $3.00 · key MISSING (set DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY)", + "providers.gemini: unreferenced · openai · gemini-3.8-flash · key absent", + "! voyage: voyage-4-lite · enabled · max_daily_usd $0.50 · key MISSING (set DAILY_EPUB_VOYAGE__API_KEY)", + "editorial.summary_model: editor", + "publish.epub_dir: ", + "publish.xtc_dir: ", + ] { + assert!(text.contains(needle), "missing {needle:?} in:\n{text}"); + } + assert!(lines.iter().filter(|line| line.starts_with("! ")).count() >= 3); + + c.llm.editor.clear(); + let text = c.check_report(None).join("\n"); + assert!(text.contains("config: built-in defaults")); + assert!(text.contains("llm.editor: none"), "{text}"); + assert!(text.contains("providers.anthropic: unreferenced"), "{text}"); + } + #[test] fn shipped_example_config_parses() { let example = Path::new(env!("CARGO_MANIFEST_DIR")).join("config.example.toml"); @@ -912,14 +1569,22 @@ mod tests { assert_eq!(c.xtc.command, "node"); assert_eq!(c.xtc.format, XtcFormat::Xtch); assert_eq!(c.server.bind, "127.0.0.1:3499"); - assert_eq!(c.deepseek.base_url, "https://api.deepseek.com/v1"); - assert_eq!(c.deepseek.max_concurrent_requests, 4); - assert_eq!(c.deepseek.triage_batch_size, 25); - assert!(c.anthropic.enabled); - assert_eq!(c.anthropic.model, "claude-opus-5"); - assert_eq!(c.anthropic.effort, "high"); - assert!(c.anthropic.api_key.is_none(), "keys never live in the file"); - assert_eq!(c.anthropic.max_daily_usd, 3.0); + assert_eq!(c.llm.bulk, "deepseek"); + assert_eq!(c.llm.editor, "anthropic"); + assert_eq!(c.llm.triage_batch_size, 25); + assert_eq!( + c.providers["deepseek"].base_url, + "https://api.deepseek.com/v1" + ); + assert_eq!(c.providers["deepseek"].max_concurrent_requests, 4); + assert_eq!(c.providers["anthropic"].model, "claude-opus-5"); + assert_eq!(c.providers["anthropic"].effort.as_deref(), Some("high")); + assert_eq!(c.providers["anthropic"].max_daily_usd, 3.0); + assert_eq!(c.providers["gemini"].effort.as_deref(), Some("high")); + assert!( + c.providers.values().all(|p| p.api_key.is_none()), + "keys never live in the file" + ); assert_eq!(c.curation.max_article_count, 28); assert_eq!(c.editorial.summary_model, SummaryModel::Editor); assert_eq!(c.editorial.summary_input_tokens, 3000); @@ -959,6 +1624,11 @@ mod tests { ); continue; } + // TOML has no null: an unset `Option` (a provider without an + // `effort`) is documented by its absence. + if value.is_null() && !documented.contains_key(key) { + continue; + } let doc = documented .get(key) .unwrap_or_else(|| panic!("{path}.{key} is missing from config.example.toml")); @@ -974,8 +1644,8 @@ mod tests { for (key, default) in defaults.as_object().expect("config is a table") { let section = match key.as_str() { - "curation" | "anthropic" | "voyage" | "editorial" | "deepseek" => key, - "target_article_count" | "max_daily_usd" | "profile_path" | "interests_opml" => key, + "curation" | "llm" | "providers" | "voyage" | "editorial" => key, + "target_article_count" | "profile_path" | "interests_opml" => key, _ => continue, }; let documented = documented @@ -991,24 +1661,27 @@ mod tests { c.curation.max_article_count = c.target_article_count - 1; assert!(c.validate().is_err(), "max_article_count below the target"); let mut c = Config::default(); - c.anthropic.effort = "turbo".into(); + c.providers.get_mut("anthropic").unwrap().effort = Some("turbo".into()); assert!(c.validate().is_err(), "unknown effort"); for effort in ["low", "medium", "high", "xhigh", "max"] { let mut c = Config::default(); - c.anthropic.effort = effort.into(); + c.providers.get_mut("anthropic").unwrap().effort = Some(effort.into()); assert!(c.validate().is_ok(), "{effort} is a valid effort"); } let mut c = Config::default(); - c.deepseek.max_concurrent_requests = 0; + c.providers + .get_mut("deepseek") + .unwrap() + .max_concurrent_requests = 0; assert!(c.validate().is_err()); let mut c = Config::default(); - c.anthropic.max_concurrent_requests = 0; + c.providers.get_mut("anthropic").unwrap().max_daily_usd = -1.0; assert!(c.validate().is_err()); let mut c = Config::default(); - c.deepseek.deep_batch_size = 0; + c.llm.deep_batch_size = 0; assert!(c.validate().is_err()); let mut c = Config::default(); - c.deepseek.triage_batch_size = 0; + c.llm.triage_batch_size = 0; assert!(c.validate().is_err()); let mut c = Config::default(); c.editorial.summary_input_tokens = 0; diff --git a/src/curate/assess.rs b/src/curate/assess.rs index 8ec595f..0f2eefe 100644 --- a/src/curate/assess.rs +++ b/src/curate/assess.rs @@ -481,8 +481,8 @@ pub async fn run( #[cfg(test)] mod tests { use super::*; - use crate::config::{CurationConfig, DeepseekConfig}; - use crate::curate::llm::{MockBackend, UsageMeter}; + use crate::config::{CurationConfig, ProviderConfig}; + use crate::curate::llm::{MockBackend, PriceTable, UsageMeter}; use crate::curate::prefilter::tests::{article, with_social}; use crate::curate::signals::{Neighbour, TopInterest}; use crate::types::{TokenUsage, Triage}; @@ -520,7 +520,7 @@ mod tests { LlmClient::with_backend( "deepseek-v4-flash", "SYSTEM".into(), - UsageMeter::new(&DeepseekConfig::default(), limit_usd), + UsageMeter::with_prices(PriceTable::from(&ProviderConfig::deepseek()), limit_usd), backend, ) } diff --git a/src/curate/editor.rs b/src/curate/editor.rs index cb68c43..b02094b 100644 --- a/src/curate/editor.rs +++ b/src/curate/editor.rs @@ -637,7 +637,7 @@ pub fn select_without_llm( #[cfg(test)] mod tests { use super::*; - use crate::config::{AnthropicConfig, CurationConfig, DeepseekConfig}; + use crate::config::{CurationConfig, ProviderConfig}; use crate::curate::llm::{ChatBackend, LlmClient, MockBackend, PriceTable, UsageMeter}; use crate::curate::prefilter::tests::article; use crate::curate::signals::{Neighbour, TopInterest}; @@ -696,9 +696,9 @@ mod tests { fn mock(provider: &'static str, backend: Arc, limit: f64) -> LlmClient { let prices = if provider == "anthropic" { - PriceTable::anthropic(&AnthropicConfig::default()) + PriceTable::from(&ProviderConfig::anthropic()) } else { - PriceTable::deepseek(&DeepseekConfig::default()) + PriceTable::from(&ProviderConfig::deepseek()) }; LlmClient::with_backend_options( provider, @@ -1079,9 +1079,7 @@ mod tests { #[tokio::test] async fn refusal_on_the_editor_falls_back_to_bulk_with_the_same_prompt() { let editor = Arc::new(MockBackend::new()); - editor.push_llm_error(LlmError::Refusal { - provider: "anthropic", - }); + editor.push_llm_error(LlmError::refusal("anthropic")); let bulk = Arc::new(MockBackend::new()); bulk.push(picks_json(5), TokenUsage::default()); let llms = editor_and_bulk(Arc::clone(&editor), Arc::clone(&bulk)); diff --git a/src/curate/editorial.rs b/src/curate/editorial.rs index 24cc001..b457eef 100644 --- a/src/curate/editorial.rs +++ b/src/curate/editorial.rs @@ -1,4 +1,4 @@ -//! Claude-first summaries and The Brief, with per-call DeepSeek fallback (§14). +//! Editor-first summaries and The Brief, with per-call bulk fallback (§14). use std::collections::BTreeMap; use std::fmt::Write as _; @@ -12,7 +12,6 @@ use crate::config::{EditorialConfig, SummaryModel}; use crate::types::{ArticleId, Editorial, Lineup, Pick}; pub const FALLBACK_SUMMARY_WORDS: usize = 45; -pub const SUMMARY_CONCURRENCY: usize = 4; pub const SUMMARY_INSTRUCTIONS: &str = "\ TASK: write the newspaper abstract for one article in today's issue. @@ -102,9 +101,7 @@ pub async fn summarize_article( let response: SummaryResponse = llm.complete_json(&prompt, temperature).await?; let summary = response.summary.trim().to_string(); if summary.is_empty() { - return Err(LlmError::EmptyResponse { - provider: llm.provider, - }); + return Err(LlmError::empty_response(llm.provider())); } Ok(summary) } @@ -172,12 +169,17 @@ pub async fn summarize_all( temperature: f32, ) -> BTreeMap { let (primary, fallback) = summary_clients(llms, config.summary_model); + // The summary provider's own `max_concurrent_requests` bounds the fan-out. + let concurrency = primary + .map(|client| client.max_concurrent_requests) + .unwrap_or(1) + .max(1); stream::iter(lineup.picks.iter()) .map(|pick| async move { let summary = summarize_pick(pick, primary, fallback, config, temperature).await; (pick.article.id, summary) }) - .buffer_unordered(SUMMARY_CONCURRENCY) + .buffer_unordered(concurrency) .filter_map(|(id, summary)| async move { summary.map(|summary| (id, summary)) }) .collect() .await @@ -232,10 +234,7 @@ pub async fn brief( ) -> Result { let prompt = build_brief_prompt(lineup, summaries); let Some(primary) = llms.editor_or_bulk() else { - return Err(LlmError::Api { - provider: "editorial", - message: "no provider configured".into(), - }); + return Err(LlmError::api("editorial", "no provider configured")); }; let response = match primary .complete_json::(&prompt, temperature) @@ -258,9 +257,7 @@ pub async fn brief( }; let brief = response.brief.trim().to_string(); if brief.is_empty() { - return Err(LlmError::EmptyResponse { - provider: primary.provider, - }); + return Err(LlmError::empty_response(primary.provider())); } Ok(brief) } @@ -381,7 +378,7 @@ pub fn summary_to_html(summary: &str) -> String { #[cfg(test)] mod tests { use super::*; - use crate::config::{AnthropicConfig, DeepseekConfig}; + use crate::config::ProviderConfig; use crate::curate::llm::{ChatBackend, MockBackend, PriceTable, UsageMeter}; use crate::curate::prefilter::tests::article; use crate::types::TokenUsage; @@ -420,9 +417,9 @@ mod tests { fn mock(provider: &'static str, backend: Arc, limit: f64) -> LlmClient { let prices = if provider == "anthropic" { - PriceTable::anthropic(&AnthropicConfig::default()) + PriceTable::from(&ProviderConfig::anthropic()) } else { - PriceTable::deepseek(&DeepseekConfig::default()) + PriceTable::from(&ProviderConfig::deepseek()) }; LlmClient::with_backend_options( provider, @@ -534,9 +531,7 @@ mod tests { r#"{"summary": "Opus wrote this one."}"#, TokenUsage::default(), ); - editor.push_llm_error(LlmError::Refusal { - provider: "anthropic", - }); + editor.push_llm_error(LlmError::refusal("anthropic")); editor.push(BRIEF_FIXTURE, TokenUsage::default()); let bulk = Arc::new(MockBackend::new()); bulk.push( diff --git a/src/curate/llm.rs b/src/curate/llm.rs index b81dc4f..0682ad2 100644 --- a/src/curate/llm.rs +++ b/src/curate/llm.rs @@ -1,24 +1,30 @@ //! Provider-neutral LLM clients, transports, retry, and token accounting (§4, §5). //! //! Two transports speak to the wire directly through the shared `reqwest` -//! client (no vendor SDK; implementation notes, cross-cutting item 5 is stale): +//! client (no vendor SDK; implementation notes, cross-cutting item 5): //! -//! - [`DeepseekBackend`]: the OpenAI-compatible chat-completions endpoint. The -//! system prompt is the first message so DeepSeek's prefix cache hits. -//! - [`AnthropicBackend`]: `POST /v1/messages` with the system prompt as one -//! `cache_control: ephemeral` block, `output_config.effort`, and server-side -//! `fallbacks: "default"` (§4.2). No sampling parameters, no `thinking`, no -//! prefill — Opus 5 rejects them. A `stop_reason: "refusal"` (HTTP 200) is -//! [`LlmError::Refusal`], which the callers use to fall back to the bulk client. +//! - [`OpenAiCompatibleBackend`]: `POST {base_url}/chat/completions` with a +//! bearer key — DeepSeek, Gemini's compatibility endpoint, OpenAI, a local +//! server. The system prompt is the first message so prefix caches hit; +//! `reasoning_effort` is sent when the provider configures an `effort`. +//! - [`AnthropicBackend`]: `POST {base_url}/v1/messages` with the system prompt +//! as one `cache_control: ephemeral` block, `output_config.effort`, and +//! server-side `fallbacks: "default"` (§4.2). No sampling parameters, no +//! `thinking`, no prefill — Opus 5 rejects them. A `stop_reason: "refusal"` +//! (HTTP 200) is [`LlmError::Refusal`], which the callers use to fall back to +//! the bulk client. //! -//! Every call goes through [`LlmClient`], which sends the byte-identical system -//! prompt on every request, folds token usage into a per-provider [`UsageMeter`] -//! priced by a [`PriceTable`], and refuses further work once that provider's -//! `max_daily_usd` is spent. [`Llms`] pairs the bulk and editor clients. +//! Which transport a role uses is decided by name: `[llm] bulk = "deepseek"` +//! looks up `[providers.deepseek]` and its `kind`. Every call goes through +//! [`LlmClient`], which sends the byte-identical system prompt on every request, +//! folds token usage into that provider's [`UsageMeter`] priced by its +//! [`PriceTable`], and refuses further work once its `max_daily_usd` is spent. +//! [`Llms`] pairs the bulk and editor clients. //! //! Tests inject [`MockBackend`] or a loopback `axum` listener; nothing here //! touches the network under test. +use std::collections::BTreeMap; use std::future::Future; use std::pin::Pin; use std::sync::atomic::{AtomicBool, Ordering}; @@ -27,12 +33,14 @@ use std::sync::{Arc, Mutex}; use serde::Deserialize; use serde_json::json; -use crate::config::{AnthropicConfig, DeepseekConfig}; +use crate::config::{Config, ProviderConfig, ProviderKind}; use crate::http::RetryPolicy; use crate::types::TokenUsage; pub const JSON_OBJECT: &str = "json_object"; -const DEEPSEEK_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(180); +/// Concurrency for clients built without a provider entry (tests, mocks). +pub const DEFAULT_MAX_CONCURRENT_REQUESTS: usize = 4; +const OPENAI_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(180); const ANTHROPIC_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(300); const ANTHROPIC_VERSION: &str = "2023-06-01"; const ANTHROPIC_BETA: &str = "server-side-fallback-2026-07-01"; @@ -40,24 +48,15 @@ const ANTHROPIC_BETA: &str = "server-side-fallback-2026-07-01"; #[derive(Debug, thiserror::Error)] pub enum LlmError { #[error("{provider} api key is not configured (set {env_var})")] - MissingApiKey { - provider: &'static str, - env_var: &'static str, - }, + MissingApiKey { provider: String, env_var: String }, #[error("{provider} request failed: {message}")] - Api { - provider: &'static str, - message: String, - }, + Api { provider: String, message: String }, #[error("{provider} request failed (transient): {message}")] - Transient { - provider: &'static str, - message: String, - }, + Transient { provider: String, message: String }, #[error("{provider} returned a refusal")] - Refusal { provider: &'static str }, + Refusal { provider: String }, #[error("{provider} returned an empty completion")] - EmptyResponse { provider: &'static str }, + EmptyResponse { provider: String }, #[error("llm returned unparseable JSON: {0}")] Json(#[from] serde_json::Error), #[error("daily cost ceiling of ${limit:.2} reached (spent ${spent:.4})")] @@ -69,45 +68,56 @@ impl LlmError { matches!(self, LlmError::Transient { .. }) } - fn api(provider: &'static str, message: impl Into) -> Self { + pub fn api(provider: impl Into, message: impl Into) -> Self { Self::Api { - provider, + provider: provider.into(), message: message.into(), } } - fn transient(provider: &'static str, message: impl Into) -> Self { + fn transient(provider: impl Into, message: impl Into) -> Self { Self::Transient { - provider, + provider: provider.into(), message: message.into(), } } + + pub fn refusal(provider: impl Into) -> Self { + Self::Refusal { + provider: provider.into(), + } + } + + pub fn empty_response(provider: impl Into) -> Self { + Self::EmptyResponse { + provider: provider.into(), + } + } + + fn missing_api_key(provider: &str) -> Self { + Self::MissingApiKey { + provider: provider.to_string(), + env_var: ProviderConfig::api_key_env_var(provider), + } + } } +/// USD per 1M tokens for the four counters of [`TokenUsage`]. #[derive(Debug, Clone, Copy, PartialEq)] pub struct PriceTable { - pub input_per_mtok: f64, - pub cache_write_per_mtok: f64, - pub cache_read_per_mtok: f64, - pub output_per_mtok: f64, + pub input: f64, + pub cache_read: f64, + pub cache_write: f64, + pub output: f64, } -impl PriceTable { - pub fn deepseek(cfg: &DeepseekConfig) -> Self { +impl From<&ProviderConfig> for PriceTable { + fn from(cfg: &ProviderConfig) -> Self { Self { - input_per_mtok: cfg.price_input_per_mtok, - cache_write_per_mtok: 0.0, - cache_read_per_mtok: cfg.price_cached_input_per_mtok, - output_per_mtok: cfg.price_output_per_mtok, - } - } - - pub fn anthropic(cfg: &AnthropicConfig) -> Self { - Self { - input_per_mtok: cfg.price_input_per_mtok, - cache_write_per_mtok: cfg.price_cache_write_per_mtok, - cache_read_per_mtok: cfg.price_cache_read_per_mtok, - output_per_mtok: cfg.price_output_per_mtok, + input: cfg.price_input_per_mtok, + cache_read: cfg.price_cache_read_per_mtok, + cache_write: cfg.price_cache_write_per_mtok, + output: cfg.price_output_per_mtok, } } } @@ -122,10 +132,9 @@ pub struct UsageMeter { } impl UsageMeter { - /// A meter priced from the `[deepseek]` table; the other providers build - /// theirs with [`UsageMeter::with_prices`]. - pub fn new(cfg: &DeepseekConfig, limit_usd: f64) -> Self { - Self::with_prices(PriceTable::deepseek(cfg), limit_usd) + /// A meter priced from a `[providers.]` entry with its `max_daily_usd`. + pub fn for_provider(cfg: &ProviderConfig) -> Self { + Self::with_prices(PriceTable::from(cfg), cfg.max_daily_usd) } pub fn with_prices(prices: PriceTable, limit_usd: f64) -> Self { @@ -195,10 +204,10 @@ impl UsageMeter { pub fn cost_of(&self, usage: TokenUsage) -> f64 { usage.cost_usd( - self.prices.input_per_mtok, - self.prices.cache_write_per_mtok, - self.prices.cache_read_per_mtok, - self.prices.output_per_mtok, + self.prices.input, + self.prices.cache_write, + self.prices.cache_read, + self.prices.output, ) } @@ -234,6 +243,16 @@ impl UsageMeter { } } +/// One [`UsageMeter`] per provider that an `[llm]` role references, keyed by +/// provider name. Two roles on one provider share one meter and one ceiling. +pub fn provider_meters(config: &Config) -> BTreeMap { + config + .referenced_providers() + .into_iter() + .map(|(name, provider)| (name.to_string(), UsageMeter::for_provider(provider))) + .collect() +} + #[derive(Debug, Clone)] pub struct ChatRequest { pub model: String, @@ -256,28 +275,26 @@ pub trait ChatBackend: std::fmt::Debug + Send + Sync { fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result>; } +/// The OpenAI-compatible chat-completions transport (`kind = "openai"`). #[derive(Debug, Clone)] -pub struct DeepseekBackend { +pub struct OpenAiCompatibleBackend { + provider: Arc, http: reqwest::Client, endpoint: String, api_key: String, } -impl DeepseekBackend { - pub fn new(cfg: &DeepseekConfig) -> Result { +impl OpenAiCompatibleBackend { + /// `name` is the `[providers.]` key; it labels errors and log lines. + pub fn new(name: &str, cfg: &ProviderConfig) -> Result { let api_key = cfg - .api_key - .as_deref() - .map(str::trim) - .filter(|key| !key.is_empty()) - .ok_or(LlmError::MissingApiKey { - provider: "deepseek", - env_var: "DAILY_EPUB_DEEPSEEK__API_KEY", - })? + .api_key() + .ok_or_else(|| LlmError::missing_api_key(name))? .to_string(); - let http = crate::http::build_client(DEEPSEEK_TIMEOUT) - .map_err(|error| LlmError::api("deepseek", format!("building http client: {error}")))?; + let http = crate::http::build_client(OPENAI_TIMEOUT) + .map_err(|error| LlmError::api(name, format!("building http client: {error}")))?; Ok(Self { + provider: Arc::from(name), http, endpoint: format!("{}/chat/completions", cfg.base_url.trim_end_matches('/')), api_key, @@ -285,9 +302,10 @@ impl DeepseekBackend { } } -impl ChatBackend for DeepseekBackend { +impl ChatBackend for OpenAiCompatibleBackend { fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result> { Box::pin(async move { + let provider = &*self.provider; let mut body = json!({ "model": req.model, "messages": [ @@ -297,10 +315,18 @@ impl ChatBackend for DeepseekBackend { "temperature": req.temperature, "stream": false, }); - if req.json - && let Some(object) = body.as_object_mut() - { - object.insert("response_format".into(), json!({"type": JSON_OBJECT})); + if let Some(object) = body.as_object_mut() { + if req.json { + object.insert("response_format".into(), json!({"type": JSON_OBJECT})); + } + if let Some(effort) = req + .effort + .as_deref() + .map(str::trim) + .filter(|e| !e.is_empty()) + { + object.insert("reasoning_effort".into(), json!(effort)); + } } let response = self .http @@ -309,13 +335,13 @@ impl ChatBackend for DeepseekBackend { .json(&body) .send() .await - .map_err(|error| classify_reqwest_error("deepseek", error))?; + .map_err(|error| classify_reqwest_error(provider, error))?; let status = response.status(); if !status.is_success() { - return Err(classify_status("deepseek", status, response).await); + return Err(classify_status(provider, status, response).await); } - let parsed: DeepseekResponse = response.json().await.map_err(|error| { - LlmError::api("deepseek", format!("decoding chat completion: {error}")) + let parsed: OpenAiResponse = response.json().await.map_err(|error| { + LlmError::api(provider, format!("decoding chat completion: {error}")) })?; let content = parsed .choices @@ -323,40 +349,42 @@ impl ChatBackend for DeepseekBackend { .next() .and_then(|choice| choice.message.content) .filter(|content| !content.trim().is_empty()) - .ok_or(LlmError::EmptyResponse { - provider: "deepseek", - })?; - let usage = parsed.usage.map(deepseek_usage).unwrap_or_default(); + .ok_or_else(|| LlmError::empty_response(provider))?; + let usage = parsed.usage.map(openai_usage).unwrap_or_default(); Ok(ChatCompletion { content, usage }) }) } } #[derive(Debug, Deserialize)] -struct DeepseekResponse { +struct OpenAiResponse { #[serde(default)] - choices: Vec, + choices: Vec, #[serde(default)] - usage: Option, + usage: Option, } #[derive(Debug, Deserialize)] -struct DeepseekChoice { - message: DeepseekMessage, +struct OpenAiChoice { + message: OpenAiMessage, } #[derive(Debug, Deserialize)] -struct DeepseekMessage { +struct OpenAiMessage { #[serde(default)] content: Option, } +/// The `usage` object. `completion_tokens` already includes reasoning tokens +/// on every provider that reports `completion_tokens_details.reasoning_tokens` +/// (OpenAI, Gemini), so that detail is deliberately not added on top. #[derive(Debug, Default, Deserialize)] -struct DeepseekUsage { +struct OpenAiUsage { #[serde(default)] prompt_tokens: i64, #[serde(default)] completion_tokens: i64, + /// DeepSeek's native cache counter, the fallback for `cached_tokens`. #[serde(default)] prompt_cache_hit_tokens: Option, #[serde(default)] @@ -369,7 +397,7 @@ struct PromptTokenDetails { cached_tokens: Option, } -fn deepseek_usage(usage: DeepseekUsage) -> TokenUsage { +fn openai_usage(usage: OpenAiUsage) -> TokenUsage { let cached = usage .prompt_tokens_details .as_ref() @@ -387,29 +415,26 @@ fn deepseek_usage(usage: DeepseekUsage) -> TokenUsage { } } +/// The Anthropic Messages API transport (`kind = "anthropic"`). #[derive(Debug, Clone)] pub struct AnthropicBackend { + provider: Arc, http: reqwest::Client, endpoint: String, api_key: String, } impl AnthropicBackend { - pub fn new(cfg: &AnthropicConfig) -> Result { + /// `name` is the `[providers.]` key; it labels errors and log lines. + pub fn new(name: &str, cfg: &ProviderConfig) -> Result { let api_key = cfg - .api_key - .as_deref() - .map(str::trim) - .filter(|key| !key.is_empty()) - .ok_or(LlmError::MissingApiKey { - provider: "anthropic", - env_var: "DAILY_EPUB_ANTHROPIC__API_KEY", - })? + .api_key() + .ok_or_else(|| LlmError::missing_api_key(name))? .to_string(); - let http = crate::http::build_client(ANTHROPIC_TIMEOUT).map_err(|error| { - LlmError::api("anthropic", format!("building http client: {error}")) - })?; + let http = crate::http::build_client(ANTHROPIC_TIMEOUT) + .map_err(|error| LlmError::api(name, format!("building http client: {error}")))?; Ok(Self { + provider: Arc::from(name), http, endpoint: format!("{}/v1/messages", cfg.base_url.trim_end_matches('/')), api_key, @@ -420,6 +445,7 @@ impl AnthropicBackend { impl ChatBackend for AnthropicBackend { fn complete<'a>(&'a self, req: ChatRequest) -> BoxFuture<'a, Result> { Box::pin(async move { + let provider = &*self.provider; let body = json!({ "model": req.model, "max_tokens": 16_000, @@ -442,18 +468,16 @@ impl ChatBackend for AnthropicBackend { .json(&body) .send() .await - .map_err(|error| classify_reqwest_error("anthropic", error))?; + .map_err(|error| classify_reqwest_error(provider, error))?; let status = response.status(); if !status.is_success() { - return Err(classify_status("anthropic", status, response).await); + return Err(classify_status(provider, status, response).await); } let parsed: AnthropicResponse = response.json().await.map_err(|error| { - LlmError::api("anthropic", format!("decoding messages response: {error}")) + LlmError::api(provider, format!("decoding messages response: {error}")) })?; if parsed.stop_reason.as_deref() == Some("refusal") { - return Err(LlmError::Refusal { - provider: "anthropic", - }); + return Err(LlmError::refusal(provider)); } let content = parsed .content @@ -463,9 +487,7 @@ impl ChatBackend for AnthropicBackend { .collect::>() .join(""); if content.trim().is_empty() { - return Err(LlmError::EmptyResponse { - provider: "anthropic", - }); + return Err(LlmError::empty_response(provider)); } Ok(ChatCompletion { content, @@ -515,7 +537,7 @@ fn anthropic_usage(usage: AnthropicUsage) -> TokenUsage { } async fn classify_status( - provider: &'static str, + provider: &str, status: reqwest::StatusCode, response: reqwest::Response, ) -> LlmError { @@ -528,7 +550,7 @@ async fn classify_status( } } -fn classify_reqwest_error(provider: &'static str, error: reqwest::Error) -> LlmError { +fn classify_reqwest_error(provider: &str, error: reqwest::Error) -> LlmError { if crate::http::is_retryable(&error) { LlmError::transient(provider, error.to_string()) } else { @@ -538,46 +560,41 @@ fn classify_reqwest_error(provider: &'static str, error: reqwest::Error) -> LlmE #[derive(Debug, Clone)] pub struct LlmClient { - pub provider: &'static str, + /// The `[providers.]` key: the `provider_costs` key, the meter's + /// preload key and the label on every log line. Never the kind. + pub provider: Arc, pub system_prompt: Arc, pub model: String, pub effort: Option, + /// Batches in flight for the stages that fan out on this client. + pub max_concurrent_requests: usize, pub meter: UsageMeter, backend: Arc, retry: RetryPolicy, } impl LlmClient { - pub fn new( - cfg: &DeepseekConfig, + /// A client for one `[providers.]` entry, dispatching on its `kind`. + pub fn for_provider( + name: &str, + cfg: &ProviderConfig, system_prompt: String, meter: UsageMeter, ) -> Result { - let backend = DeepseekBackend::new(cfg)?; - Ok(Self::with_backend_options( - "deepseek", + let backend: Arc = match cfg.kind { + ProviderKind::OpenAi => Arc::new(OpenAiCompatibleBackend::new(name, cfg)?), + ProviderKind::Anthropic => Arc::new(AnthropicBackend::new(name, cfg)?), + }; + let mut client = Self::with_backend_options( + name, &cfg.model, system_prompt, - None, + cfg.effort.clone(), meter, - Arc::new(backend), - )) - } - - pub fn new_anthropic( - cfg: &AnthropicConfig, - system_prompt: String, - meter: UsageMeter, - ) -> Result { - let backend = AnthropicBackend::new(cfg)?; - Ok(Self::with_backend_options( - "anthropic", - &cfg.model, - system_prompt, - Some(cfg.effort.clone()), - meter, - Arc::new(backend), - )) + backend, + ); + client.max_concurrent_requests = cfg.max_concurrent_requests.max(1); + Ok(client) } pub fn with_backend( @@ -590,7 +607,7 @@ impl LlmClient { } pub fn with_backend_options( - provider: &'static str, + provider: &str, model: &str, system_prompt: String, effort: Option, @@ -598,10 +615,11 @@ impl LlmClient { backend: Arc, ) -> Self { Self { - provider, + provider: Arc::from(provider), system_prompt: Arc::new(system_prompt), model: model.to_string(), effort, + max_concurrent_requests: DEFAULT_MAX_CONCURRENT_REQUESTS, meter, backend, retry: RetryPolicy::default(), @@ -614,6 +632,11 @@ impl LlmClient { self } + /// The provider name as a plain `&str`. + pub fn provider(&self) -> &str { + &self.provider + } + pub async fn complete( &self, user_prompt: &str, @@ -652,7 +675,7 @@ impl LlmClient { Ok(value) => Ok(value), Err(error) => { tracing::warn!( - provider = self.provider, + provider = %self.provider, %error, preview = %cleaned.chars().take(400).collect::(), "llm returned malformed JSON" @@ -671,48 +694,66 @@ impl LlmClient { } } -/// The two provider clients the pipeline works with (§4.2). +/// The two role clients the pipeline works with (§4.2). /// -/// Both share the exact same system prompt string (§8.4). Each has its own -/// [`UsageMeter`] with its own price table and `max_daily_usd` (§5). +/// Both share the exact same system prompt string (§8.4). Each has the +/// [`UsageMeter`] of its provider, with that provider's price table and +/// `max_daily_usd` (§5); two roles on one provider share one meter. #[derive(Debug, Clone, Default)] pub struct Llms { - /// DeepSeek — scoring, and the fallback for every editor call. + /// `[llm] bulk` — triage, deep assessment, and the fallback for every editor call. pub bulk: Option, - /// Claude — selection, summaries, the brief, the profile rebuild. + /// `[llm] editor` — selection, summaries, the brief, the profile rebuild. pub editor: Option, } impl Llms { - /// Build both clients from config with one shared system prompt. + /// Build both role clients by provider name with one shared system prompt. /// - /// A missing key or `anthropic.enabled = false` leaves that slot `None` with - /// a log line; nothing here is fatal because the paper always publishes (§17). + /// `meters` holds one meter per referenced provider (see + /// [`provider_meters`]); a provider missing from it gets a fresh meter. A + /// missing key or an unassigned role leaves that slot `None` with a log + /// line naming the provider; nothing here is fatal because the paper + /// always publishes (§17). When both roles name the same provider they + /// share one client and so one meter. pub fn from_config( - deepseek: &DeepseekConfig, - anthropic: &AnthropicConfig, + config: &Config, system_prompt: String, - bulk_meter: UsageMeter, - editor_meter: UsageMeter, + meters: &BTreeMap, ) -> Self { - let bulk = match LlmClient::new(deepseek, system_prompt.clone(), bulk_meter) { - Ok(client) => Some(client), - Err(error) => { - tracing::warn!(%error, "DeepSeek (bulk) is unavailable"); - None - } - }; - let editor = if anthropic.enabled { - match LlmClient::new_anthropic(anthropic, system_prompt, editor_meter) { + let build = |role: &str, name: &str, cfg: &ProviderConfig| { + let meter = meters + .get(name) + .cloned() + .unwrap_or_else(|| UsageMeter::for_provider(cfg)); + match LlmClient::for_provider(name, cfg, system_prompt.clone(), meter) { Ok(client) => Some(client), Err(error) => { - tracing::warn!(%error, "Anthropic (editor) is unavailable; editor work falls back to bulk"); + tracing::warn!(role, provider = name, %error, "provider is unavailable"); None } } - } else { - tracing::info!("anthropic.enabled = false; editor work runs on the bulk provider"); - None + }; + let bulk = match config.bulk_provider() { + Some((name, cfg)) => build("bulk", name, cfg), + None => { + tracing::info!("no bulk provider: triage and deep assessment are skipped"); + None + } + }; + let editor = match config.editor_provider() { + Some((name, _)) if config.llm.bulk_name() == Some(name) => { + tracing::info!( + provider = name, + "editor and bulk share one provider, client and ceiling" + ); + bulk.clone() + } + Some((name, cfg)) => build("editor", name, cfg), + None => { + tracing::info!("no editor provider; editor work runs on the bulk provider"); + None + } }; Self { bulk, editor } } @@ -812,8 +853,16 @@ mod tests { use axum::routing::post; use axum::{Json, Router}; - fn cfg() -> DeepseekConfig { - DeepseekConfig::default() + fn deepseek() -> ProviderConfig { + ProviderConfig::deepseek() + } + + fn anthropic() -> ProviderConfig { + ProviderConfig::anthropic() + } + + fn deepseek_meter(limit: f64) -> UsageMeter { + UsageMeter::with_prices(PriceTable::from(&deepseek()), limit) } pub(crate) fn tokens(input: i64, cached: i64, output: i64) -> TokenUsage { @@ -831,7 +880,8 @@ mod tests { #[test] fn meter_accumulates_and_prices() { - let meter = UsageMeter::new(&cfg(), 2.0); + let meter = UsageMeter::for_provider(&deepseek()); + assert_eq!(meter.limit_usd(), 2.0, "the provider's own ceiling"); meter.record(tokens(1_000_000, 0, 0)); meter.record(tokens(0, 1_000_000, 1_000_000)); let total = meter.total(); @@ -846,8 +896,7 @@ mod tests { #[test] fn anthropic_price_table_charges_cache_reads_and_writes() { - let meter = - UsageMeter::with_prices(PriceTable::anthropic(&AnthropicConfig::default()), 100.0); + let meter = UsageMeter::with_prices(PriceTable::from(&anthropic()), 100.0); meter.record(TokenUsage { input_tokens: 1_000_000, cached_tokens: 1_000_000, @@ -861,7 +910,7 @@ mod tests { #[test] fn meter_trips_the_budget_flag_and_stays_tripped() { // Ceiling of $0.10; 1M cache-miss input tokens costs $0.14. - let meter = UsageMeter::new(&cfg(), 0.10); + let meter = deepseek_meter(0.10); meter.record(tokens(1_000_000, 0, 0)); assert!(meter.budget_exceeded()); assert!(matches!( @@ -874,7 +923,7 @@ mod tests { #[test] fn preloaded_daily_spend_trips_the_flag() { - let meter = UsageMeter::new(&cfg(), 1.0); + let meter = deepseek_meter(1.0); meter.preload_cost(0.5); assert!(!meter.budget_exceeded()); assert!((meter.spent_usd() - 0.5).abs() < 1e-9); @@ -883,26 +932,26 @@ mod tests { } #[test] - fn deepseek_usage_split_uses_prompt_token_details() { - let u: DeepseekUsage = serde_json::from_str( + fn openai_usage_split_uses_prompt_token_details() { + let u: OpenAiUsage = serde_json::from_str( r#"{"prompt_tokens": 1000, "completion_tokens": 120, "total_tokens": 1120, "prompt_tokens_details": {"cached_tokens": 800}}"#, ) .expect("fixture usage"); - assert_eq!(deepseek_usage(u), tokens(200, 800, 120)); + assert_eq!(openai_usage(u), tokens(200, 800, 120)); } #[test] - fn deepseek_usage_falls_back_to_native_cache_fields() { - let u: DeepseekUsage = serde_json::from_str( + fn openai_usage_falls_back_to_native_cache_fields() { + let u: OpenAiUsage = serde_json::from_str( r#"{"prompt_tokens": 500, "completion_tokens": 40, "prompt_cache_hit_tokens": 448, "prompt_cache_miss_tokens": 52}"#, ) .expect("fixture usage"); - assert_eq!(deepseek_usage(u), tokens(52, 448, 40)); + assert_eq!(openai_usage(u), tokens(52, 448, 40)); // Missing usage is not an error, just zero. - let empty: DeepseekUsage = serde_json::from_str("{}").expect("empty usage"); - assert_eq!(deepseek_usage(empty), TokenUsage::default()); + let empty: OpenAiUsage = serde_json::from_str("{}").expect("empty usage"); + assert_eq!(openai_usage(empty), TokenUsage::default()); } #[test] @@ -938,7 +987,7 @@ mod tests { LlmClient::with_backend( "deepseek-v4-flash", "SYSTEM PROMPT".into(), - UsageMeter::new(&cfg(), limit), + deepseek_meter(limit), backend, ) } @@ -1006,9 +1055,7 @@ mod tests { #[tokio::test] async fn refusals_are_not_retried_and_keep_their_variant() { let backend = Arc::new(MockBackend::new()); - backend.push_llm_error(LlmError::Refusal { - provider: "anthropic", - }); + backend.push_llm_error(LlmError::refusal("anthropic")); backend.push("{}", TokenUsage::default()); let llm = client(Arc::clone(&backend), 2.0).with_retry(RetryPolicy { max_attempts: 3, @@ -1016,41 +1063,47 @@ mod tests { max_delay: Duration::from_millis(2), }); let err = llm.complete_text("x", 0.3).await.expect_err("refusal"); - assert!(matches!( - err, - LlmError::Refusal { - provider: "anthropic" - } - )); + assert!(matches!(err, LlmError::Refusal { ref provider } if provider == "anthropic")); assert_eq!(backend.calls(), 1, "a refusal is terminal for that client"); } #[test] fn missing_api_keys_name_their_provider() { - let deepseek = DeepseekConfig { + let blank = ProviderConfig { api_key: Some(" ".into()), - ..DeepseekConfig::default() + ..deepseek() }; - let err = DeepseekBackend::new(&deepseek).expect_err("blank key"); - assert!(matches!( - err, - LlmError::MissingApiKey { - provider: "deepseek", - .. - } - )); - assert!(err.to_string().contains("DAILY_EPUB_DEEPSEEK__API_KEY")); + let err = OpenAiCompatibleBackend::new("bulkprov", &blank).expect_err("blank key"); + assert!( + matches!(err, LlmError::MissingApiKey { ref provider, .. } if provider == "bulkprov") + ); + assert!( + err.to_string() + .contains("DAILY_EPUB_PROVIDERS__BULKPROV__API_KEY"), + "{err}" + ); - let anthropic = AnthropicConfig::default(); - let err = AnthropicBackend::new(&anthropic).expect_err("no key"); - assert!(matches!( - err, - LlmError::MissingApiKey { - provider: "anthropic", - .. - } - )); - assert!(err.to_string().contains("DAILY_EPUB_ANTHROPIC__API_KEY")); + let err = AnthropicBackend::new("anthropic", &anthropic()).expect_err("no key"); + assert!( + matches!(err, LlmError::MissingApiKey { ref provider, .. } if provider == "anthropic") + ); + assert!( + err.to_string() + .contains("DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY") + ); + + // The dispatching constructor reports the same error for either kind. + let err = LlmClient::for_provider( + "gemini", + &ProviderConfig::gemini(), + "S".into(), + deepseek_meter(1.0), + ) + .expect_err("no key"); + assert!( + err.to_string() + .contains("DAILY_EPUB_PROVIDERS__GEMINI__API_KEY") + ); } // ----------------------------------------------------------------------- @@ -1060,9 +1113,9 @@ mod tests { fn mock_client(provider: &'static str, limit: f64) -> (LlmClient, Arc) { let backend = Arc::new(MockBackend::new()); let prices = if provider == "anthropic" { - PriceTable::anthropic(&AnthropicConfig::default()) + PriceTable::from(&anthropic()) } else { - PriceTable::deepseek(&cfg()) + PriceTable::from(&deepseek()) }; let client = LlmClient::with_backend_options( provider, @@ -1083,14 +1136,20 @@ mod tests { bulk: Some(bulk), editor: Some(editor), }; - assert_eq!(llms.editor_or_bulk().map(|c| c.provider), Some("anthropic")); + assert_eq!( + llms.editor_or_bulk().map(LlmClient::provider), + Some("anthropic") + ); // Trip the editor's meter: bulk takes over. llms.editor .as_ref() .expect("editor") .meter .preload_cost(10.0); - assert_eq!(llms.editor_or_bulk().map(|c| c.provider), Some("deepseek")); + assert_eq!( + llms.editor_or_bulk().map(LlmClient::provider), + Some("deepseek") + ); // No bulk and a tripped editor means no client at all. let only_editor = Llms { bulk: None, @@ -1103,27 +1162,122 @@ mod tests { #[test] fn from_config_without_keys_yields_no_clients() { - let llms = Llms::from_config( - &cfg(), - &AnthropicConfig::default(), - "SYSTEM".into(), - UsageMeter::new(&cfg(), 1.0), - UsageMeter::with_prices(PriceTable::anthropic(&AnthropicConfig::default()), 1.0), + let config = Config::default(); + let meters = provider_meters(&config); + assert_eq!( + meters.keys().collect::>(), + vec!["anthropic", "deepseek"], + "one meter per referenced provider, not per registry entry" ); + let llms = Llms::from_config(&config, "SYSTEM".into(), &meters); assert!(llms.is_empty()); } + fn keyed_config() -> Config { + let mut config = Config::default(); + for (name, provider) in config.providers.iter_mut() { + provider.api_key = Some(format!("{name}-key")); + } + config + } + + #[test] + fn from_config_resolves_both_roles_by_name() { + let mut config = keyed_config(); + config.llm.editor = "gemini".into(); + config + .providers + .get_mut("deepseek") + .unwrap() + .max_concurrent_requests = 7; + let meters = provider_meters(&config); + let llms = Llms::from_config(&config, "SYSTEM".into(), &meters); + + let bulk = llms.bulk.as_ref().expect("bulk"); + assert_eq!(bulk.provider(), "deepseek"); + assert_eq!(bulk.model, "deepseek-v4-flash"); + assert_eq!(bulk.effort, None); + assert_eq!(bulk.max_concurrent_requests, 7); + assert_eq!(bulk.meter.limit_usd(), 2.0); + let editor = llms.editor.as_ref().expect("editor"); + assert_eq!(editor.provider(), "gemini"); + assert_eq!(editor.model, "gemini-3.8-flash"); + assert_eq!(editor.effort.as_deref(), Some("high")); + assert_eq!(editor.meter.limit_usd(), 3.0); + assert_eq!(bulk.system_prompt, editor.system_prompt); + // The clients share the meters the pipeline preloads and reports from. + meters["gemini"].preload_cost(0.25); + assert!((editor.meter.spent_usd() - 0.25).abs() < 1e-9); + assert_eq!( + llms.editor_or_bulk().map(LlmClient::provider), + Some("gemini") + ); + + // The Anthropic kind resolves the same way. + let config = keyed_config(); + let llms = Llms::from_config(&config, "SYSTEM".into(), &provider_meters(&config)); + let editor = llms.editor.as_ref().expect("editor"); + assert_eq!(editor.provider(), "anthropic"); + assert_eq!(editor.model, "claude-opus-5"); + } + + #[test] + fn same_provider_for_both_roles_shares_one_client_and_meter() { + let mut config = keyed_config(); + config.llm.bulk = "gemini".into(); + config.llm.editor = "gemini".into(); + let meters = provider_meters(&config); + assert_eq!(meters.len(), 1); + let llms = Llms::from_config(&config, "SYSTEM".into(), &meters); + let bulk = llms.bulk.as_ref().expect("bulk"); + let editor = llms.editor.as_ref().expect("editor"); + assert!(Arc::ptr_eq(&bulk.backend, &editor.backend)); + assert!(Arc::ptr_eq(&bulk.meter.inner, &editor.meter.inner)); + assert_eq!(bulk.provider(), editor.provider()); + } + + #[test] + fn missing_key_leaves_that_role_empty_and_no_editor_is_honoured() { + let mut config = keyed_config(); + config.providers.get_mut("anthropic").unwrap().api_key = None; + let llms = Llms::from_config(&config, "SYSTEM".into(), &provider_meters(&config)); + assert!(llms.bulk.is_some()); + assert!(llms.editor.is_none(), "no key, no editor client"); + assert_eq!( + llms.editor_or_bulk().map(LlmClient::provider), + Some("deepseek") + ); + + let mut config = keyed_config(); + config.llm.editor.clear(); + let llms = Llms::from_config(&config, "SYSTEM".into(), &provider_meters(&config)); + assert!(llms.editor.is_none()); + assert_eq!( + llms.editor_or_bulk().map(LlmClient::provider), + Some("deepseek") + ); + + let mut config = keyed_config(); + config.llm.bulk.clear(); + let llms = Llms::from_config(&config, "SYSTEM".into(), &provider_meters(&config)); + assert!(llms.bulk.is_none()); + assert_eq!( + llms.editor_or_bulk().map(LlmClient::provider), + Some("anthropic") + ); + } + // ----------------------------------------------------------------------- // AnthropicBackend against a loopback listener (§4.2, §20) // ----------------------------------------------------------------------- #[derive(Clone, Default)] - struct FakeAnthropic { + struct FakeServer { seen: Arc>>, scripted: Arc>>, } - impl FakeAnthropic { + impl FakeServer { fn push(&self, status: StatusCode, body: serde_json::Value) { self.scripted .lock() @@ -1137,7 +1291,7 @@ mod tests { } async fn handle( - State(fake): State, + State(fake): State, headers: HeaderMap, Json(body): Json, ) -> (StatusCode, Json) { @@ -1154,9 +1308,10 @@ mod tests { (status, Json(body)) } - async fn serve(fake: FakeAnthropic) -> String { + async fn serve(fake: FakeServer) -> String { let app = Router::new() .route("/v1/messages", post(handle)) + .route("/chat/completions", post(handle)) .with_state(fake); let listener = tokio::net::TcpListener::bind("127.0.0.1:0") .await @@ -1168,18 +1323,19 @@ mod tests { format!("http://{addr}") } - async fn anthropic_client(fake: FakeAnthropic, limit: f64) -> LlmClient { + async fn anthropic_client(fake: FakeServer, limit: f64) -> LlmClient { let base_url = serve(fake).await; - let config = AnthropicConfig { + let config = ProviderConfig { base_url, api_key: Some("test-key-never-logged".into()), - effort: "medium".into(), - ..AnthropicConfig::default() + effort: Some("medium".into()), + ..anthropic() }; - LlmClient::new_anthropic( + LlmClient::for_provider( + "anthropic", &config, "PROFILE SYSTEM PROMPT".into(), - UsageMeter::with_prices(PriceTable::anthropic(&config), limit), + UsageMeter::with_prices(PriceTable::from(&config), limit), ) .expect("client") .with_retry(RetryPolicy { @@ -1211,7 +1367,7 @@ mod tests { #[tokio::test] async fn anthropic_request_has_the_documented_shape() { - let fake = FakeAnthropic::default(); + let fake = FakeServer::default(); fake.push( StatusCode::OK, ok_message("```json\n{\"ok\": true}\n```", "end_turn"), @@ -1290,7 +1446,7 @@ mod tests { #[tokio::test] async fn anthropic_refusal_surfaces_as_the_fallback_error() { - let fake = FakeAnthropic::default(); + let fake = FakeServer::default(); fake.push( StatusCode::OK, json!({ @@ -1302,19 +1458,14 @@ mod tests { ); let llm = anthropic_client(fake.clone(), 100.0).await; let err = llm.complete_text("x", 0.3).await.expect_err("refusal"); - assert!(matches!( - err, - LlmError::Refusal { - provider: "anthropic" - } - )); + assert!(matches!(err, LlmError::Refusal { ref provider } if provider == "anthropic")); assert!(!err.is_transient()); assert_eq!(fake.requests().len(), 1, "a refusal is never retried"); } #[tokio::test] async fn anthropic_429_is_retried_but_400_is_not() { - let fake = FakeAnthropic::default(); + let fake = FakeServer::default(); fake.push( StatusCode::TOO_MANY_REQUESTS, json!({"type": "error", "error": {"type": "rate_limit_error"}}), @@ -1328,27 +1479,21 @@ mod tests { assert_eq!(text, "{\"after\": \"retry\"}"); assert_eq!(fake.requests().len(), 2); - let fake = FakeAnthropic::default(); + let fake = FakeServer::default(); fake.push( StatusCode::BAD_REQUEST, json!({"type": "error", "error": {"type": "invalid_request_error", "message": "nope"}}), ); let llm = anthropic_client(fake.clone(), 100.0).await; let err = llm.complete_text("x", 0.3).await.expect_err("400"); - assert!(matches!( - err, - LlmError::Api { - provider: "anthropic", - .. - } - )); + assert!(matches!(err, LlmError::Api { ref provider, .. } if provider == "anthropic")); assert!(err.to_string().contains("400")); assert_eq!(fake.requests().len(), 1, "400 is never retried"); } #[tokio::test] async fn anthropic_concatenates_text_blocks_and_rejects_empty_output() { - let fake = FakeAnthropic::default(); + let fake = FakeServer::default(); fake.push( StatusCode::OK, json!({ @@ -1373,11 +1518,146 @@ mod tests { let out: serde_json::Value = llm.complete_json("x", 0.3).await.expect("joined"); assert_eq!(out, json!({"a": 1})); let err = llm.complete_text("y", 0.3).await.expect_err("empty"); - assert!(matches!( - err, - LlmError::EmptyResponse { - provider: "anthropic" + assert!(matches!(err, LlmError::EmptyResponse { ref provider } if provider == "anthropic")); + } + + // ----------------------------------------------------------------------- + // OpenAiCompatibleBackend against a loopback listener + // ----------------------------------------------------------------------- + + async fn openai_client(fake: FakeServer, name: &str, effort: Option<&str>) -> LlmClient { + let base_url = serve(fake).await; + let config = ProviderConfig { + base_url, + api_key: Some("bearer-key-never-logged".into()), + effort: effort.map(str::to_string), + ..ProviderConfig::gemini() + }; + LlmClient::for_provider( + name, + &config, + "PROFILE SYSTEM PROMPT".into(), + UsageMeter::for_provider(&config), + ) + .expect("client") + .with_retry(RetryPolicy { + max_attempts: 3, + base_delay: Duration::from_millis(1), + max_delay: Duration::from_millis(2), + }) + } + + fn ok_completion(text: &str) -> serde_json::Value { + json!({ + "id": "chatcmpl-01", + "object": "chat.completion", + "model": "gemini-3.8-flash", + "choices": [{ + "index": 0, + "message": {"role": "assistant", "content": text}, + "finish_reason": "stop" + }], + "usage": { + "prompt_tokens": 1000, + "completion_tokens": 300, + "total_tokens": 1300, + "prompt_tokens_details": {"cached_tokens": 800}, + "completion_tokens_details": {"reasoning_tokens": 250} } - )); + }) + } + + #[tokio::test] + async fn openai_request_carries_effort_json_mode_and_bearer_key() { + let fake = FakeServer::default(); + fake.push(StatusCode::OK, ok_completion("{\"ok\": true}")); + let llm = openai_client(fake.clone(), "gemini", Some("high")).await; + let out: serde_json::Value = llm + .complete_json("the task", 0.3) + .await + .expect("completion"); + assert_eq!(out, json!({"ok": true})); + + let requests = fake.requests(); + assert_eq!(requests.len(), 1); + let (headers, body) = &requests[0]; + assert_eq!( + headers.get("authorization").and_then(|v| v.to_str().ok()), + Some("Bearer bearer-key-never-logged") + ); + assert_eq!(body["model"], "gemini-3.8-flash"); + assert_eq!(body["messages"][0]["role"], "system"); + assert_eq!(body["messages"][0]["content"], "PROFILE SYSTEM PROMPT"); + assert_eq!(body["messages"][1]["role"], "user"); + assert_eq!(body["messages"][1]["content"], "the task"); + let temperature = body["temperature"].as_f64().expect("temperature"); + assert!( + (temperature - 0.3).abs() < 1e-6, + "f32 widened: {temperature}" + ); + assert_eq!(body["response_format"]["type"], "json_object"); + assert_eq!(body["reasoning_effort"], "high"); + assert!(body.get("output_config").is_none()); + + // Cached prompt tokens come from `prompt_tokens_details`; reasoning + // tokens are already inside `completion_tokens` and are not added twice. + assert_eq!(llm.meter.total(), tokens(200, 800, 300)); + let expected = 200.0 * 0.75 / 1e6 + 800.0 * 0.075 / 1e6 + 300.0 * 3.75 / 1e6; + assert!((llm.meter.cost_usd() - expected).abs() < 1e-12); + } + + #[tokio::test] + async fn openai_request_omits_effort_and_json_mode_when_unset() { + let fake = FakeServer::default(); + fake.push(StatusCode::OK, ok_completion("plain prose")); + let llm = openai_client(fake.clone(), "deepseek", None).await; + let text = llm.complete_text("write", 0.8).await.expect("completion"); + assert_eq!(text, "plain prose"); + let (_, body) = &fake.requests()[0]; + assert!( + body.get("reasoning_effort").is_none(), + "no effort configured" + ); + assert!(body.get("response_format").is_none(), "not a json call"); + assert_eq!(body["stream"], false); + } + + #[tokio::test] + async fn openai_errors_name_the_provider_and_retry_only_transient_statuses() { + let fake = FakeServer::default(); + fake.push( + StatusCode::SERVICE_UNAVAILABLE, + json!({"error": "warming up"}), + ); + fake.push(StatusCode::OK, ok_completion("after retry")); + let llm = openai_client(fake.clone(), "bulkprov", None).await; + assert_eq!( + llm.complete_text("x", 0.3).await.expect("retried"), + "after retry" + ); + assert_eq!(fake.requests().len(), 2); + + let fake = FakeServer::default(); + fake.push( + StatusCode::UNAUTHORIZED, + json!({"error": {"message": "bad key"}}), + ); + let llm = openai_client(fake.clone(), "bulkprov", None).await; + let err = llm.complete_text("x", 0.3).await.expect_err("401"); + assert!(matches!(err, LlmError::Api { ref provider, .. } if provider == "bulkprov")); + assert!( + err.to_string().starts_with("bulkprov request failed"), + "{err}" + ); + assert_eq!(fake.requests().len(), 1, "401 is never retried"); + + let fake = FakeServer::default(); + fake.push( + StatusCode::OK, + json!({"choices": [{"message": {"role": "assistant", "content": ""}}]}), + ); + let llm = openai_client(fake.clone(), "bulkprov", None).await; + let err = llm.complete_text("x", 0.3).await.expect_err("empty"); + assert!(matches!(err, LlmError::EmptyResponse { ref provider } if provider == "bulkprov")); } } diff --git a/src/curate/mod.rs b/src/curate/mod.rs index c1677da..cab708e 100644 --- a/src/curate/mod.rs +++ b/src/curate/mod.rs @@ -39,7 +39,7 @@ pub struct Curator { impl Curator { /// An empty [`llm::Llms`] corresponds to `--skip-llm`: cheap-signal order is /// used for selection and feed excerpts stand in for summaries (notes §6). - /// With only `bulk`, every editor call runs on DeepSeek (§4.2). + /// With only `bulk`, every editor call runs on the bulk provider (§4.2). pub fn new(config: Config, db: Db, llms: llm::Llms) -> Self { Self { config, db, llms } } @@ -67,15 +67,15 @@ impl Curator { assess::run( &self.db, Some(bulk), - &self.config.deepseek.model, + &bulk.model, candidates, - self.config.deepseek.deep_batch_size, - self.config.deepseek.max_concurrent_requests, + self.config.llm.deep_batch_size, + bulk.max_concurrent_requests, self.config.curation.ranking.assessment_reuse_days, rescore, profile_version, assessed_at, - self.config.deepseek.score_temperature, + self.config.llm.score_temperature, &self.config.curation.sections, ) .await @@ -137,7 +137,7 @@ impl Curator { &self.llms, lineup, &self.config.editorial, - self.config.deepseek.editorial_temperature, + self.config.llm.editorial_temperature, ) .await) } diff --git a/src/curate/profile/mod.rs b/src/curate/profile/mod.rs index 7e26577..e49a984 100644 --- a/src/curate/profile/mod.rs +++ b/src/curate/profile/mod.rs @@ -624,7 +624,7 @@ mod tests { use std::sync::Arc; use super::super::llm::{MockBackend, UsageMeter}; - use crate::config::DeepseekConfig; + use crate::config::ProviderConfig; use crate::types::{RatingEvent, TokenUsage}; let dir = tempfile::tempdir().unwrap(); @@ -667,7 +667,7 @@ mod tests { let llm = LlmClient::with_backend( "deepseek-v4-flash", initial.text, - UsageMeter::new(&DeepseekConfig::default(), 2.0), + UsageMeter::for_provider(&ProviderConfig::deepseek()), backend.clone(), ); let rebuilt = rebuild(&db, &llm, &opml, &profile_path, 60).await.unwrap(); diff --git a/src/curate/telemetry.rs b/src/curate/telemetry.rs index e32ec13..171e14d 100644 --- a/src/curate/telemetry.rs +++ b/src/curate/telemetry.rs @@ -1370,7 +1370,7 @@ mod tests { "2026-09-01", "2026-09-01T09:30:00Z", "2026-09-01T09:45:00Z", - r#"{"deepseek":{"input_tokens":1,"cached_tokens":0,"cache_write_tokens":0,"output_tokens":1,"cost_usd":0.14}}"#, + r#"{"deepseek":{"input_tokens":1,"cached_tokens":0,"cache_write_tokens":0,"output_tokens":1,"cost_usd":0.14},"gemini":{"input_tokens":1,"cached_tokens":0,"cache_write_tokens":0,"output_tokens":1,"cost_usd":0.07}}"#, ), ] { let run_id = db @@ -1452,8 +1452,9 @@ mod tests { "exploration rated positively: 1", "cost per day (anthropic): $0.043", "cost per day (deepseek): $0.020", + "cost per day (gemini): $0.005", "cost per day (voyage): $0.001", - "cost per day (total): $0.064", + "cost per day (total): $0.069", "mean generation time: 15m00s (3 runs)", ] { assert!(text.contains(line), "missing {line:?} in:\n{text}"); diff --git a/src/curate/triage.rs b/src/curate/triage.rs index 4d7f2c1..c0b03c0 100644 --- a/src/curate/triage.rs +++ b/src/curate/triage.rs @@ -500,8 +500,8 @@ pub async fn run( #[cfg(test)] mod tests { use super::*; - use crate::config::DeepseekConfig; - use crate::curate::llm::{MockBackend, UsageMeter}; + use crate::config::ProviderConfig; + use crate::curate::llm::{MockBackend, PriceTable, UsageMeter}; use crate::curate::prefilter::tests::article; use crate::curate::signals::{Neighbour, TopInterest}; use crate::types::TokenUsage; @@ -584,11 +584,11 @@ mod tests { r#"{"articles":[{"id":42,"interest":8,"kind":"essay","why":"first answer"}]}"#, TokenUsage::default(), ); - let config = DeepseekConfig::default(); + let config = ProviderConfig::deepseek(); let llm = LlmClient::with_backend( &config.model, "profile".into(), - UsageMeter::new(&config, 10.0), + UsageMeter::with_prices(PriceTable::from(&config), 10.0), backend.clone(), ); let pool = HashSet::from([42]); diff --git a/src/main.rs b/src/main.rs index 81bff95..45cc7b1 100644 --- a/src/main.rs +++ b/src/main.rs @@ -55,6 +55,15 @@ enum Command { /// Database maintenance. #[command(subcommand)] Db(DbCommand), + /// Inspect the resolved configuration. + #[command(subcommand)] + Config(ConfigCommand), +} + +#[derive(Debug, Subcommand)] +enum ConfigCommand { + /// Load and validate the config as `generate` would, then print one fact per line. + Check, } #[derive(Debug, clap::Args)] @@ -309,6 +318,14 @@ async fn main() -> Result<()> { db.migrate().await?; println!("migrations up to date: {}", config.database_path.display()); } + Command::Config(ConfigCommand::Check) => { + // Reaching here means `Config::load` already validated it; a bad + // config exited non-zero above. Nothing is opened, nothing locked. + let path = Config::resolve_path(cli.config.as_deref()); + for line in config.check_report(path.as_deref()) { + println!("{line}"); + } + } } Ok(()) } @@ -327,7 +344,8 @@ fn lock_holder(command: &Command) -> Option<&'static str> { | Command::Explain(_) | Command::Stats(_) | Command::Features(FeaturesCommand::Prune) - | Command::Db(_) => None, + | Command::Db(_) + | Command::Config(_) => None, } } @@ -463,13 +481,7 @@ fn print_lineup(issue: &daily_epub::types::Issue) { /// `profile rebuild` runs on the editor when configured, else bulk (§14.3). async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> { - use curate::llm::{Llms, PriceTable, UsageMeter}; - let bulk_meter = - UsageMeter::with_prices(PriceTable::deepseek(&config.deepseek), config.max_daily_usd); - let editor_meter = UsageMeter::with_prices( - PriceTable::anthropic(&config.anthropic), - config.anthropic.max_daily_usd, - ); + use curate::llm::{Llms, provider_meters}; let profile = curate::profile::load_or_build( db, &config.interests_opml, @@ -477,19 +489,23 @@ async fn cmd_profile_rebuild(config: &Config, db: &Db) -> Result<()> { config.curation.feedback.verdicts_in_prompt, ) .await?; - let llms = Llms::from_config( - &config.deepseek, - &config.anthropic, - profile.text, - bulk_meter, - editor_meter, - ); + let llms = Llms::from_config(config, profile.text, &provider_meters(config)); let Some(llm) = llms.editor_or_bulk() else { + let keys = config + .referenced_providers() + .iter() + .map(|(name, _)| daily_epub::config::ProviderConfig::api_key_env_var(name)) + .collect::>(); anyhow::bail!( - "no LLM provider is configured; set DAILY_EPUB_ANTHROPIC__API_KEY or DAILY_EPUB_DEEPSEEK__API_KEY" + "no LLM provider is available; assign [llm] roles and set {}", + if keys.is_empty() { + "a provider key".to_string() + } else { + keys.join(" or ") + } ); }; - tracing::info!(provider = llm.provider, model = %llm.model, "rebuilding the profile"); + tracing::info!(provider = llm.provider(), model = %llm.model, "rebuilding the profile"); let rebuilt = curate::profile::rebuild( db, llm, @@ -841,11 +857,23 @@ mod tests { vec!["ratings", "list"], vec!["db", "migrate"], vec!["features", "prune"], + vec!["config", "check"], ] { assert_eq!(lock_holder(&parse(&args)), None, "{args:?}"); } } + #[test] + fn parses_config_check() { + assert!(matches!( + Cli::try_parse_from(["daily-epub", "--config", "/etc/x.toml", "config", "check"]) + .unwrap() + .command, + Command::Config(ConfigCommand::Check) + )); + assert!(Cli::try_parse_from(["daily-epub", "config"]).is_err()); + } + #[test] fn parses_stats() { match Cli::try_parse_from(["daily-epub", "stats"]) diff --git a/src/pipeline.rs b/src/pipeline.rs index 80a4dd7..2dd4f95 100644 --- a/src/pipeline.rs +++ b/src/pipeline.rs @@ -16,8 +16,8 @@ //! * **Best effort** — social enrichment, comments, the world briefing, images and //! the XTC conversion. They log, add a warning to the report (status `degraded`) //! and the run continues. -//! * **Degrading** — every DeepSeek stage. A missing key, a dead API or a tripped -//! `max_daily_usd` guardrail turns the run into the `--skip-llm` shape +//! * **Degrading** — every LLM stage. A missing key, a dead API or a tripped +//! provider `max_daily_usd` guardrail turns the run into the `--skip-llm` shape //! (cheap-signal admission, feed excerpts as summaries) rather than //! losing the day's issue. //! @@ -33,7 +33,7 @@ use jiff::civil::Date; use jiff::{Timestamp, Zoned}; use crate::config::Config; -use crate::curate::llm::{Llms, PriceTable, UsageMeter}; +use crate::curate::llm::{Llms, UsageMeter, provider_meters}; use crate::curate::{ Curator, admit, editorial, embedding, profile, rank, signals, telemetry, triage, }; @@ -59,7 +59,7 @@ pub struct GenerateOptions { pub out: Option, /// `--max-articles N`, overriding `target_article_count`. pub max_articles: Option, - /// `--skip-llm`: no DeepSeek call at all. + /// `--skip-llm`: no chat-provider call at all. pub skip_llm: bool, /// `--skip-embeddings`: read the cache but make zero Voyage calls. pub skip_embeddings: bool, @@ -435,26 +435,23 @@ async fn run_stages( let article_embeddings = prepare_features(ctx, &mut personalized, &embeddings, report).await; // Build the provider clients before triage. A missing or failed bulk client - // skips triage and deep assessment, while the editor can still run on Claude (§17). + // skips triage and deep assessment, while the editor can still run (§17). + // One meter per referenced provider, keyed by its `[providers.*]` name and + // preloaded with what earlier runs on this UTC day already spent on it. let stage = Timestamp::now(); - let bulk_meter = - UsageMeter::with_prices(PriceTable::deepseek(&config.deepseek), config.max_daily_usd); - let editor_meter = UsageMeter::with_prices( - PriceTable::anthropic(&config.anthropic), - config.anthropic.max_daily_usd, - ); + let meters = provider_meters(config); match db.provider_spend_for_utc_day(ctx.started_at).await { Ok(spend) => { - bulk_meter.preload_cost(spend.get("deepseek").copied().unwrap_or(0.0)); - editor_meter.preload_cost(spend.get("anthropic").copied().unwrap_or(0.0)); + for (name, meter) in &meters { + meter.preload_cost(spend.get(name).copied().unwrap_or(0.0)); + } } Err(error) => { tracing::warn!(%error, "could not preload provider spend; starting from zero") } } - let llms = build_llms(ctx, &bulk_meter, &editor_meter, report).await; - let bulk_available = llms.bulk.is_some(); + let llms = build_llms(ctx, &meters, report).await; let mut curator_config = config.clone(); curator_config.target_article_count = ctx.soft_target; curator_config.curation.max_article_count = ctx.hard_max; @@ -477,13 +474,13 @@ async fn run_stages( bulk, &mut personalized, &triage_pool, - config.deepseek.triage_batch_size, - config.deepseek.max_concurrent_requests, + config.llm.triage_batch_size, + bulk.max_concurrent_requests, config.curation.ranking.assessment_reuse_days, ctx.rescore, profile_version, Timestamp::now(), - config.deepseek.score_temperature, + config.llm.score_temperature, ) .await { @@ -492,7 +489,7 @@ async fn run_stages( )); } } else { - tracing::info!("--skip-llm or DeepSeek unavailable: triage skipped"); + tracing::info!("--skip-llm or no bulk provider: triage skipped"); } report.counts.triaged = personalized .iter() @@ -661,20 +658,7 @@ async fn run_stages( .next_issue_number(date) .await .context("computing the issue number")?; - report.provider_costs.insert( - "deepseek".into(), - ProviderUsage { - usage: bulk_meter.total(), - cost_usd: bulk_meter.cost_usd(), - }, - ); - report.provider_costs.insert( - "anthropic".into(), - ProviderUsage { - usage: editor_meter.total(), - cost_usd: editor_meter.cost_usd(), - }, - ); + let llm_cost = record_provider_costs(report, &meters); // Voyage rides along in `provider_costs_json` (§7.6) so `stats` can price // it per day; its tokens are embedding input, kept out of the LLM aggregate. report.provider_costs.insert( @@ -687,31 +671,30 @@ async fn run_stages( cost_usd: report.voyage_cost_usd, }, ); - let total_cost = bulk_meter.cost_usd() + editor_meter.cost_usd() + report.voyage_cost_usd; + let total_cost = llm_cost + report.voyage_cost_usd; let summary_model = match config.editorial.summary_model { crate::config::SummaryModel::Editor if curator.llms.editor.is_some() => { - config.anthropic.model.clone() + curator.llms.editor.as_ref().map(|c| c.model.clone()) } - _ if curator.llms.bulk.is_some() => config.deepseek.model.clone(), - _ => "none".into(), - }; + _ => curator.llms.bulk.as_ref().map(|c| c.model.clone()), + } + .unwrap_or_else(|| "none".into()); let provider_costs = report .provider_costs .iter() .map(|(provider, usage)| (provider.clone(), usage.cost_usd)) .collect(); let models = Models { - bulk: if bulk_available { - config.deepseek.model.clone() - } else { - "none".into() - }, - editor: if curator.llms.editor.is_some() { - config.anthropic.model.clone() - } else if bulk_available { - format!("{} (bulk fallback)", config.deepseek.model) - } else { - "none".into() + bulk: curator + .llms + .bulk + .as_ref() + .map(|c| c.model.clone()) + .unwrap_or_else(|| "none".into()), + editor: match (&curator.llms.editor, &curator.llms.bulk) { + (Some(editor), _) => editor.model.clone(), + (None, Some(bulk)) => format!("{} (bulk fallback)", bulk.model), + (None, None) => "none".into(), }, summaries: summary_model, }; @@ -1069,15 +1052,14 @@ async fn record_issue(db: &Db, issue: &Issue, published: &Published) -> Result<( Ok(()) } -/// Build the bulk (DeepSeek) and editor (Claude) clients, running the weekly +/// Build the bulk and editor clients named in `[llm]`, running the weekly /// profile rebuild when it is due. /// /// Each client is `None` for `--skip-llm` and for every configuration/API /// problem: the pipeline then degrades per §17 instead of failing the run. async fn build_llms( ctx: &StageContext<'_>, - bulk_meter: &UsageMeter, - editor_meter: &UsageMeter, + meters: &BTreeMap, report: &mut RunReport, ) -> Llms { let profile = match profile::load_or_build( @@ -1102,15 +1084,7 @@ async fn build_llms( return Llms::default(); } - let make_clients = |prompt: String| { - Llms::from_config( - &ctx.config.deepseek, - &ctx.config.anthropic, - prompt, - bulk_meter.clone(), - editor_meter.clone(), - ) - }; + let make_clients = |prompt: String| Llms::from_config(ctx.config, prompt, meters); let mut llms = make_clients(profile.text); let Some(rebuild_client) = llms.editor_or_bulk() else { @@ -1150,22 +1124,34 @@ pub fn issue_size_bounds(config: &Config, max_articles: Option) -> (usize (config.target_article_count.min(hard_max), hard_max) } -/// Startup line naming the resolved models and whether each provider is on -/// (§19): the root config ignores unknown sections, so an `[anthropics]` or -/// `[voyages]` typo would otherwise be silent. Keys are never logged, only -/// their presence. +/// Startup lines naming each role's resolved provider and whether it is on +/// (§19): the root config ignores unknown sections, so a `[voyages]` typo +/// would otherwise be silent. Keys are never logged, only their presence. fn log_resolved_providers(config: &Config, skip_llm: bool, skip_embeddings: bool) { let has_key = |key: Option<&str>| key.is_some_and(|k| !k.trim().is_empty()); + for (role, name) in config.llm.roles() { + match config.providers.get(name) { + Some(provider) => tracing::info!( + role, + provider = name, + kind = provider.kind.as_str(), + model = %provider.model, + effort = provider.effort.as_deref().unwrap_or("-"), + enabled = !skip_llm && provider.api_key().is_some(), + key_present = provider.api_key().is_some(), + max_daily_usd = provider.max_daily_usd, + "resolved llm role" + ), + None => tracing::error!(role, provider = name, "role names an unknown provider"), + } + } + if config.llm.bulk_name().is_none() { + tracing::info!("no bulk provider: triage and deep assessment are skipped"); + } + if config.llm.editor_name().is_none() { + tracing::info!("no editor provider: editor work runs on bulk"); + } tracing::info!( - bulk_model = %config.deepseek.model, - bulk_enabled = !skip_llm && has_key(config.deepseek.api_key.as_deref()), - bulk_max_daily_usd = config.max_daily_usd, - editor_model = %config.anthropic.model, - editor_enabled = !skip_llm - && config.anthropic.enabled - && has_key(config.anthropic.api_key.as_deref()), - editor_effort = %config.anthropic.effort, - editor_max_daily_usd = config.anthropic.max_daily_usd, summary_model = ?config.editorial.summary_model, embedding_model = %config.voyage.model, embedding_enabled = !skip_embeddings @@ -1177,6 +1163,24 @@ fn log_resolved_providers(config: &Config, skip_llm: bool, skip_embeddings: bool ); } +/// Every referenced provider's usage into `report.provider_costs`, keyed by +/// its `[providers.*]` name; returns the summed LLM cost. +fn record_provider_costs(report: &mut RunReport, meters: &BTreeMap) -> f64 { + let mut total = 0.0; + for (name, meter) in meters { + let cost_usd = meter.cost_usd(); + total += cost_usd; + report.provider_costs.insert( + name.clone(), + ProviderUsage { + usage: meter.total(), + cost_usd, + }, + ); + } + total +} + /// Prompt versions recorded per run so old telemetry stays interpretable (§7.6). /// Bump a number when the corresponding instruction block changes. const PROMPT_VERSIONS: &[(&str, u32)] = &[ @@ -1189,13 +1193,17 @@ const PROMPT_VERSIONS: &[(&str, u32)] = &[ ]; /// The resolved `[curation]` (ranking included), `[editorial]`, `[voyage]`, -/// model names and prompt versions written to `runs.config_json` (§7.6, §19). -/// Never includes keys. +/// `[llm]`, the provider registry, model names and prompt versions written to +/// `runs.config_json` (§7.6, §19). Never includes keys. fn resolved_run_config(config: &Config, soft_target: usize, hard_max: usize) -> serde_json::Value { let mut curation = config.curation.clone(); curation.max_article_count = hard_max; let mut voyage = config.voyage.clone(); voyage.api_key = None; + let model_of = |role: Option<(&str, &crate::config::ProviderConfig)>| { + role.map(|(_, provider)| provider.model.clone()) + .unwrap_or_else(|| "disabled".into()) + }; serde_json::json!({ "target_article_count": soft_target, "TRIAGE_PROMPT_VERSION": triage::TRIAGE_PROMPT_VERSION, @@ -1203,10 +1211,14 @@ fn resolved_run_config(config: &Config, soft_target: usize, hard_max: usize) -> "curation": curation, "editorial": config.editorial, "voyage": voyage, + "llm": config.llm, + "providers": config.providers_redacted(), "models": { - "bulk": config.deepseek.model, - "editor": if config.anthropic.enabled { config.anthropic.model.as_str() } else { "disabled" }, - "editor_effort": config.anthropic.effort, + "bulk": model_of(config.bulk_provider()), + "editor": model_of(config.editor_provider()), + "editor_effort": config + .editor_provider() + .and_then(|(_, provider)| provider.effort.clone()), "embedding": if config.voyage.enabled { config.voyage.model.as_str() } else { "disabled" }, }, "prompt_versions": PROMPT_VERSIONS @@ -1277,8 +1289,9 @@ mod tests { #[test] fn run_config_json_records_the_resolved_settings_and_no_keys() { let mut config = Config::default(); - config.anthropic.api_key = Some("sk-secret".into()); - config.deepseek.api_key = Some("ds-secret".into()); + for provider in config.providers.values_mut() { + provider.api_key = Some("sk-secret".into()); + } config.voyage.api_key = Some("pa-secret".into()); let value = resolved_run_config(&config, 6, 6); assert_eq!(value["target_article_count"], 6); @@ -1296,6 +1309,19 @@ mod tests { assert_eq!(value["editorial"]["summary_input_tokens"], 3000); assert_eq!(value["models"]["bulk"], "deepseek-v4-flash"); assert_eq!(value["models"]["editor"], "claude-opus-5"); + assert_eq!(value["models"]["editor_effort"], "high"); + assert_eq!(value["llm"]["bulk"], "deepseek"); + assert_eq!(value["llm"]["editor"], "anthropic"); + assert_eq!(value["llm"]["triage_batch_size"], 25); + assert_eq!(value["providers"]["gemini"]["kind"], "openai"); + assert_eq!(value["providers"]["anthropic"]["max_daily_usd"], 3.0); + for provider in value["providers"].as_object().expect("providers") { + assert!( + provider.1["api_key"].is_null(), + "{} leaked its key", + provider.0 + ); + } assert!(value["prompt_versions"]["editor"].is_number()); assert_eq!( value["TRIAGE_PROMPT_VERSION"], @@ -1314,6 +1340,44 @@ mod tests { !text.contains("secret"), "keys must never reach the database" ); + + let mut config = Config::default(); + config.llm.editor.clear(); + let value = resolved_run_config(&config, 6, 6); + assert_eq!(value["models"]["editor"], "disabled"); + assert!(value["models"]["editor_effort"].is_null()); + } + + /// `provider_costs` is keyed by whatever the operator named the providers, + /// never by a hard-coded "deepseek" / "anthropic". + #[test] + fn provider_costs_are_keyed_by_the_configured_provider_names() { + let mut config = Config::default(); + let bulk = config.providers.remove("deepseek").expect("deepseek"); + config.providers.insert("bulkprov".into(), bulk); + config.llm.bulk = "bulkprov".into(); + config.llm.editor = "gemini".into(); + config.validate().expect("renamed provider validates"); + + let meters = provider_meters(&config); + assert_eq!( + meters.keys().collect::>(), + vec!["bulkprov", "gemini"] + ); + meters["bulkprov"].record(TokenUsage { + input_tokens: 1_000_000, + ..TokenUsage::default() + }); + let mut report = RunReport::new(run_date(), now()); + let total = record_provider_costs(&mut report, &meters); + assert_eq!( + report.provider_costs.keys().collect::>(), + vec!["bulkprov", "gemini"] + ); + assert!(!report.provider_costs.contains_key("deepseek")); + assert!((report.provider_costs["bulkprov"].cost_usd - 0.14).abs() < 1e-9); + assert_eq!(report.provider_costs["gemini"].cost_usd, 0.0); + assert!((total - 0.14).abs() < 1e-9); } #[test] @@ -1602,14 +1666,14 @@ mod tests { assert_eq!(thin, "{}"); // Admission replaces the old prefilter and carries retriever telemetry. - // DeepSeek is "down": the bulk client exists but every call fails, so + // The bulk provider is "down": the client exists but every call fails, so // the deep set is ranked on present signals and the editor falls back // to utility order (§17). let bulk_backend = Arc::new(ChatMockBackend::new()); let bulk = LlmClient::with_backend( - &h.config.deepseek.model, + &h.config.providers["deepseek"].model, "SYSTEM".into(), - UsageMeter::new(&h.config.deepseek, h.config.max_daily_usd), + UsageMeter::for_provider(&h.config.providers["deepseek"]), bulk_backend.clone(), ); let curator = Curator::new( diff --git a/src/world.rs b/src/world.rs index 42d9b88..df3cd49 100644 --- a/src/world.rs +++ b/src/world.rs @@ -718,11 +718,14 @@ mod tests { backend: std::sync::Arc, limit: f64, ) -> LlmClient { - let config = crate::config::DeepseekConfig::default(); + let config = crate::config::ProviderConfig::deepseek(); LlmClient::with_backend( "mock", "World Briefing test".into(), - crate::curate::llm::UsageMeter::new(&config, limit), + crate::curate::llm::UsageMeter::with_prices( + crate::curate::llm::PriceTable::from(&config), + limit, + ), backend, ) } diff --git a/tests/config_check.rs b/tests/config_check.rs new file mode 100644 index 0000000..57a4b3e --- /dev/null +++ b/tests/config_check.rs @@ -0,0 +1,69 @@ +//! `daily-epub config check` end to end: the built binary, a temp config, no +//! environment. It must validate like `generate`, print the provider table +//! with `key MISSING` warnings, exit 0, and exit non-zero on an invalid file — +//! all without a database or the run lock. + +use std::path::Path; +use std::process::Command; + +fn run(config_body: &str) -> (i32, String, String) { + let dir = tempfile::tempdir().expect("tempdir"); + let path = dir.path().join("config.toml"); + std::fs::write(&path, config_body).expect("write config"); + let output = Command::new(env!("CARGO_BIN_EXE_daily-epub")) + .env_clear() + .env("PATH", std::env::var_os("PATH").unwrap_or_default()) + .args(["--config"]) + .arg(&path) + .args(["config", "check"]) + .output() + .expect("run daily-epub"); + ( + output.status.code().unwrap_or(-1), + String::from_utf8_lossy(&output.stdout).into_owned(), + String::from_utf8_lossy(&output.stderr).into_owned(), + ) +} + +#[test] +fn config_check_prints_the_facts_and_exits_zero_without_keys() { + let example = Path::new(env!("CARGO_MANIFEST_DIR")).join("config.example.toml"); + let body = std::fs::read_to_string(example).expect("example config"); + let (code, stdout, stderr) = run(&body); + assert_eq!(code, 0, "stdout:\n{stdout}\nstderr:\n{stderr}"); + for needle in [ + "config: ", + "database_path: /var/lib/daily-epub/daily-epub.db", + "profile_path: data/profile.md", + "interests_opml: data/scour-interests.opml", + "llm.bulk: deepseek · openai · deepseek-v4-flash", + "key MISSING (set DAILY_EPUB_PROVIDERS__DEEPSEEK__API_KEY)", + "llm.editor: anthropic · anthropic · claude-opus-5 · effort high · max_daily_usd $3.00", + "key MISSING (set DAILY_EPUB_PROVIDERS__ANTHROPIC__API_KEY)", + "providers.gemini: unreferenced", + "voyage: voyage-4-lite · enabled · max_daily_usd $0.50 · key MISSING (set DAILY_EPUB_VOYAGE__API_KEY)", + "editorial.summary_model: editor", + "publish.epub_dir: /srv/bookorbit/libraries/daily-epub", + "publish.xtc_dir: /var/lib/daily-epub/xtc", + ] { + assert!(stdout.contains(needle), "missing {needle:?} in:\n{stdout}"); + } + assert!( + stdout.lines().any(|line| line.starts_with("! ")), + "missing keys are flagged with a `!` prefix:\n{stdout}" + ); + // No lock file, no database: the command is read-only. + assert!(!Path::new("/var/lib/daily-epub/daily-epub.db.lock").exists()); +} + +#[test] +fn config_check_exits_non_zero_on_an_invalid_config() { + let (code, stdout, stderr) = run("[llm]\nbulk = \"nope\"\n"); + assert_ne!(code, 0); + assert!(stdout.is_empty(), "{stdout}"); + assert!(stderr.contains("providers.nope"), "{stderr}"); + + let (code, _, stderr) = run("[deepseek]\nmodel = \"x\"\n"); + assert_ne!(code, 0); + assert!(stderr.contains("[providers.deepseek]"), "{stderr}"); +} diff --git a/tests/e2e_pipeline.rs b/tests/e2e_pipeline.rs index 0e184a7..01f3dd1 100644 --- a/tests/e2e_pipeline.rs +++ b/tests/e2e_pipeline.rs @@ -570,9 +570,9 @@ async fn llm_pipeline_runs_against_a_mock_backend() { usage, ); - let meter = UsageMeter::new(&cfg.deepseek, cfg.max_daily_usd); + let meter = UsageMeter::for_provider(&cfg.providers["deepseek"]); let llm = LlmClient::with_backend( - &cfg.deepseek.model, + &cfg.providers["deepseek"].model, "You are the editor of The Daily EPUB.".into(), meter.clone(), backend.clone(), @@ -642,9 +642,9 @@ async fn llm_pipeline_runs_against_a_mock_backend() { let colophon = Colophon { provider_costs: BTreeMap::from([("deepseek".to_string(), meter.cost_usd())]), models: Models { - bulk: cfg.deepseek.model.clone(), - editor: format!("{} (bulk fallback)", cfg.deepseek.model), - summaries: cfg.deepseek.model.clone(), + bulk: cfg.providers["deepseek"].model.clone(), + editor: format!("{} (bulk fallback)", cfg.providers["deepseek"].model), + summaries: cfg.providers["deepseek"].model.clone(), }, entries_fetched: 8, feeds_seen: 8, @@ -655,7 +655,7 @@ async fn llm_pipeline_runs_against_a_mock_backend() { let mut lineup = lineup; pipeline::apply_summaries(&mut lineup, &editorial_doc); let issue = assemble_build_publish(&db, &cfg, lineup, colophon).await; - assert_eq!(issue.colophon.models.bulk, cfg.deepseek.model); + assert_eq!(issue.colophon.models.bulk, cfg.providers["deepseek"].model); assert!(issue.colophon.cost_usd > 0.0); } @@ -682,9 +682,9 @@ async fn failing_deepseek_still_publishes_with_heuristic_fallbacks() { let backend = std::sync::Arc::new(MockBackend::new()); let client = LlmClient::with_backend( - &cfg.deepseek.model, + &cfg.providers["deepseek"].model, "reader profile".into(), - UsageMeter::new(&cfg.deepseek, cfg.max_daily_usd), + UsageMeter::for_provider(&cfg.providers["deepseek"]), backend.clone(), ); let curator = Curator::new( @@ -732,9 +732,9 @@ async fn failing_deepseek_still_publishes_with_heuristic_fallbacks() { Colophon { provider_costs: BTreeMap::new(), models: Models { - bulk: cfg.deepseek.model.clone(), - editor: format!("{} (bulk fallback)", cfg.deepseek.model), - summaries: cfg.deepseek.model.clone(), + bulk: cfg.providers["deepseek"].model.clone(), + editor: format!("{} (bulk fallback)", cfg.providers["deepseek"].model), + summaries: cfg.providers["deepseek"].model.clone(), }, entries_fetched: 8, feeds_seen: 8,