Fill in the dashboard read pages of the web dashboard plan (§9): the
overview card set (last run from report_json, budget today per provider,
ratings this week, unrated picks with the rating widget, jobs summary,
config-on-disk warnings), the runs list and run detail (funnel, admission
mix, preference state, timings, provider usage, warnings, top feeds,
config diff against the previous non-dry run, near misses, and the
candidates table with allow-listed filters/sorts and per-row signals),
and the articles list and article detail (assessments including
provider rejections, run history, neighbours and interests, embedding
metadata, rating events, and the text explain).
Dynamic list queries go through AssertSqlSafe with SQL built only from
constants and allow-listed fragments; every user value is bound. Adds
the shared signals-table, candidate-row and pager partials, a CSS block
and the data-filter table behaviour in app.js, plus the step 3 handoff.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM
Add /dashboard/ratings with the preference-state header, the "How ratings
enter the algorithm" explainer, the Current tab (decayed neighbour weight,
feed credit, prompt/rebuild membership, last-run neighbour usage, inline
rating widget with note) and the paginated, filterable Events tab with
superseded marking.
Add /dashboard/profile: profile.md editor with atomic mode-preserving
writes and profile_versions history plus restore, the parsed preview,
OPML interests by theme, learned adjustments with staleness, the stored
system prompt, and the profile-rebuild job form.
Make profile::MAX_RATINGS_IN_REBUILD and profile::stored_version pub.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM
Signed-in issue page with the Brief, downloads, index and colophon; article,
World Briefing and Behind the paper pages under the login guard; the rating
widget with its fetch enhancement; the admin-only POST /rate writing
dashboard-attributed rating events; the HMAC confirmation page links to the
site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM
Migration 0004 (users, sessions, config_changes, profile_versions, jobs,
rating_events.user_id, runs.report_json, issues.issue_json), the issue
snapshot writer and loader, the web module skeleton with layout and static
assets, axum-login/tower-sessions over a sqlx session store, password-auth
users with a CLI, the login throttle, the origin check, security headers,
and the public issue pages, archive, Atom feed and robots.txt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM
DeepSeek's content filter rejects a whole request (400 "Content Exists
Risk") when one article trips it, which cost the other articles in the
batch their assessment and retried them every run. New curate/batch.rs
runs both stages through a bisecting runner: a rejected batch is split
until the offending article is isolated, that article is retried once on
the editor provider when it is a different one, and a still-rejected
article is recorded as a provider_rejected assessment row so it is not
retried for assessment_reuse_days. Cache reuse accepts rows from either
configured model. Each stage logs reused/requested/rejected counts, the
curation: line shows rejections when non-zero, explain prints them, and
the llm_assess span reports the deep-set size.
Implemented by a Claude agent from an orchestrator brief; verified
fmt/clippy(-W dead_code)/test green (354 lib tests).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
[llm] assigns the bulk and editor roles by name over a [providers.*]
registry (kind = openai | anthropic, per-provider model, effort, daily
ceiling and price table); DeepseekBackend becomes OpenAiCompatibleBackend
(reasoning_effort passthrough), AnthropicBackend builds from the same
ProviderConfig, meters and provider_costs are keyed by provider name.
Gemini 3.8 Flash is declared via Google's OpenAI-compatible endpoint so
switching the editor is one line (or DAILY_EPUB_LLM__EDITOR=gemini for an
A/B dry run). Stale [deepseek]/[anthropic] tables, the top-level
max_daily_usd and the old key env vars fail loudly.
daily-epub config check validates and prints the resolved roles, models,
key presence and paths without opening the database.
docs/runbooks/curation-v2-migration.md walks the server upgrade from v1.
Registry implemented by a Claude agent from an orchestrator brief;
verified fmt/clippy(-W dead_code)/test green.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
Dead code and stale v1 comments removed (clippy -W dead_code clean, the
three world.rs warnings fixed), the Brief chapter's TOC title renamed from
"From the Editor", features prune now also sweeps article_assessments and
generate runs the sweep once after publishing, the example config is
tested key-for-key against Config::default(), README commands match
--help, and docs/plans/2026-08-15-implementation-notes.md records the
Anthropic and Voyage facts, the new tables, the budget-day rule and the
lock.
Implemented by a Claude agent from docs/plans/curation-v2-briefs/step7.md.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
Behind-the-paper chapter (behind.xhtml, both editions) built from the run's
StageCounts and candidate_runs near misses; daily-epub stats [--days N];
StageCounts gains knn/feed gates and verdicts_in_prompt, timings split
into summaries + brief, the four-line §15.4 info block logged once per
run and printed by print_report; src/lock.rs flock guard on
<database_path>.lock for generate, profile rebuild, features backfill and
backfill-social; README updated for the new CLI, env vars and costs.
Implemented by a Claude agent from docs/plans/curation-v2-briefs/step6.md;
reviewed against plan §5, §15.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
assess.rs replaces score.rs (DEEP_INSTRUCTIONS, representative sample
with [BEGINNING]/[MIDDLE]/[END], facets, cached deep rows with --rescore
bypass), rank.rs adds the utility blend over present signals with gate
ramps and the leader-clustered shortlist (cap 2 → 3 → uncapped, protected
top-N, exploration reserve), editor.rs replaces select.rs with the §13
rendering and utility-ordered fallbacks. ScoredArticle is gone; Candidate
is the only flow type. deep_batch_size replaces score_batch_size.
Started by Codex (cut off by its usage limit mid-verification) and
finished by a Claude agent from docs/plans/curation-v2-briefs/step5.md;
reviewed against plan §12–§13.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
DeepSeek triage over every eligible article (plan §10) with cached
assessments in article_assessments, the union admission with quotas and
exploration slots (§11), hygiene moved to admit.rs with the churn rule
reading assessments, prefilter.rs reduced to hygiene and text heuristic,
prefilter_keep removed in favour of curation.ranking.deep_keep, the
scores table dropped (migration 0003), and --rescore on generate.
Implemented by Codex (gpt-5.4, high effort) from
docs/plans/curation-v2-briefs/step4.md; reviewed against plan §10–§11.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
Union merge of the Claude editor (step 2) and the embeddings/signals/telemetry
work (step 3): both config trees, per-provider LLM budgets plus the Voyage
meter, one startup provider log, runs.config_json with ranking and voyage
settings, and selected picks' why lines landing in candidate_runs.editor_why.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
- embedding.rs: EmbeddingBackend + VoyageBackend, batched bounded-concurrency
client with its own UsageMeter, f32 BLOB codec, article/interest embedding
cache keyed by model, dimension and sha256 of the embedded text.
- signals.rs: z-scored interest match, decayed rated-neighbour preference
with the knn gate, feed affinity with the feed gate, social, text heuristic
without social terms, mid-rank percentile normalizer, preliminary blend.
- telemetry.rs: candidate_runs writer with §7.5 signals_json, explain and
near-misses renderers, prune.
- [voyage] and the full [curation.ranking] config with validation.
- CLI: explain, features backfill|prune, generate --skip-embeddings.
- Pipeline: hygiene rows, embed and signals stages before the old prefilter;
same-date regeneration no longer excludes its own picks.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
- AnthropicBackend (Messages API, cached system block, output_config.effort,
server-side fallbacks, refusal surfaced as an error); Llms { bulk, editor }
with editor_or_bulk(); PriceTable-based UsageMeter per provider.
- [anthropic], [editorial], deepseek.max_concurrent_requests and
curation.max_article_count config; startup logs resolved providers.
- Budget day is the UTC date of started_at, preloaded from
runs.provider_costs_json; finish_run writes provider_costs_json and
config_json. Stage A batches run concurrently with per-batch budget checks.
- Editor prompt with one-line "why" per pick; no minimum lineup size;
--max-articles is a ceiling; top-up branch deleted; why stored on picks and
issue_articles.why and rendered in chapters and In this issue.
- Summaries on the editor client (3k-token input, concurrency 4, bulk then
excerpt fallback); "The Brief" replaces From the Editor; section intros gone.
- Colophon carries per-provider costs and models.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
- Migration 0002: rating_events, article_embeddings, interest_embeddings,
article_assessments, candidate_runs, runs.config_json/provider_costs_json,
issue_articles.why; copies ratings into rating_events and drops ratings and
feed_priors (scores stays until step 4).
- Vote is Loved | Good | NotForMe; legacy `up` links still verify as Loved.
- Footer offers Loved it / Good / Not for me; the confirmation page offers the
other two so a mis-tap can be corrected. handle_rating appends one event.
- db::current_ratings implements the latest-explicit-event rule with summaries
and facets; `ratings list|set|clear` CLI appends source='cli' events.
- data/profile.md replaces the hard-coded reader prose; the system prompt is
rebuilt every run in the §8.4 order with a recent-verdicts block.
- Weekly rebuild reads summaries, facets and notes; feed priors removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
- Download cap raised from 5 MiB to 100 MiB in src/images/fetch.rs:14.
- Added 16,384×16,384 source dimension limits.
- Added an explicit 512 MiB per-image decoder allocation limit in
src/images/encode.rs:15.
- Existing output resizing and 25 MiB final issue budget remain
unchanged.
- Added regression coverage for a large image source size and oversized
image dimensions.
- Improved overflow safety when accumulating streamed image bytes.
The image fixes left two problems of navigation. Image handling was
spread across `extract.rs` (300 lines of normalization) and
`epub/images.rs` (download, re-encode, markup rewriting, plus generic
HTML helpers that comments, world and x4 were all reaching into a
module named "images" to borrow). And `epub/build.rs` had grown to
1,148 lines of code covering cover rendering, ten askama templates,
every chapter renderer and the zip assembly.
New homes:
- `src/html.rs` — markup helpers that do not care what the markup is
about: tag scanning, attribute parsing, entity decoding, escaping,
XHTML fixups, reading a fragment as text. Previously scattered between
`epub/images.rs` and `extract.rs`.
- `src/images/` — one module per stage of an article's images, in the
order they run: `normalize` (make `<img>` usable, pre-readability),
`refs` (what an article references), `fetch` + `encode` (download and
re-encode per edition), `embed` (point the markup at what shipped).
- `src/epub/{cover,chapters,build}.rs` — the cover, the chapter
renderers, and the ordering plus assembly that puts them together.
`epub/fixtures.rs` takes the shared test issue, which was a public
module wedged inside `build.rs`.
- `src/curate/profile/themes.rs` — a 260-line keyword table that sat in
the middle of the profile logic.
`curate::html_to_text` is renamed `prompt_text`: it is a different
function from `html::html_to_text` (collapses whitespace, no DOM, sized
for prompt budgets) and sharing a name with it was a trap.
Largest module drops from 1,148 code lines to 828, and no file mixes
two subjects. Behaviour is unchanged: 231 lib tests plus 25 integration
tests green, and the real-world audit over issues 1–3 still reports 214
images referenced, 214 shown, 0 placeholders, 0 orphaned assets.
`image_audit` gains `--epub-out DIR`, which writes a readable EPUB of
the audited articles so images can be checked on a device.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Across the first three issues, 145 of 208 referenced images reached the
page; 38 that had been downloaded, re-encoded and written into the EPUB
were never referenced by any chapter, and 69 became `[image: …]` lines.
Six independent causes, each verified against the real articles:
- **Entity-encoded URLs never matched their asset.** `prepare_body`
re-runs `ammonia::clean`, which writes `&` as `&`, and the tag
scanner that looks the asset up compared raw attribute text against a
URL a real parser had decoded. Every image with a query string lost.
`parse_attrs` now decodes entities.
- **Readability clobbers a working `src`.** Its lazy-image heuristic
copies any attribute containing `.jpg` over `src`, which on NPR meant
a `{width}` template (the CDN answers those with a grey square reading
"Image"), on Substack a JSON blob, on dfarq an entire `srcset` string.
Pages are now normalized before readability sees them: every `<img>`
is reduced to `src`/`alt`/`title` with the best candidate from the
lazy attributes, `src`, `srcset` and `<picture><source>`, so there is
nothing left for the heuristic to substitute. Candidates that cannot
resolve — braces, whitespace, quotes, `data:` — are rejected by shape
rather than by publisher.
- **Readability deletes `<button>` and its subtree**, taking lightbox
images with it and leaving the captions behind. Image-only wrappers
are unwrapped first.
- **SVG was undecodable**, so vector charts became placeholders. They
are rasterized with resvg, which the cover already depends on.
- **Relative URLs resolved against the pre-redirect URL**, 404ing every
image on an article reached through a shortener. `fetch_readable` now
reports where it landed.
- **The 12-image cap** silently truncated photo essays, and the images
past it were advertised as failures. Removed; the issue-wide byte
budget is the real backstop.
An image that still cannot be embedded is now dropped rather than
announced, unless its alt text is a real description — decorative rules,
spacers and dead links were generating most of the placeholder noise.
`examples/image_audit.rs` replays the pipeline over the articles of
published issues and reports what reaches the page. On issues 1–3:
208→214 images referenced, 145→214 shown, 69→0 placeholders, 38→0
orphaned assets. The three images still not embedded are a 14×14 favicon
and a 650×2 divider — correctly declined, and no longer announced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- World Briefing now starts with yesterday, preserves complete nested
event hierarchies and Wikipedia links, fetches deduplicated article
leads concurrently, and adds stable-ID summaries plus a daily overview.
- Enrichment runs after normal editorial work and degrades safely with
report warnings.
- Colophon facts now use X4-safe block paragraphs.
- Standard cover remains 1200×1600 RGB PNG.
- X4 cover is a 480×800 baseline RGB JPEG with consistent XHTML and OPF
declarations.
- Added CrossInk cache-clearing guidance to README.md.
- Worked around an epub-builder duplicate XML-ID defect discovered by
validation.
It turns out it is currently impossible for the Xteink X4 to download
XTC files from an OPDS server. So I switched the built-in OPDS server in
this binary (daily.hallada.net/opds) to serve the EPUB files instead of
the XTC files. The X4 is pretty capable of reading the X4 edition of the
EPUB that was optimized for it anyways, and I prefer the flexibility of
EPUB, so I might end up eventually deleting the XTC conversion. For now,
I kept the conversion step (in case I ever want to ever try manually
copying them to the device) and I keep a limited number of XTC issues on
the server since they are quite big in filesize. The BookOrbit
integration is now optional since the built-in OPDS server is able to
serve the EPUBs. In my installation, I serve the EPUBs through both. The
daily.hallada.net/opds server is just a little quicker to navigate and
download the EPUBs on the X4 since the BookOrbit OPDS requires diving
into a couple layers of folders before you get to the files.
Full implementation of a personalized daily newspaper delivered as an
EPUB.
Articles are pulled from a local self-hosted Miniflux instance, enriched
with comments, summarized and filtered by DeepSeek AI, and then
assembled into two EPUB editions: standard and optimized for the Xteink
X4 e-ink reader. Both are served by the local self-hosted BookOrbit OPDS
server in a separate library. Then the X4 edition is futher converted to
XTC format and served over a separate OPDS server hosted by the Rust
binary. Runs are tracked in a local SQLite database so runs are
idempotent per date.
Full documentation of the plan is in docs/plans and setup and install
instructions are in the README.md file.
thalladaandClaude Fable 5.1