Commit Graph
18 Commits
Author SHA1 Message Date
thallada 31d2690701 Merge branch 'index-author' 2026-09-09 18:12:38 +00:00
thalladaandClaude Fable 5.1 1a009a2113 Name the author ahead of the feed on issue index lines
Every list of articles now reads "Author · Feed · Publication · N min":
the signed-in issue page, the public issue page, the RSS feed and the
EPUB's In This Issue page. `author_label` drops an author that is empty
or only repeats the feed's title, as a dozen personal blogs do. Article
pages keep their own byline.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VXYGBPoHZSDSfE5WcJ9bvj
2026-09-09 18:12:34 +00:00
thalladaandClaude Fable 5.1 c538afa540 Unwrap Scour's /r/rss/ redirector when canonicalizing URLs
Scour interest feeds link every entry through
https://scour.ing/r/rss/<percent-encoded article URL>, so the wrapper
became the article's canonical URL: the same post arriving via Hacker
News and Scour never merged, and the publication fallback showed
"scour.ing" instead of the site's own domain. Resolve the wrapper like
the Google News redirector. `publication_label` re-canonicalizes the
stored URL so the 477 articles saved with the wrapper show the real
domain too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VXYGBPoHZSDSfE5WcJ9bvj
2026-09-09 18:12:30 +00:00
thalladaandClaude Fable 5.1 f71a4c2799 Keep the publication an article's page declares
Readability already reads og:site_name and JSON-LD publisher.name; carry
that through Page and Extracted and store it in articles.publication, the
same way the page byline is kept. `publication_label` is what the readers
will see after the feed name: the site name, else the domain, and nothing
when it would only repeat the feed's own title.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YWmCpUojfHXhSZ2129Z7Nv
2026-09-09 05:48:46 +00:00
thalladaandClaude Fable 5.1 b02e1c7b8c Add an AI slop verdict that downrates the author's future articles
A fourth explicit verdict, "AI slop" (Vote::Slop, label `slop`), joins the
rating widget on the web, the EPUB footer links, the /r/ confirmation page,
the CLI, imports and the dashboard filters. It counts as a full negative
(curation.feedback.slop_value, -1.0) in the neighbour and affinity signals.

Beyond that, each run loads the authors whose current verdict is slop, with
no lookback, and multiplies the preliminary blend and the utility of every
candidate by that author by 1 - curation.ranking.slop_author_penalty (0.75),
so they sink before triage. The flag is recorded in signals_json, shown by
`explain` and the dashboard signals table, and the confirmation names the
author (or says no author is known, in which case only the rating applies).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VXYGBPoHZSDSfE5WcJ9bvj
2026-09-09 03:36:50 +00:00
thalladaandClaude Fable 5.1 bedbcb218f Show what the pipeline understood about each article
A muted one-line note now follows every article: the deep assessment's
topic group, format, depth and technicality, the specific topics it
extracted, and the reader interests the article matched best, e.g.

  Software engineering · analysis essay · in depth · highly technical ·
  Topics: copy-on-write, ZFS · Interests: Filesystems, Rust

It renders on the web article page footer, on the public and private
issue-page entries, in the EPUB article footer and in the EPUB "In This
Issue" index (both editions), always styled like the existing meta lines
so it stays secondary to the content.

`Pick` gains `top_interests` (serde default, no migration), filled from
the signals stage when the editor assembles the lineup; the facets were
already on the issue snapshot via `Pick.llm`, so earlier issues show the
facet and topic parts and only future issues carry interests. One shared
builder, `understanding_line` in epub::chapters, feeds every surface.

The dev seed gives four picks facets and interests so the line can be
eyeballed locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QVPagF6jfDv78CC5Jv2wp4
2026-09-08 19:44:00 +00:00
thalladaandClaude Fable 5.1 7ce8c65939 Take the author from the article page, not the aggregator submitter
Readability already finds a byline (JSON-LD author, meta author tags,
byline markup); keep it on the article and let it replace an aggregator
entry's author, which for HN and friends is the submitter. A direct feed's
own author is still trusted over the page, and an aggregator name is only
used at all when no direct feed carried the story. The author is stored on
articles so it survives independently of the best entry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QVPagF6jfDv78CC5Jv2wp4
2026-09-07 18:34:21 +00:00
thalladaandClaude Fable 5.1 9aff78ebf1 Add the Read in BookOrbit button and /issues/{date}/read redirect
The signed-in issue page shows a Read in BookOrbit button (only when the
integration is active and the Standard EPUB exists) that hits
/issues/{date}/read. The route redirects to the cached BookOrbit reader
URL, or resolves the ids through OPDS on first click and caches them;
?refresh=1 re-resolves. Not indexed yet is a 503, upstream failures 502.
Includes the implementation plan.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5eMEmWEnjMXBsBob5FDW
2026-09-05 05:20:34 +00:00
thalladaandClaude Fable 5.1 69cd736455 Web dashboard step 1: migration, sessions, users CLI, public site
Migration 0004 (users, sessions, config_changes, profile_versions, jobs,
rating_events.user_id, runs.report_json, issues.issue_json), the issue
snapshot writer and loader, the web module skeleton with layout and static
assets, axum-login/tower-sessions over a sqlx session store, password-auth
users with a CLI, the login throttle, the origin check, security headers,
and the public issue pages, archive, Atom feed and robots.txt.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NHyYupFdBiR4VfoUM7NjSM
2026-09-03 04:38:22 +00:00
thalladaandClaude Fable 5.1 d403c51edf Curation v2 step 7: cleanup, prune paths, implementation notes
Dead code and stale v1 comments removed (clippy -W dead_code clean, the
three world.rs warnings fixed), the Brief chapter's TOC title renamed from
"From the Editor", features prune now also sweeps article_assessments and
generate runs the sweep once after publishing, the example config is
tested key-for-key against Config::default(), README commands match
--help, and docs/plans/2026-08-15-implementation-notes.md records the
Anthropic and Voyage facts, the new tables, the budget-day rule and the
lock.

Implemented by a Claude agent from docs/plans/curation-v2-briefs/step7.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
2026-09-02 16:32:18 +00:00
thalladaandClaude Fable 5.1 d261cd485d Curation v2 step 6: Behind the paper, stats, run report block, lock
Behind-the-paper chapter (behind.xhtml, both editions) built from the run's
StageCounts and candidate_runs near misses; daily-epub stats [--days N];
StageCounts gains knn/feed gates and verdicts_in_prompt, timings split
into summaries + brief, the four-line §15.4 info block logged once per
run and printed by print_report; src/lock.rs flock guard on
<database_path>.lock for generate, profile rebuild, features backfill and
backfill-social; README updated for the new CLI, env vars and costs.

Implemented by a Claude agent from docs/plans/curation-v2-briefs/step6.md;
reviewed against plan §5, §15.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
2026-09-02 16:21:06 +00:00
thalladaandClaude Fable 5.1 05a74a0dcf Curation v2 step 5: deep assessment, utility, diversified shortlist
assess.rs replaces score.rs (DEEP_INSTRUCTIONS, representative sample
with [BEGINNING]/[MIDDLE]/[END], facets, cached deep rows with --rescore
bypass), rank.rs adds the utility blend over present signals with gate
ramps and the leader-clustered shortlist (cap 2 → 3 → uncapped, protected
top-N, exploration reserve), editor.rs replaces select.rs with the §13
rendering and utility-ordered fallbacks. ScoredArticle is gone; Candidate
is the only flow type. deep_batch_size replaces score_batch_size.

Started by Codex (cut off by its usage limit mid-verification) and
finished by a Claude agent from docs/plans/curation-v2-briefs/step5.md;
reviewed against plan §12–§13.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
2026-09-02 16:04:43 +00:00
thalladaandClaude Fable 5.1 10f4afd091 Curation v2 step 4: triage replaces the gate
DeepSeek triage over every eligible article (plan §10) with cached
assessments in article_assessments, the union admission with quotas and
exploration slots (§11), hygiene moved to admit.rs with the churn rule
reading assessments, prefilter.rs reduced to hygiene and text heuristic,
prefilter_keep removed in favour of curation.ranking.deep_keep, the
scores table dropped (migration 0003), and --rescore on generate.

Implemented by Codex (gpt-5.4, high effort) from
docs/plans/curation-v2-briefs/step4.md; reviewed against plan §10–§11.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
2026-09-02 15:25:04 +00:00
thalladaandClaude Fable 5.1 57efbb49b4 Curation v2 step 2: Claude Opus 5 editor, the Brief, per-provider budgets
- AnthropicBackend (Messages API, cached system block, output_config.effort,
  server-side fallbacks, refusal surfaced as an error); Llms { bulk, editor }
  with editor_or_bulk(); PriceTable-based UsageMeter per provider.
- [anthropic], [editorial], deepseek.max_concurrent_requests and
  curation.max_article_count config; startup logs resolved providers.
- Budget day is the UTC date of started_at, preloaded from
  runs.provider_costs_json; finish_run writes provider_costs_json and
  config_json. Stage A batches run concurrently with per-batch budget checks.
- Editor prompt with one-line "why" per pick; no minimum lineup size;
  --max-articles is a ceiling; top-up branch deleted; why stored on picks and
  issue_articles.why and rendered in chapters and In this issue.
- Summaries on the editor client (3k-token input, concurrency 4, bulk then
  excerpt fallback); "The Brief" replaces From the Editor; section intros gone.
- Colophon carries per-provider costs and models.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
2026-09-02 03:56:14 +00:00
thalladaandClaude Fable 5.1 3a9f4b99e0 Curation v2 step 1: three-way feedback, rating events, reader profile
- Migration 0002: rating_events, article_embeddings, interest_embeddings,
  article_assessments, candidate_runs, runs.config_json/provider_costs_json,
  issue_articles.why; copies ratings into rating_events and drops ratings and
  feed_priors (scores stays until step 4).
- Vote is Loved | Good | NotForMe; legacy `up` links still verify as Loved.
- Footer offers Loved it / Good / Not for me; the confirmation page offers the
  other two so a mis-tap can be corrected. handle_rating appends one event.
- db::current_ratings implements the latest-explicit-event rule with summaries
  and facets; `ratings list|set|clear` CLI appends source='cli' events.
- data/profile.md replaces the hard-coded reader prose; the system prompt is
  rebuilt every run in the §8.4 order with a recent-verdicts block.
- Weekly rebuild reads summaries, facets and notes; feed priors removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
2026-09-02 03:23:39 +00:00
thallada 38fb493989 Fix world briefing & add LLM enrichment, colophon fix, x4 cover fix
- World Briefing now starts with yesterday, preserves complete nested
event hierarchies and Wikipedia links, fetches deduplicated article
leads concurrently, and adds stable-ID summaries plus a daily overview.
- Enrichment runs after normal editorial work and degrades safely with
report warnings.
- Colophon facts now use X4-safe block paragraphs.
- Standard cover remains 1200×1600 RGB PNG.
- X4 cover is a 480×800 baseline RGB JPEG with consistent XHTML and OPF
declarations.
- Added CrossInk cache-clearing guidance to README.md.
- Worked around an epub-builder duplicate XML-ID defect discovered by
validation.
2026-08-15 21:13:56 +00:00
thallada 9e27a32fb6 Serve EPUBs instead XTC in daily OPDS
It turns out it is currently impossible for the Xteink X4 to download
XTC files from an OPDS server. So I switched the built-in OPDS server in
this binary (daily.hallada.net/opds) to serve the EPUB files instead of
the XTC files. The X4 is pretty capable of reading the X4 edition of the
EPUB that was optimized for it anyways, and I prefer the flexibility of
EPUB, so I might end up eventually deleting the XTC conversion. For now,
I kept the conversion step (in case I ever want to ever try manually
copying them to the device) and I keep a limited number of XTC issues on
the server since they are quite big in filesize. The BookOrbit
integration is now optional since the built-in OPDS server is able to
serve the EPUBs. In my installation, I serve the EPUBs through both. The
daily.hallada.net/opds server is just a little quicker to navigate and
download the EPUBs on the X4 since the BookOrbit OPDS requires diving
into a couple layers of folders before you get to the files.
2026-08-15 20:24:20 +00:00
thallada 9e30c1dcdf Initial commit: The Daily EPUB full implementation
Full implementation of a personalized daily newspaper delivered as an
EPUB.

Articles are pulled from a local self-hosted Miniflux instance, enriched
with comments, summarized and filtered by DeepSeek AI, and then
assembled into two EPUB editions: standard and optimized for the Xteink
X4 e-ink reader. Both are served by the local self-hosted BookOrbit OPDS
server in a separate library. Then the X4 edition is futher converted to
XTC format and served over a separate OPDS server hosted by the Rust
binary. Runs are tracked in a local SQLite database so runs are
idempotent per date.

Full documentation of the plan is in docs/plans and setup and install
instructions are in the README.md file.
2026-08-15 17:46:19 +00:00