Break up the image and EPUB god files

The image fixes left two problems of navigation. Image handling was
spread across `extract.rs` (300 lines of normalization) and
`epub/images.rs` (download, re-encode, markup rewriting, plus generic
HTML helpers that comments, world and x4 were all reaching into a
module named "images" to borrow). And `epub/build.rs` had grown to
1,148 lines of code covering cover rendering, ten askama templates,
every chapter renderer and the zip assembly.

New homes:

- `src/html.rs` — markup helpers that do not care what the markup is
  about: tag scanning, attribute parsing, entity decoding, escaping,
  XHTML fixups, reading a fragment as text. Previously scattered between
  `epub/images.rs` and `extract.rs`.
- `src/images/` — one module per stage of an article's images, in the
  order they run: `normalize` (make `<img>` usable, pre-readability),
  `refs` (what an article references), `fetch` + `encode` (download and
  re-encode per edition), `embed` (point the markup at what shipped).
- `src/epub/{cover,chapters,build}.rs` — the cover, the chapter
  renderers, and the ordering plus assembly that puts them together.
  `epub/fixtures.rs` takes the shared test issue, which was a public
  module wedged inside `build.rs`.
- `src/curate/profile/themes.rs` — a 260-line keyword table that sat in
  the middle of the profile logic.

`curate::html_to_text` is renamed `prompt_text`: it is a different
function from `html::html_to_text` (collapses whitespace, no DOM, sized
for prompt budgets) and sharing a name with it was a trap.

Largest module drops from 1,148 code lines to 828, and no file mixes
two subjects. Behaviour is unchanged: 231 lib tests plus 25 integration
tests green, and the real-world audit over issues 1–3 still reports 214
images referenced, 214 shown, 0 placeholders, 0 orphaned assets.

`image_audit` gains `--epub-out DIR`, which writes a readable EPUB of
the audited articles so images can be checked on a device.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
thalladaandClaude Opus 5 committed 2026-08-18 03:35:21 +00:00
1 parent 254eaeb713
commit db19d08257
27 files changed
+3513 -3132

No files matched your search

+3 -3
View File
@@ -14,7 +14,7 @@ use std::fmt::Write as _;
use serde::{Deserialize, Serialize};
use super::llm::{LlmClient, LlmError};
use super::{escape_html, html_to_text, text_to_paragraphs, truncate_tokens, truncate_words};
use super::{escape_html, prompt_text, text_to_paragraphs, truncate_tokens, truncate_words};
use crate::types::{ArticleId, Editorial, Lineup, Pick};
/// Article text is truncated to roughly this many tokens per summary call (§3.6).
@@ -116,7 +116,7 @@ pub async fn summarize_article(
temperature: f32,
) -> Result<String, LlmError> {
llm.meter.check_budget()?;
let body = truncate_tokens(&html_to_text(body_html), SUMMARY_INPUT_TOKEN_BUDGET);
let body = truncate_tokens(&prompt_text(body_html), SUMMARY_INPUT_TOKEN_BUDGET);
let mut prompt = String::with_capacity(body.len() + SUMMARY_INSTRUCTIONS.len() + 256);
prompt.push_str(SUMMARY_INSTRUCTIONS);
let _ = write!(
@@ -309,7 +309,7 @@ fn social_note(pick: &Pick) -> String {
/// The article's own opening words, used when no LLM summary exists (§3.6).
pub fn excerpt_summary(pick: &Pick) -> String {
let text = truncate_words(
&html_to_text(&pick.article.content_html),
&prompt_text(&pick.article.content_html),
FALLBACK_SUMMARY_WORDS,
);
if text.is_empty() {