Fix the six ways article images went missing

Across the first three issues, 145 of 208 referenced images reached the
page; 38 that had been downloaded, re-encoded and written into the EPUB
were never referenced by any chapter, and 69 became `[image: …]` lines.
Six independent causes, each verified against the real articles:

- **Entity-encoded URLs never matched their asset.** `prepare_body`
  re-runs `ammonia::clean`, which writes `&` as `&`, and the tag
  scanner that looks the asset up compared raw attribute text against a
  URL a real parser had decoded. Every image with a query string lost.
  `parse_attrs` now decodes entities.
- **Readability clobbers a working `src`.** Its lazy-image heuristic
  copies any attribute containing `.jpg` over `src`, which on NPR meant
  a `{width}` template (the CDN answers those with a grey square reading
  "Image"), on Substack a JSON blob, on dfarq an entire `srcset` string.
  Pages are now normalized before readability sees them: every `<img>`
  is reduced to `src`/`alt`/`title` with the best candidate from the
  lazy attributes, `src`, `srcset` and `<picture><source>`, so there is
  nothing left for the heuristic to substitute. Candidates that cannot
  resolve — braces, whitespace, quotes, `data:` — are rejected by shape
  rather than by publisher.
- **Readability deletes `<button>` and its subtree**, taking lightbox
  images with it and leaving the captions behind. Image-only wrappers
  are unwrapped first.
- **SVG was undecodable**, so vector charts became placeholders. They
  are rasterized with resvg, which the cover already depends on.
- **Relative URLs resolved against the pre-redirect URL**, 404ing every
  image on an article reached through a shortener. `fetch_readable` now
  reports where it landed.
- **The 12-image cap** silently truncated photo essays, and the images
  past it were advertised as failures. Removed; the issue-wide byte
  budget is the real backstop.

An image that still cannot be embedded is now dropped rather than
announced, unless its alt text is a real description — decorative rules,
spacers and dead links were generating most of the placeholder noise.

`examples/image_audit.rs` replays the pipeline over the articles of
published issues and reports what reaches the page. On issues 1–3:
208→214 images referenced, 145→214 shown, 69→0 placeholders, 38→0
orphaned assets. The three images still not embedded are a 14×14 favicon
and a 650×2 divider — correctly declined, and no longer announced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
thalladaandClaude Opus 5 committed 2026-08-18 02:51:28 +00:00
1 parent 6ba3c3d525
commit 254eaeb713
5 files changed
+1351 -42

No files matched your search

+7 -3
View File
@@ -451,7 +451,7 @@ fn encode_cover(pixmap: tiny_skia::Pixmap, edition: Edition) -> Result<Vec<u8>,
// ---------------------------------------------------------------------------
/// Prepare article markup for XHTML: rewrite images, sanitize, self-close voids.
fn prepare_body(html: &str, images_: &[ImageAsset]) -> String {
pub fn prepare_body(html: &str, images_: &[ImageAsset]) -> String {
// Sanitize first: image rewriting emits our own trusted markup (including the
// `image-placeholder` class, which ammonia would otherwise strip).
let cleaned = ammonia::clean(html);
@@ -1013,7 +1013,7 @@ pub mod fixtures {
title: title.to_string(),
best_entry_id: entry_id,
content_html: format!(
"<p>Body of <em>{title}</em> with an image.</p><img src=\"https://img.example/{entry_id}.png\" alt=\"A chart\"><p>More words &amp; things.</p>"
"<p>Body of <em>{title}</em> with an image.</p><img src=\"https://img.example/{entry_id}.png\" alt=\"A chart of the daily figures\"><p>More words &amp; things.</p>"
),
word_count: 1200,
excerpt_only: false,
@@ -1307,7 +1307,11 @@ mod tests {
assert!(chapter.xhtml.contains("Read online"));
assert!(chapter.xhtml.contains("href=\"disc-1001.xhtml\""));
// The un-downloaded image degrades to a placeholder.
assert!(chapter.xhtml.contains("[image: A chart]"));
assert!(
chapter
.xhtml
.contains("[image: A chart of the daily figures]")
);
assert_xml_ok(&chapter.xhtml);
}