Fix the six ways article images went missing
Across the first three issues, 145 of 208 referenced images reached the
page; 38 that had been downloaded, re-encoded and written into the EPUB
were never referenced by any chapter, and 69 became `[image: …]` lines.
Six independent causes, each verified against the real articles:
- **Entity-encoded URLs never matched their asset.** `prepare_body`
re-runs `ammonia::clean`, which writes `&` as `&`, and the tag
scanner that looks the asset up compared raw attribute text against a
URL a real parser had decoded. Every image with a query string lost.
`parse_attrs` now decodes entities.
- **Readability clobbers a working `src`.** Its lazy-image heuristic
copies any attribute containing `.jpg` over `src`, which on NPR meant
a `{width}` template (the CDN answers those with a grey square reading
"Image"), on Substack a JSON blob, on dfarq an entire `srcset` string.
Pages are now normalized before readability sees them: every `<img>`
is reduced to `src`/`alt`/`title` with the best candidate from the
lazy attributes, `src`, `srcset` and `<picture><source>`, so there is
nothing left for the heuristic to substitute. Candidates that cannot
resolve — braces, whitespace, quotes, `data:` — are rejected by shape
rather than by publisher.
- **Readability deletes `<button>` and its subtree**, taking lightbox
images with it and leaving the captions behind. Image-only wrappers
are unwrapped first.
- **SVG was undecodable**, so vector charts became placeholders. They
are rasterized with resvg, which the cover already depends on.
- **Relative URLs resolved against the pre-redirect URL**, 404ing every
image on an article reached through a shortener. `fetch_readable` now
reports where it landed.
- **The 12-image cap** silently truncated photo essays, and the images
past it were advertised as failures. Removed; the issue-wide byte
budget is the real backstop.
An image that still cannot be embedded is now dropped rather than
announced, unless its alt text is a real description — decorative rules,
spacers and dead links were generating most of the placeholder noise.
`examples/image_audit.rs` replays the pipeline over the articles of
published issues and reports what reaches the page. On issues 1–3:
208→214 images referenced, 145→214 shown, 69→0 placeholders, 38→0
orphaned assets. The three images still not embedded are a 14×14 favicon
and a 650×2 divider — correctly declined, and no longer announced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6ba3c3d525
commit
254eaeb713
5 files changed
+1351
-42
No files matched your search
+7
-3
@@ -451,7 +451,7 @@ fn encode_cover(pixmap: tiny_skia::Pixmap, edition: Edition) -> Result<Vec<u8>,
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Prepare article markup for XHTML: rewrite images, sanitize, self-close voids.
|
||||
fn prepare_body(html: &str, images_: &[ImageAsset]) -> String {
|
||||
pub fn prepare_body(html: &str, images_: &[ImageAsset]) -> String {
|
||||
// Sanitize first: image rewriting emits our own trusted markup (including the
|
||||
// `image-placeholder` class, which ammonia would otherwise strip).
|
||||
let cleaned = ammonia::clean(html);
|
||||
@@ -1013,7 +1013,7 @@ pub mod fixtures {
|
||||
title: title.to_string(),
|
||||
best_entry_id: entry_id,
|
||||
content_html: format!(
|
||||
"<p>Body of <em>{title}</em> with an image.</p><img src=\"https://img.example/{entry_id}.png\" alt=\"A chart\"><p>More words & things.</p>"
|
||||
"<p>Body of <em>{title}</em> with an image.</p><img src=\"https://img.example/{entry_id}.png\" alt=\"A chart of the daily figures\"><p>More words & things.</p>"
|
||||
),
|
||||
word_count: 1200,
|
||||
excerpt_only: false,
|
||||
@@ -1307,7 +1307,11 @@ mod tests {
|
||||
assert!(chapter.xhtml.contains("Read online"));
|
||||
assert!(chapter.xhtml.contains("href=\"disc-1001.xhtml\""));
|
||||
// The un-downloaded image degrades to a placeholder.
|
||||
assert!(chapter.xhtml.contains("[image: A chart]"));
|
||||
assert!(
|
||||
chapter
|
||||
.xhtml
|
||||
.contains("[image: A chart of the daily figures]")
|
||||
);
|
||||
assert_xml_ok(&chapter.xhtml);
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user