Settle on a five-minute public cache and drop the purge rationale

The edge-purge integration was reverted in the previous commit; this
finishes the job. Public pages, the feed and /issues.json now say
`public, max-age=300` for browsers and shared caches alike, the code
comments and README explain why the longer s-maxage plus purge design
was dropped, and the rollout runbook loses its purge step, marks the
HTML cache rule optional, and warns to keep the record DNS-only until
the Universal SSL certificate is active.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5eMEmWEnjMXBsBob5FDW
This commit is contained in:
2026-09-04 22:04:07 +00:00
co-authored by Claude Fable 5.1
parent de332cd4b3
commit f634f06bf1
6 changed files with 57 additions and 99 deletions
+8 -13
View File
@@ -135,7 +135,6 @@ daily-epub features prune # stale embeddings, old telemetry and assessment
daily-epub backfill-social [--days 7] # re-poll social scores for recent articles daily-epub backfill-social [--days 7] # re-poll social scores for recent articles
daily-epub db migrate # run migrations (also automatic on every start) daily-epub db migrate # run migrations (also automatic on every start)
daily-epub config check # validate the config, print the resolved roles, keys and paths daily-epub config check # validate the config, print the resolved roles, keys and paths
daily-epub cdn purge # purge the CDN edge cache for the configured zone
daily-epub users add USER [--admin] [--password-stdin] daily-epub users add USER [--admin] [--password-stdin]
daily-epub users passwd USER [--password-stdin] daily-epub users passwd USER [--password-stdin]
daily-epub users role USER user|admin daily-epub users role USER user|admin
@@ -370,10 +369,6 @@ prints what resolved.
| `server.login_window_minutes` | `15` | Length of the login throttle window. | | `server.login_window_minutes` | `15` | Length of the login throttle window. |
| `server.jobs_enabled` | `true` | Allow the dashboard to start the fixed systemd job catalogue. | | `server.jobs_enabled` | `true` | Allow the dashboard to start the fixed systemd job catalogue. |
| `server.journal_lines` | `300` | Journal lines shown on a dashboard job page (105000). | | `server.journal_lines` | `300` | Journal lines shown on a dashboard job page (105000). |
| `cdn.provider` | unset | `cloudflare`, or unset for no CDN integration. Setting it requires the zone id and the token. |
| `cdn.cloudflare_zone_id` | unset | Zone id from the Cloudflare dashboard overview. |
| `cdn.api_token` | — | **`DAILY_EPUB_CDN__API_TOKEN`**. Needs exactly one permission: `Zone → Cache Purge`, scoped to that zone. |
| `cdn.purge_after_publish` | `true` | Purge the whole edge cache after `generate` publishes. A failure is logged, never fatal; dry runs never purge. |
`[curation.ranking]` holds the ranker's tunables. The learned signals are `[curation.ranking]` holds the ranker's tunables. The learned signals are
gated: `knn` (rated-neighbour preference) ramps from `knn_floor` (8) to gated: `knn` (rated-neighbour preference) ramps from `knn_floor` (8) to
@@ -591,7 +586,7 @@ respect it (runbook §2). The whole matrix:
| route | `Cache-Control` | | route | `Cache-Control` |
|---|---| |---|---|
| `/`, `/issues`, `/issues/{date}`, `/feed.xml`, `/issues.json` (anonymous) | `public, max-age=300, s-maxage=86400` | | `/`, `/issues`, `/issues/{date}`, `/feed.xml`, `/issues.json` (anonymous) | `public, max-age=300` |
| the same pages with a `daily_session=` cookie | `private, no-store` | | the same pages with a `daily_session=` cookie | `private, no-store` |
| `/robots.txt` | `public, max-age=86400` | | `/robots.txt` | `public, max-age=86400` |
| `/static/*?v=<hash>` | `public, max-age=31536000, immutable` | | `/static/*?v=<hash>` | `public, max-age=31536000, immutable` |
@@ -599,13 +594,13 @@ respect it (runbook §2). The whole matrix:
| `/files/epub/*`, `/files/xtc/*`, `/opds*` (every status) | `private, no-store` | | `/files/epub/*`, `/files/xtc/*`, `/opds*` (every status) | `private, no-store` |
| `/dashboard*`, `/login`, `/account`, errors, anything else | `no-store` | | `/dashboard*`, `/login`, `/account`, errors, anything else | `no-store` |
The five-minute browser age and the one-day edge age are the same header doing Five minutes is the whole freshness story: an issue changes once a day, a
two jobs: a reader's tab revalidates soon after a new issue lands, while the reader's tab or a CDN edge revalidates within five minutes of the new one
edge is allowed to answer for a whole day because `generate` purges it the landing, and signed-in requests never touch a shared cache at all. A longer
moment it publishes (`[cdn]`, `daily-epub cdn purge`). The `no-store` on the edge age with an API purge after each publish was considered and dropped as
last row is a default applied by the `security_headers` middleware to any not worth its moving parts. The `no-store` on the last row is a default applied
response that set no policy of its own, so a route added later cannot silently by the `security_headers` middleware to any response that set no policy of its
inherit the CDN's default TTL. own, so a route added later cannot silently inherit the CDN's default TTL.
### Installing the XTC converter ### Installing the XTC converter
+35 -72
View File
@@ -1,9 +1,9 @@
# Runbook — putting daily.hallada.net behind Cloudflare # Runbook — putting daily.hallada.net behind Cloudflare
**Written:** 2026-09-04 for the production host. Steps 12 happen in the Cloudflare **Written:** 2026-09-04 for the production host. Steps 13 happen in the Cloudflare
and registrar dashboards; steps 36 are on the server as an operator with `sudo`. and registrar dashboards; steps 45 are on the server as an operator with `sudo`.
The origin-side changes (the `Cache-Control` matrix and `daily-epub cdn purge`) The origin-side changes (the `Cache-Control` matrix below) ship in the same
ship in the same release, so deploy the new binary before step 5. release; deploy the new binary before enabling the proxy.
Read the whole thing before starting step 1: a nameserver change is the one step Read the whole thing before starting step 1: a nameserver change is the one step
that cannot be undone in seconds. that cannot be undone in seconds.
@@ -14,7 +14,7 @@ The app decides what is cacheable; Cloudflare is configured only to obey it.
| route | `Cache-Control` | | route | `Cache-Control` |
|---|---| |---|---|
| `/`, `/issues`, `/issues/{date}`, `/feed.xml`, `/issues.json` (anonymous) | `public, max-age=300, s-maxage=86400` | | `/`, `/issues`, `/issues/{date}`, `/feed.xml`, `/issues.json` (anonymous) | `public, max-age=300` |
| the same pages with a `daily_session=` cookie | `private, no-store` | | the same pages with a `daily_session=` cookie | `private, no-store` |
| `/robots.txt` | `public, max-age=86400` | | `/robots.txt` | `public, max-age=86400` |
| `/static/*?v=<hash>` | `public, max-age=31536000, immutable` | | `/static/*?v=<hash>` | `public, max-age=31536000, immutable` |
@@ -24,10 +24,12 @@ The app decides what is cacheable; Cloudflare is configured only to obey it.
Two facts follow from that table and matter for every choice below: Two facts follow from that table and matter for every choice below:
- The edge is allowed to hold a public page for a **day** (`s-maxage=86400`). - A public page is fresh for **five minutes** everywhere: browser and edge alike
That is only correct because `generate` purges the whole zone right after it revalidate within five minutes of a new issue landing, and nothing has to tell
publishes. If the purge is broken, the site serves yesterday's paper until the the edge that the day changed. (A one-day edge age with an API purge after
day elapses. Step 5 is not optional. every publish was tried and dropped: at this traffic an edge rarely keeps a
page long enough for the longer age to matter, and the purge was one more
token, config section and failure mode.)
- Anything gated by a cookie or Basic auth already says `private, no-store`, so - Anything gated by a cookie or Basic auth already says `private, no-store`, so
even a misconfigured cache-everything rule cannot make a download public. The even a misconfigured cache-everything rule cannot make a download public. The
cookie bypass rule in step 3 is defence in depth, not the only defence. cookie bypass rule in step 3 is defence in depth, not the only defence.
@@ -61,7 +63,13 @@ host. Everything else in the zone must survive the move unchanged.
4. Change the nameservers at the registrar to the two Cloudflare assigns. 4. Change the nameservers at the registrar to the two Cloudflare assigns.
Propagation is usually minutes and can be hours. Cloudflare emails when the Propagation is usually minutes and can be hours. Cloudflare emails when the
zone goes active. zone goes active. **Keep `daily` DNS-only until the zone is Active *and*
SSL/TLS → Edge Certificates shows the Universal certificate as Active.** A
proxied host with no edge certificate yet fails every TLS handshake
(`SSL alert number 40`) while plain HTTP still redirects, which is exactly
what happened on the first attempt: readers whose resolvers had picked up
the new nameservers got a broken site. Set Full (strict) (step 2) before
flipping the record to Proxied.
5. Before the switch, drop the TTL on the `daily` record (60 s) so a rollback is 5. Before the switch, drop the TTL on the `daily` record (60 s) so a rollback is
quick. Verify afterwards: quick. Verify afterwards:
@@ -84,7 +92,7 @@ Under the zone's **SSL/TLS**, **Speed** and **Scrape Shield** sections:
This one is easy to miss and quietly wrong: the default (**4 hours**) *raises* This one is easy to miss and quietly wrong: the default (**4 hours**) *raises*
the `max-age` Cloudflare sends to browsers, so the deliberate 5-minute browser the `max-age` Cloudflare sends to browsers, so the deliberate 5-minute browser
age on the public pages would become 4 hours and a reader's tab would show a age on the public pages would become 4 hours and a reader's tab would show a
stale paper long after a purge. stale paper for most of a morning.
- **Speed → Optimization → Rocket Loader: OFF.** - **Speed → Optimization → Rocket Loader: OFF.**
- **Scrape Shield → Email Address Obfuscation: OFF.** - **Scrape Shield → Email Address Obfuscation: OFF.**
Both inject a Cloudflare-hosted script into the HTML. The site's CSP is Both inject a Cloudflare-hosted script into the HTML. The site's CSP is
@@ -111,7 +119,13 @@ The origin already sends `private, no-store` to a cookie-bearing request, but
this makes the bypass a property of the request rather than of the response, so this makes the bypass a property of the request rather than of the response, so
nothing is ever *looked up* in a shared cache for a signed-in reader. nothing is ever *looked up* in a shared cache for a signed-in reader.
**Rule 2 — "Cache by origin headers"** **Rule 2 — "Cache by origin headers"** *(optional)*
Without this rule Cloudflare caches only its default static extensions (the
CSS, JS, fonts and SVG here) and passes HTML straight through, which already
captures most of the benefit: TLS terminates at the edge and the origin is
reached over a warm connection. Add the rule if you want the HTML itself
served from the edge; it is only safe together with rule 1.
- When incoming requests match: `http.host eq "daily.hallada.net"` - When incoming requests match: `http.host eq "daily.hallada.net"`
- Then: **Eligible for cache** - Then: **Eligible for cache**
@@ -119,7 +133,7 @@ nothing is ever *looked up* in a shared cache for a signed-in reader.
- **Browser TTL:** *Respect origin* - **Browser TTL:** *Respect origin*
That Edge TTL mode is the whole point of the origin work: a response with That Edge TTL mode is the whole point of the origin work: a response with
`s-maxage` is cached for exactly that long, and a response with `no-store` (or `max-age` is cached for exactly that long, and a response with `no-store` (or
one that somehow arrives with no policy at all) is not cached. It is the reason one that somehow arrives with no policy at all) is not cached. It is the reason
the `security_headers` middleware defaults unknown routes to `no-store` — with the `security_headers` middleware defaults unknown routes to `no-store` — with
this mode, "no header" means "do not cache" rather than "cache for 2 hours". this mode, "no header" means "do not cache" rather than "cache for 2 hours".
@@ -166,60 +180,11 @@ Check a login attempt from a phone on cellular and one from the LAN land in
different throttle buckets, and that `journalctl -u daily-epub` shows real client different throttle buckets, and that `journalctl -u daily-epub` shows real client
addresses rather than Cloudflare's. addresses rather than Cloudflare's.
## 5. Configure the purge ## 5. Verify
Create the API token in the Cloudflare dashboard: **My Profile → API Tokens → Anonymous public page — with rule 2, expect `HIT` on the second request (the
Create Token → Custom token**, with the single permission **Zone → Cache Purge → first fills the edge); without it, `DYNAMIC`. Either way the origin's own
Purge**, **Zone Resources → Include → Specific zone → hallada.net**. Nothing `Cache-Control` must come through unchanged:
else — the app makes exactly one API call. Copy the token once; it is not shown
again. The zone id is on the zone's **Overview** page.
In `/etc/daily-epub/config.toml`:
```toml
[cdn]
provider = "cloudflare"
cloudflare_zone_id = "…"
purge_after_publish = true
```
and in the systemd environment file (the one the units already load, mode `0600`,
owned by `daily-epub`):
```
DAILY_EPUB_CDN__API_TOKEN=…
```
Setting `provider` without both the zone id and the token is a **config error**:
the app refuses to start rather than publishing into a stale edge. Confirm and
then purge by hand:
```sh
sudo systemctl restart daily-epub
sudo -u daily-epub daily-epub --config /etc/daily-epub/config.toml config check | grep '^cdn'
# cdn: cloudflare · zone … · token present · purge_after_publish true
sudo -u daily-epub daily-epub --config /etc/daily-epub/config.toml cdn purge
# purged the whole cloudflare cache (id …)
```
`cdn purge` takes no run lock and touches neither the database nor the publish
directories, so it is safe to run at any time, including during a `generate`.
From then on `generate` purges the whole zone after every successful publish. The
purge is *purge everything* on purpose: a new issue also changes the previous
issue's page (its "latest" nav marker moves), `/issues`, `/feed.xml` and
`/issues.json`, and a per-URL list of that set would rot. A purge failure is
logged at `warn` and never fails the run — so check for it after the first live
run:
```sh
journalctl -u daily-epub-generate.service --since today | grep -i 'purge'
```
## 6. Verify
Anonymous public page — expect `HIT` on the second request (the first fills the
edge) and the origin's own two-part `Cache-Control`:
```sh ```sh
curl -sI https://daily.hallada.net/ | grep -iE 'cf-cache-status|cache-control|age' curl -sI https://daily.hallada.net/ | grep -iE 'cf-cache-status|cache-control|age'
@@ -261,10 +226,9 @@ curl -sI https://daily.hallada.net/login | grep -iE 'cf-cache-status|cache-contr
# cache-control: no-store # cache-control: no-store
``` ```
Finally, the end-to-end check the whole exercise is about: run a `generate`, then Finally, the morning after: within five minutes of the `generate` timer firing,
immediately `curl -sI https://daily.hallada.net/ | grep -i cf-cache-status` and `curl -s https://daily.hallada.net/ | grep -o 'issues/[0-9-]*' | head -1` shows
confirm it reads `MISS` (the purge emptied the edge) and that the page shows the the new date from a network that is not signed in.
new issue.
## Rollback ## Rollback
@@ -274,8 +238,7 @@ new issue.
- **Something worse:** set the `daily` DNS record back to **DNS only** (grey - **Something worse:** set the `daily` DNS record back to **DNS only** (grey
cloud). Traffic goes straight to the origin again within the record's TTL, and cloud). Traffic goes straight to the origin again within the record's TTL, and
nothing about the origin's behaviour depends on Cloudflare being there — the nothing about the origin's behaviour depends on Cloudflare being there — the
`Cache-Control` headers are correct without it, and `cdn.provider` can be `Cache-Control` headers are correct without it.
removed from the config at leisure.
- **Full retreat:** point the registrar's nameservers back at Route 53. The - **Full retreat:** point the registrar's nameservers back at Route 53. The
hosted zone still exists unless it was deleted; do not delete it until the hosted zone still exists unless it was deleted; do not delete it until the
Cloudflare setup has run for a few weeks. Cloudflare setup has run for a few weeks.
+1 -1
View File
@@ -304,7 +304,7 @@ async fn handle_issues_json(State(state): State<AppState>) -> Response {
}) })
.collect(); .collect();
match serde_json::to_string_pretty(&issues) { match serde_json::to_string_pretty(&issues) {
// Only publishing changes this, and publishing purges the edge (§3.12). // Only publishing changes this; five minutes of staleness is fine (§3.12).
Ok(body) => ( Ok(body) => (
StatusCode::OK, StatusCode::OK,
[ [
+4 -4
View File
@@ -1671,7 +1671,7 @@ mod tests {
assert_eq!(issue.status(), StatusCode::OK); assert_eq!(issue.status(), StatusCode::OK);
assert_eq!( assert_eq!(
issue.headers().get(header::CACHE_CONTROL).unwrap(), issue.headers().get(header::CACHE_CONTROL).unwrap(),
"public, max-age=300, s-maxage=86400" "public, max-age=300"
); );
let html = String::from_utf8( let html = String::from_utf8(
to_bytes(issue.into_body(), 1024 * 1024) to_bytes(issue.into_body(), 1024 * 1024)
@@ -1705,7 +1705,7 @@ mod tests {
assert_eq!(archive.status(), StatusCode::OK); assert_eq!(archive.status(), StatusCode::OK);
assert_eq!( assert_eq!(
archive.headers().get(header::CACHE_CONTROL).unwrap(), archive.headers().get(header::CACHE_CONTROL).unwrap(),
"public, max-age=300, s-maxage=86400" "public, max-age=300"
); );
let feed = app let feed = app
@@ -1724,7 +1724,7 @@ mod tests {
); );
assert_eq!( assert_eq!(
feed.headers().get(header::CACHE_CONTROL).unwrap(), feed.headers().get(header::CACHE_CONTROL).unwrap(),
"public, max-age=300, s-maxage=86400" "public, max-age=300"
); );
let feed = String::from_utf8( let feed = String::from_utf8(
to_bytes(feed.into_body(), 1024 * 1024) to_bytes(feed.into_body(), 1024 * 1024)
@@ -1777,7 +1777,7 @@ mod tests {
.unwrap(); .unwrap();
assert_eq!( assert_eq!(
reports.headers().get(header::CACHE_CONTROL).unwrap(), reports.headers().get(header::CACHE_CONTROL).unwrap(),
"public, max-age=300, s-maxage=86400" "public, max-age=300"
); );
let reports = let reports =
String::from_utf8(to_bytes(reports.into_body(), 4096).await.unwrap().to_vec()).unwrap(); String::from_utf8(to_bytes(reports.into_body(), 4096).await.unwrap().to_vec()).unwrap();
+1 -1
View File
@@ -771,7 +771,7 @@ mod tests {
assert!(anonymous.headers().get(header::SET_COOKIE).is_none()); assert!(anonymous.headers().get(header::SET_COOKIE).is_none());
assert_eq!( assert_eq!(
anonymous.headers().get(header::CACHE_CONTROL).unwrap(), anonymous.headers().get(header::CACHE_CONTROL).unwrap(),
"public, max-age=300, s-maxage=86400" "public, max-age=300"
); );
assert_eq!( assert_eq!(
anonymous.headers().get("speculation-rules").unwrap(), anonymous.headers().get("speculation-rules").unwrap(),
+8 -8
View File
@@ -360,14 +360,14 @@ pub async fn robots() -> Response {
/// `Cache-Control` for an anonymous public page (§3.12). /// `Cache-Control` for an anonymous public page (§3.12).
/// ///
/// The two ages are aimed at two different caches. `max-age=300` is the /// Five minutes, for browsers and shared caches alike. An issue changes once a
/// browser's: a reader who leaves a tab open revalidates within five minutes of /// day, so the only staleness this allows is the few minutes after a publish,
/// a new issue landing. `s-maxage=86400` is the CDN's: an issue changes once a /// and the operator never sees even that: signed-in requests get
/// day, so the edge should be allowed to answer for a day rather than asking /// [`PRIVATE_CACHE`]. A longer shared-cache age was tried and rejected: it
/// the origin every five minutes. That long edge life is only safe because /// needs a purge call after every publish to stay correct, and at this site's
/// [`crate::cdn::purge_all`] runs right after a publish; without the purge the /// traffic an edge rarely keeps a page long enough for the extra age to pay
/// edge would keep yesterday's paper for its whole day. /// for that machinery.
pub const PUBLIC_CACHE: &str = "public, max-age=300, s-maxage=86400"; pub const PUBLIC_CACHE: &str = "public, max-age=300";
/// `Cache-Control` for anything a signed-in reader sees, and for every /// `Cache-Control` for anything a signed-in reader sees, and for every
/// authenticated download. `private` keeps it out of shared caches even if a /// authenticated download. `private` keeps it out of shared caches even if a