thallada/the-daily-epub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document the CDN rollout and refresh the reverse-proxy section

Browse Source 
The README's nginx snippet had drifted from the box: it was missing http2, the
security-headers include, the gzip block, the keepalive bump and the trusted
certificate. Replace it with what actually runs, then add the two CDN changes.

`set_real_ip_from` for Cloudflare's ranges plus `real_ip_header
CF-Connecting-IP` has to come first, or every request looks like it came from
Cloudflare and the login throttle becomes global. Then `X-Forwarded-For` is
*set* from `$remote_addr` rather than appended to: the throttle keys on the
first entry, so a client-supplied header must never survive into the app.
The ranges are listed for reference but the reader is told to regenerate them
from cloudflare.com/ips-v4 and /ips-v6, with a one-liner that does it.

The gzip comment claimed the stylesheet carries 445 KB of base64 fonts; it has
not since the faces moved back out to their own URLs.

The new runbook covers the order that matters: the DNS move (Free plan means a
full nameserver change, so the rest of the zone has to survive it), the zone
settings that are quietly wrong by default — Browser Cache TTL *raises* the
origin's max-age unless set to "Respect Existing Headers", and Rocket Loader and
Email Obfuscation both inject scripts the site's `script-src 'self'` CSP blocks
— the two cache rules, locking the origin to Cloudflare, wiring the purge token,
and the `cf-cache-status` checks that prove each row of the header matrix.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Va5eMEmWEnjMXBsBob5FDW
This commit is contained in:
thallada
2026-09-04 21:29:10 +00:00
co-authored by Claude Fable 5.1
parent 7d12f53314
commit e616638c36
2 changed files with 396 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+115 -11
View File
@@ -135,6 +135,7 @@ daily-epub features prune # stale embeddings, old telemetry and assessment
daily-epub backfill-social [--days 7] # re-poll social scores for recent articles daily-epub backfill-social [--days 7] # re-poll social scores for recent articles
daily-epub db migrate # run migrations (also automatic on every start) daily-epub db migrate # run migrations (also automatic on every start)
daily-epub config check # validate the config, print the resolved roles, keys and paths daily-epub config check # validate the config, print the resolved roles, keys and paths
daily-epub cdn purge # purge the CDN edge cache for the configured zone
daily-epub users add USER [--admin] [--password-stdin] daily-epub users add USER [--admin] [--password-stdin]
daily-epub users passwd USER [--password-stdin] daily-epub users passwd USER [--password-stdin]
daily-epub users role USER user|admin daily-epub users role USER user|admin
@@ -369,6 +370,10 @@ prints what resolved.
| `server.login_window_minutes` | `15` | Length of the login throttle window. | | `server.login_window_minutes` | `15` | Length of the login throttle window. |
| `server.jobs_enabled` | `true` | Allow the dashboard to start the fixed systemd job catalogue. | | `server.jobs_enabled` | `true` | Allow the dashboard to start the fixed systemd job catalogue. |
| `server.journal_lines` | `300` | Journal lines shown on a dashboard job page (10–5000). | | `server.journal_lines` | `300` | Journal lines shown on a dashboard job page (10–5000). |
| `cdn.provider` | unset | `cloudflare`, or unset for no CDN integration. Setting it requires the zone id and the token. |
| `cdn.cloudflare_zone_id` | unset | Zone id from the Cloudflare dashboard overview. |
| `cdn.api_token` | — | **`DAILY_EPUB_CDN__API_TOKEN`**. Needs exactly one permission: `Zone → Cache Purge`, scoped to that zone. |
| `cdn.purge_after_publish` | `true` | Purge the whole edge cache after `generate` publishes. A failure is logged, never fatal; dry runs never purge. |
`[curation.ranking]` holds the ranker's tunables. The learned signals are `[curation.ranking]` holds the ranker's tunables. The learned signals are
gated: `knn` (rated-neighbour preference) ramps from `knn_floor` (8) to gated: `knn` (rated-neighbour preference) ramps from `knn_floor` (8) to
@@ -464,27 +469,63 @@ Node for the XTC converter, whose JIT needs W+X pages.
The rating links baked into every article chapter point at The rating links baked into every article chapter point at
`server.public_url`, so `daily.hallada.net` must resolve and serve TLS from the `server.public_url`, so `daily.hallada.net` must resolve and serve TLS from the
internet (e-readers tap these links). The OPDS catalog rides on the same host. internet (e-readers tap these links). The OPDS catalog rides on the same host.
The login throttle's `SmartIpKeyExtractor` trusts `X-Forwarded-For`; this is
safe only because the configured bind address is loopback and nginx is the The login throttle's `SmartIpKeyExtractor` keys on the first entry of
only process that can reach it. `X-Forwarded-For`, so nginx must *set* that header from the connection's peer
With an existing certificate, a minimal nginx site is: address rather than appending to whatever the client sent — otherwise anyone
can mint a fresh throttle bucket per attempt by sending their own header. With
Cloudflare in front, the connection's peer is Cloudflare, so `real_ip` has to
rewrite `$remote_addr` from `CF-Connecting-IP` first (see the snippet below).
The whole arrangement is only safe because the configured bind address is
loopback and nginx is the only process that can reach it.
The site is served through Cloudflare; `docs/runbooks/cdn-rollout.md` covers the
zone setup and cache rules. The nginx side of it is the `set_real_ip_from`
snippet and the `X-Forwarded-For` line below.
```nginx ```nginx
server { server {
listen 443 ssl; listen 443 ssl http2;
listen [::]:443 ssl; listen [::]:443 ssl http2;
server_name daily.hallada.net; server_name daily.hallada.net;
ssl_certificate /etc/letsencrypt/live/daily.hallada.net/fullchain.pem; ssl_certificate /etc/letsencrypt/live/daily.hallada.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/daily.hallada.net/privkey.pem; ssl_certificate_key /etc/letsencrypt/live/daily.hallada.net/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/daily.hallada.net/fullchain.pem;
include /etc/nginx/snippets/security-headers.conf;
# Trust Cloudflare's edge for the client address: $remote_addr becomes the
# visitor, not the proxy. Must come before the X-Forwarded-For line below.
include /etc/nginx/snippets/cloudflare-real-ip.conf;
# The global `gzip on` only covers text/html. The stylesheet is ~12 KB of
# plain CSS again — the Newsreader faces are separate .woff2 URLs, not
# base64 inside it (see the doc comment on `web::APP_CSS`) — so this list is
# about the JSON, XML and SVG responses. woff2 is already compressed and is
# deliberately absent.
gzip_types text/css application/javascript application/json
application/speculationrules+json image/svg+xml
application/atom+xml application/xml text/plain;
gzip_min_length 1024;
gzip_vary on;
gzip_proxied any;
# Keep a browser's connection open longer than the 75 s default so the
# first click after a pause reuses it instead of paying a new TLS handshake.
keepalive_timeout 300s;
# XTCH files can be tens of MB; don't buffer them through nginx memory. # XTCH files can be tens of MB; don't buffer them through nginx memory.
proxy_max_temp_file_size 0; proxy_max_temp_file_size 0;
location / { location / {
proxy_pass http://127.0.0.1:3499; proxy_pass http://127.0.0.1:3499;
proxy_http_version 1.1;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # SET, not append. The login throttle keys on the first X-Forwarded-For
# entry, so a client-supplied header must never survive into the app.
# $remote_addr is the real visitor because of the real_ip snippet above.
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
} }
@@ -497,12 +538,75 @@ server {
} }
``` ```
Enable it (`ln -s` into `sites-enabled`, `nginx -t`, `systemctl reload nginx`), `/etc/nginx/snippets/cloudflare-real-ip.conf` is one `set_real_ip_from` line per
then `curl https://daily.hallada.net/healthz` should return `ok`. No auth is published Cloudflare range plus the header to read. Cloudflare adds ranges from
needed at the proxy layer: rating links are self-authenticating (HMAC tokens) time to time, so **regenerate it from the source rather than copying this list**
and the OPDS/file routes use the app-level Basic auth from (<https://www.cloudflare.com/ips-v4>, <https://www.cloudflare.com/ips-v6>):
```sh
{ curl -s https://www.cloudflare.com/ips-v4; echo; \
curl -s https://www.cloudflare.com/ips-v6; echo; } \
| awk 'NF {print "set_real_ip_from " $0 ";"} END {print "real_ip_header CF-Connecting-IP;"}' \
| sudo tee /etc/nginx/snippets/cloudflare-real-ip.conf
```
As of this writing that produces:
```nginx
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/13;
set_real_ip_from 104.24.0.0/14;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2a06:98c0::/29;
set_real_ip_from 2c0f:f248::/32;
real_ip_header CF-Connecting-IP;
```
Enable the site (`ln -s` into `sites-enabled`, `nginx -t`, `systemctl reload
nginx`), then `curl https://daily.hallada.net/healthz` should return `ok`. No
auth is needed at the proxy layer: rating links are self-authenticating (HMAC
tokens) and the OPDS/file routes use the app-level Basic auth from
`server.basic_auth_user`/`_pass` if you set them. `server.basic_auth_user`/`_pass` if you set them.
#### HTTP caching
The origin says what may be cached and for how long; the CDN is configured to
respect it (runbook §2). The whole matrix:
| route | `Cache-Control` |
|---|---|
| `/`, `/issues`, `/issues/{date}`, `/feed.xml`, `/issues.json` (anonymous) | `public, max-age=300, s-maxage=86400` |
| the same pages with a `daily_session=` cookie | `private, no-store` |
| `/robots.txt` | `public, max-age=86400` |
| `/static/*?v=<hash>` | `public, max-age=31536000, immutable` |
| `/static/*` with no `?v=` | `public, max-age=3600` |
| `/files/epub/*`, `/files/xtc/*`, `/opds*` (every status) | `private, no-store` |
| `/dashboard*`, `/login`, `/account`, errors, anything else | `no-store` |
The five-minute browser age and the one-day edge age are the same header doing
two jobs: a reader's tab revalidates soon after a new issue lands, while the
edge is allowed to answer for a whole day because `generate` purges it the
moment it publishes (`[cdn]`, `daily-epub cdn purge`). The `no-store` on the
last row is a default applied by the `security_headers` middleware to any
response that set no policy of its own, so a route added later cannot silently
inherit the CDN's default TTL.
### Installing the XTC converter ### Installing the XTC converter
```sh ```sh
docs/runbooks/cdn-rollout.md
+281
View File
@@ -0,0 +1,281 @@
# Runbook — putting daily.hallada.net behind Cloudflare
**Written:** 2026-09-04 for the production host. Steps 1–2 happen in the Cloudflare
and registrar dashboards; steps 3–6 are on the server as an operator with `sudo`.
The origin-side changes (the `Cache-Control` matrix and `daily-epub cdn purge`)
ship in the same release, so deploy the new binary before step 5.
Read the whole thing before starting step 1: a nameserver change is the one step
that cannot be undone in seconds.
## 0. What the origin already does
The app decides what is cacheable; Cloudflare is configured only to obey it.
| route | `Cache-Control` |
|---|---|
| `/`, `/issues`, `/issues/{date}`, `/feed.xml`, `/issues.json` (anonymous) | `public, max-age=300, s-maxage=86400` |
| the same pages with a `daily_session=` cookie | `private, no-store` |
| `/robots.txt` | `public, max-age=86400` |
| `/static/*?v=<hash>` | `public, max-age=31536000, immutable` |
| `/static/*` with no `?v=` | `public, max-age=3600` |
| `/files/epub/*`, `/files/xtc/*`, `/opds*` (every status, 401 and 404 included) | `private, no-store` |
| `/dashboard*`, `/login`, `/account`, error pages, anything else | `no-store` |
Two facts follow from that table and matter for every choice below:
- The edge is allowed to hold a public page for a **day** (`s-maxage=86400`).
That is only correct because `generate` purges the whole zone right after it
publishes. If the purge is broken, the site serves yesterday's paper until the
day elapses. Step 5 is not optional.
- Anything gated by a cookie or Basic auth already says `private, no-store`, so
even a misconfigured cache-everything rule cannot make a download public. The
cookie bypass rule in step 3 is defence in depth, not the only defence.
## 1. Move DNS to Cloudflare
The Free plan has no partial (CNAME) setup, so this is a full nameserver move:
Cloudflare becomes authoritative for the entire `hallada.net` zone, not just this
host. Everything else in the zone must survive the move unchanged.
1. Export the current zone from Route 53 as BIND:
```sh
ZONE=$(aws route53 list-hosted-zones-by-name --dns-name hallada.net. \
--query 'HostedZones[0].Id' --output text | cut -d/ -f3)
aws route53 list-resource-record-sets --hosted-zone-id "$ZONE" > /tmp/hallada-net.json
```
The AWS CLI has no BIND exporter; the console's **Export zone file** button on
the hosted zone does produce one. Keep both files — the JSON is the rollback
reference.
2. In Cloudflare, **Add a site** → `hallada.net` → Free plan → **Import DNS
records** and upload the BIND file. Then read the imported list against the
export line by line. Pay attention to `MX`, `TXT` (SPF/DKIM/DMARC), and any
`CNAME` used for verification: those must all stay **DNS only** (grey cloud).
3. Set **only** the `daily` record to **Proxied** (orange cloud). Nothing else in
the zone should be proxied — proxying an MX target or a mail-related host
breaks it.
4. Change the nameservers at the registrar to the two Cloudflare assigns.
Propagation is usually minutes and can be hours. Cloudflare emails when the
zone goes active.
5. Before the switch, drop the TTL on the `daily` record (60 s) so a rollback is
quick. Verify afterwards:
```sh
dig +short NS hallada.net
dig +short daily.hallada.net # Cloudflare anycast addresses once proxied
dig +short MX hallada.net # unchanged
```
## 2. Zone settings
Under the zone's **SSL/TLS**, **Speed** and **Scrape Shield** sections:
- **SSL/TLS → Overview → Full (strict)**. The origin has a real Let's Encrypt
certificate, so there is no reason to accept anything weaker. *Flexible* would
make Cloudflare talk plain HTTP to the origin and would break the redirect in
the nginx `:80` server block into a loop.
- **Caching → Configuration → Browser Cache TTL → Respect Existing Headers.**
This one is easy to miss and quietly wrong: the default (**4 hours**) *raises*
the `max-age` Cloudflare sends to browsers, so the deliberate 5-minute browser
age on the public pages would become 4 hours and a reader's tab would show a
stale paper long after a purge.
- **Speed → Optimization → Rocket Loader: OFF.**
- **Scrape Shield → Email Address Obfuscation: OFF.**
Both inject a Cloudflare-hosted script into the HTML. The site's CSP is
`script-src 'self'`, so the browser blocks the injected script and the page
either breaks or logs CSP violations on every load. There is nothing to gain:
the site has no email addresses in its markup and its own JS is two small
files.
- **Do not enable Web Analytics with automatic injection** — same reason, it is
an injected third-party script. If analytics are wanted later, they have to be
first-party and listed in the CSP.
- **Always Use HTTPS: ON** is fine and lets the origin's `:80` block stay as a
backstop.
## 3. Cache Rules
**Caching → Cache Rules.** Order matters: rule 1 must sit above rule 2.
**Rule 1 — "Bypass cache for signed-in readers"**
- When incoming requests match: `http.cookie contains "daily_session="`
- Then: **Bypass cache**
The origin already sends `private, no-store` to a cookie-bearing request, but
this makes the bypass a property of the request rather than of the response, so
nothing is ever *looked up* in a shared cache for a signed-in reader.
**Rule 2 — "Cache by origin headers"**
- When incoming requests match: `http.host eq "daily.hallada.net"`
- Then: **Eligible for cache**
- **Edge TTL:** *Use cache-control header if present, bypass cache if not*
- **Browser TTL:** *Respect origin*
That Edge TTL mode is the whole point of the origin work: a response with
`s-maxage` is cached for exactly that long, and a response with `no-store` (or
one that somehow arrives with no policy at all) is not cached. It is the reason
the `security_headers` middleware defaults unknown routes to `no-store` — with
this mode, "no header" means "do not cache" rather than "cache for 2 hours".
Leave **Cache Key** at its default. Do not add "Ignore query string": the
`?v=<hash>` on `/static/*` is what makes the immutable one-year `max-age` safe.
## 4. Lock the origin to Cloudflare
Once traffic arrives through the edge, direct hits on the origin's port 443
should stop. Either is enough; the first is simpler.
- **Firewall:** allow 443 only from the published Cloudflare ranges.
```sh
{ curl -s https://www.cloudflare.com/ips-v4; echo; \
curl -s https://www.cloudflare.com/ips-v6; echo; } \
| awk 'NF {print $0}' \
| xargs -I{} sudo ufw allow proto tcp from {} to any port 443 comment 'cloudflare'
sudo ufw delete allow 443/tcp # remove the open rule last
```
Keep a way in that does not depend on this (SSH from your own address) before
removing the open rule.
- **Authenticated Origin Pulls** (SSL/TLS → Origin Server) instead: install
Cloudflare's client CA on the origin and add `ssl_client_certificate` +
`ssl_verify_client on` to the nginx server block. Stronger, but one more
certificate to keep track of.
Also apply the nginx changes from the README's *Reverse proxy* section now:
the `include /etc/nginx/snippets/cloudflare-real-ip.conf;` (generated from
Cloudflare's published ranges, `real_ip_header CF-Connecting-IP;` at the end) and
`proxy_set_header X-Forwarded-For $remote_addr;` in place of
`$proxy_add_x_forwarded_for`. Without the first, every request looks like it came
from Cloudflare and the login throttle becomes global; without the second, a
client could forge the throttle key. Then:
```sh
sudo nginx -t && sudo systemctl reload nginx
```
Check a login attempt from a phone on cellular and one from the LAN land in
different throttle buckets, and that `journalctl -u daily-epub` shows real client
addresses rather than Cloudflare's.
## 5. Configure the purge
Create the API token in the Cloudflare dashboard: **My Profile → API Tokens →
Create Token → Custom token**, with the single permission **Zone → Cache Purge →
Purge**, **Zone Resources → Include → Specific zone → hallada.net**. Nothing
else — the app makes exactly one API call. Copy the token once; it is not shown
again. The zone id is on the zone's **Overview** page.
In `/etc/daily-epub/config.toml`:
```toml
[cdn]
provider = "cloudflare"
cloudflare_zone_id = "…"
purge_after_publish = true
```
and in the systemd environment file (the one the units already load, mode `0600`,
owned by `daily-epub`):
```
DAILY_EPUB_CDN__API_TOKEN=…
```
Setting `provider` without both the zone id and the token is a **config error**:
the app refuses to start rather than publishing into a stale edge. Confirm and
then purge by hand:
```sh
sudo systemctl restart daily-epub
sudo -u daily-epub daily-epub --config /etc/daily-epub/config.toml config check | grep '^cdn'
# cdn: cloudflare · zone … · token present · purge_after_publish true
sudo -u daily-epub daily-epub --config /etc/daily-epub/config.toml cdn purge
# purged the whole cloudflare cache (id …)
```
`cdn purge` takes no run lock and touches neither the database nor the publish
directories, so it is safe to run at any time, including during a `generate`.
From then on `generate` purges the whole zone after every successful publish. The
purge is *purge everything* on purpose: a new issue also changes the previous
issue's page (its "latest" nav marker moves), `/issues`, `/feed.xml` and
`/issues.json`, and a per-URL list of that set would rot. A purge failure is
logged at `warn` and never fails the run — so check for it after the first live
run:
```sh
journalctl -u daily-epub-generate.service --since today | grep -i 'purge'
```
## 6. Verify
Anonymous public page — expect `HIT` on the second request (the first fills the
edge) and the origin's own two-part `Cache-Control`:
```sh
curl -sI https://daily.hallada.net/ | grep -iE 'cf-cache-status|cache-control|age'
curl -sI https://daily.hallada.net/ | grep -i cf-cache-status # HIT
```
Signed in — the cookie rule must take it out of the cache entirely:
```sh
curl -sI -H 'Cookie: daily_session=whatever' https://daily.hallada.net/ \
| grep -iE 'cf-cache-status|cache-control'
# cf-cache-status: BYPASS (DYNAMIC is also acceptable)
# cache-control: private, no-store
```
Static assets — versioned URLs should settle on `HIT`:
```sh
curl -s https://daily.hallada.net/ | grep -o '/static/app.css?v=[0-9a-f]*'
curl -sI 'https://daily.hallada.net/static/app.css?v=<hash>' \
| grep -iE 'cf-cache-status|cache-control'
# cf-cache-status: HIT
# cache-control: public, max-age=31536000, immutable
```
Downloads and OPDS — never cached, at any status:
```sh
curl -sI https://daily.hallada.net/opds | grep -iE 'cf-cache-status|cache-control'
curl -sI https://daily.hallada.net/files/epub/does-not-exist.epub \
| grep -iE 'cf-cache-status|cache-control'
# cache-control: private, no-store (cf-cache-status: BYPASS or DYNAMIC)
```
The dashboard and the login page:
```sh
curl -sI https://daily.hallada.net/login | grep -iE 'cf-cache-status|cache-control'
# cache-control: no-store
```
Finally, the end-to-end check the whole exercise is about: run a `generate`, then
immediately `curl -sI https://daily.hallada.net/ | grep -i cf-cache-status` and
confirm it reads `MISS` (the purge emptied the edge) and that the page shows the
new issue.
## Rollback
- **Cache misbehaving:** turn on **Development Mode** (Caching → Configuration)
for a three-hour edge bypass while you look, or **Purge Everything**. Neither
needs a deploy.
- **Something worse:** set the `daily` DNS record back to **DNS only** (grey
cloud). Traffic goes straight to the origin again within the record's TTL, and
nothing about the origin's behaviour depends on Cloudflare being there — the
`Cache-Control` headers are correct without it, and `cdn.provider` can be
removed from the config at leisure.
- **Full retreat:** point the registrar's nameservers back at Route 53. The
hosted zone still exists unless it was deleted; do not delete it until the
Cloudflare setup has run for a few weeks.