Commit Graph

5 Commits

Author SHA1 Message Date
d17f909312 Add CrawlScheduler actor, shared client w/ last modified headers 2023-07-15 21:40:31 -04:00
923776d7a1 Add DomainLocks to serialize requests to each domain
All async tasks must wait 1 second since the last request to make
another to the same domain.
2023-07-14 00:55:35 -04:00
f13c7e5e70 Add an async actor FeedCrawler for fetching feed details
Currently, this allows the browser to subscribe to the response of the
asynchronous crawl after they add a new feed.

Eventually I will also use this in the main scheduled crawls. Right now,
it only upserts feed metadata based on the parsed feed.
2023-07-09 21:30:23 -04:00
6713a7a440 Complete log stream implementation
Sets up a watch channel to send tracing lines from tracing-subscriber to
receivers in a axum handler which streams Server Sent Events to any
number of connected /log/stream clients.
2023-06-03 19:03:58 -04:00
f4da3f3ab3 Adding title to config and utilizing it in layout 2023-06-02 00:08:20 -04:00