Commit Graph

10 Commits

Author SHA1 Message Date
ceac234ce7 Sanitize entry html content with ammonia 2023-08-29 23:04:35 -04:00
2f39be4152 Finish implemtning OPML importer
Now with progress messages!
2023-08-29 00:35:19 -04:00
eddf39b62e Added importer actor, file upload still not working 2023-07-22 22:54:57 -04:00
d17f909312 Add CrawlScheduler actor, shared client w/ last modified headers 2023-07-15 21:40:31 -04:00
ae95921966 Update feed last_crawl_error & crawl_interval_minutes 2023-07-15 01:30:45 -04:00
4837cbb903 Add crawl metadata to feed & improve model interface 2023-07-15 00:40:10 -04:00
0fa0cfc508 Only crawl new entries 2023-07-14 21:27:23 -04:00
923776d7a1 Add DomainLocks to serialize requests to each domain
All async tasks must wait 1 second since the last request to make
another to the same domain.
2023-07-14 00:55:35 -04:00
b7efc61cfc Add EntryCrawler that uses readability lib
Actors delegating to actors baybeeee
2023-07-11 01:52:38 -04:00
f13c7e5e70 Add an async actor FeedCrawler for fetching feed details
Currently, this allows the browser to subscribe to the response of the
asynchronous crawl after they add a new feed.

Eventually I will also use this in the main scheduled crawls. Right now,
it only upserts feed metadata based on the parsed feed.
2023-07-09 21:30:23 -04:00