e6a37703be
Use tokio::sync::mutex for Crawls
2023-07-19 23:03:35 -04:00
d17f909312
Add CrawlScheduler actor, shared client w/ last modified headers
2023-07-15 21:40:31 -04:00
0dfde7cd31
Begin adding pagination for feeds
2023-07-15 02:12:59 -04:00
4837cbb903
Add crawl metadata to feed & improve model interface
2023-07-15 00:40:10 -04:00
923776d7a1
Add DomainLocks to serialize requests to each domain
...
All async tasks must wait 1 second since the last request to make
another to the same domain.
2023-07-14 00:55:35 -04:00
b7efc61cfc
Add EntryCrawler that uses readability lib
...
Actors delegating to actors baybeeee
2023-07-11 01:52:38 -04:00
f13c7e5e70
Add an async actor FeedCrawler for fetching feed details
...
Currently, this allows the browser to subscribe to the response of the
asynchronous crawl after they add a new feed.
Eventually I will also use this in the main scheduled crawls. Right now,
it only upserts feed metadata based on the parsed feed.
2023-07-09 21:30:23 -04:00
b94555d346
Delete feed button
...
And a non-functional edit button
2023-07-08 01:52:15 -04:00
478e72d8f0
Add feed form
2023-07-08 01:05:49 -04:00
f69d0f2752
Improve feed list and add feed page
2023-07-07 16:03:59 -04:00
1698ef9c60
Add maud for frontend and move api to sub module
2023-06-01 00:03:00 -04:00
ae8f15f19b
Add very basic crawl job
...
Loops through feeds and adds items from each feed.
2023-05-09 23:55:42 -04:00
b2a5bf5882
Add feed model, link items to feeds
2023-05-07 21:25:22 -04:00
