02d5cb9976
Remove article_scraper crate
2023-07-15 02:31:45 -04:00
b7efc61cfc
Add EntryCrawler that uses readability lib
...
Actors delegating to actors baybeeee
2023-07-11 01:52:38 -04:00
f13c7e5e70
Add an async actor FeedCrawler for fetching feed details
...
Currently, this allows the browser to subscribe to the response of the
asynchronous crawl after they add a new feed.
Eventually I will also use this in the main scheduled crawls. Right now,
it only upserts feed metadata based on the parsed feed.
2023-07-09 21:30:23 -04:00
478e72d8f0
Add feed form
2023-07-08 01:05:49 -04:00
7289151318
Upgrade sqlx to 0.7 + some tweaks
2023-07-05 21:14:31 -04:00
7e06d23bba
Replace argh with clap
...
Mostly for the more concise Config parsing and error handling.
2023-06-27 14:03:52 -04:00
abd540d2ff
Better database layout with uuid primary keys
...
Serialize and deserialize the uuid ids as base62 strings in the URLs.
2023-06-27 14:03:52 -04:00
4e41bbd6e1
Add domain to entry list
2023-06-27 14:03:52 -04:00
97c4ae73f0
Add static file server, CSS styles for home
...
Also fix the livereloading.
2023-06-09 01:06:19 -04:00
3f29138bd1
Fetch and save entry HTML content with metadata
...
And render the extracted HTML on the entry page in the frontend.
2023-06-07 01:06:03 -04:00
786f3a194f
Streaming log page with colors
...
Using the ansi-to-html crate.
2023-06-06 21:14:29 -04:00
6713a7a440
Complete log stream implementation
...
Sets up a watch channel to send tracing lines from tracing-subscriber to
receivers in a axum handler which streams Server Sent Events to any
number of connected /log/stream clients.
2023-06-03 19:03:58 -04:00
951d6d23e2
Add /log page that displays server log
...
This will eventually be behind an authorization guard so that only I
have access, but it's useful to have for monitoring and testing out
turbo streams.
2023-06-03 01:09:25 -04:00
74f353b894
Add (semi) live reloading for debug server
2023-06-01 00:59:46 -04:00
1698ef9c60
Add maud for frontend and move api to sub module
2023-06-01 00:03:00 -04:00
6f364b4c44
Rename to crawlnicle
2023-05-10 00:00:48 -04:00
ae8f15f19b
Add very basic crawl job
...
Loops through feeds and adds items from each feed.
2023-05-09 23:55:42 -04:00
89fdf8f95a
Create cli binary
...
Just has `add-feed` command so far.
2023-05-09 00:08:55 -04:00
de157a3b1e
Improve error handling
2023-05-07 18:20:19 -04:00
c2c0f7a28d
Initial commit with basic axum and sqlx API
2023-05-07 17:41:45 -04:00
