thallada/crawlnicle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
88 Commits 3 Branches 0 Tags
7bff966ac0778b015031ac16960f03406b8bf94a
Commit Graph

70 Commits

Author SHA1 Message Date
thallada 8feb2ec10e Prevent decoding empty uuid strings 2023-10-17 01:09:17 -04:00
thallada 5f9d64f2d9 Prevent Base62Uuid decoding from panicing 
An extra long encoded uuid could crash the server, now it's handled as
an error.
 2023-10-17 00:59:01 -04:00
thallada 7f86612899 Switch to async-fred-session, default config vals 
Fixup some issues in README too
 2023-10-17 00:15:44 -04:00
thallada 835e9dc748 Do reset password operations in db transaction 
And modify signature of model methods to accept an executor instead of a
pool connection which will also allow transactions.
 2023-10-13 14:44:40 +02:00
thallada 60671d5865 Complete reset password flow 2023-10-13 14:07:38 +02:00
thallada d5c5185351 Fix sessions persisting across server restart 
Need a non-random session secret.
 2023-10-07 00:54:15 -04:00
thallada 609f6d3d9f Fixes to email verification process 
Also make registration form progressively enhanced.
 2023-10-06 17:50:08 -04:00
thallada e59c6d596e Finish up email verification form 2023-09-29 23:48:37 -04:00
thallada c95334a7e2 Email verification form submit 2023-09-29 20:56:43 -04:00
thallada cdc8eb9b02 Basic email verification done 
Still need to add forms to confirmation page to allow resending and add
rate limiting.
 2023-09-28 23:53:46 -04:00
thallada f938a6b46b WIP email sending for registration 2023-09-27 23:11:52 -04:00
thallada a72bfa15bd Use HX-Target, not HX-Boost for Layout render 
HX-Boost is not always sent in all AJAX requests that htmx sends, only
those initiated by an element with hx-boost enabled. It was not showing
up on requests following an HX-Redirect response.

After reading the docs more, I realized HX-Target was what I wanted. If
I can see that the request is targeting `#main-content` then I know to
only return HTML inside that element. Simple.
 2023-09-27 01:28:53 -04:00
thallada bea3529e22 Rename signup to register and center forms 2023-09-27 00:23:28 -04:00
thallada 092a38ad52 Add hx-boosted support to Layout 
It will now skip rendering the layout if the request is coming from an
hx-boosted link or form and only update the head title with a
hx-swap-oob.
 2023-09-26 23:36:31 -04:00
thallada 81b4ef860e Fix login/signup redirection 
HTMX needs special treatment for redirects on submit.
 2023-09-26 01:48:05 -04:00
thallada 6fd2f150a0 Store sessions in redis 2023-09-26 01:47:50 -04:00
thallada 306059c355 Add basic user auth 2023-09-25 01:38:18 -04:00
thallada ec394fc170 Implement entry and feed pagination 2023-09-02 14:01:18 -04:00
thallada 0607b46283 Add a footer 2023-09-01 20:13:44 -04:00
thallada e50e165b87 Remove dbg! statements 2023-09-01 19:59:56 -04:00
thallada 1d6f98c6bb Replace hotwire with htmx 
In the process, also improve the feedback from the import/add feed
forms.

I also replaced the frontend code to replace utc timestamps with local
time strings with @hotwired/stimulus with vanilla js.
 2023-09-01 00:25:05 -04:00
thallada ff0b218da1 Adjust entry html/css 2023-08-29 23:36:05 -04:00
thallada 69ed8aa8bb Use ceil in calculating crawl_interval 
So that it can increase from 1 minute.
 2023-08-29 23:32:05 -04:00
thallada 276f0e17a8 Remove self from Crawls and Imports in actors at end of task 
In case the user never listens to the stream so that I do not create
inifinitely growing hashmaps in the server memory.
 2023-08-29 23:30:00 -04:00
thallada ceac234ce7 Sanitize entry html content with ammonia 2023-08-29 23:04:35 -04:00
thallada 2f39be4152 Finish implemtning OPML importer 
Now with progress messages!
 2023-08-29 00:35:19 -04:00
thallada eddf39b62e Added importer actor, file upload still not working 2023-07-22 22:54:57 -04:00
thallada e6a37703be Use tokio::sync::mutex for Crawls 2023-07-19 23:03:35 -04:00
thallada d17f909312 Add CrawlScheduler actor, shared client w/ last modified headers 2023-07-15 21:40:31 -04:00
thallada ea67690d1d Start feed crawlers at startup 2023-07-15 02:30:25 -04:00
thallada 0dfde7cd31 Begin adding pagination for feeds 2023-07-15 02:12:59 -04:00
thallada ae95921966 Update feed last_crawl_error & crawl_interval_minutes 2023-07-15 01:30:45 -04:00
thallada 4837cbb903 Add crawl metadata to feed & improve model interface 2023-07-15 00:40:10 -04:00
thallada 0fa0cfc508 Only crawl new entries 2023-07-14 21:27:23 -04:00
thallada 923776d7a1 Add DomainLocks to serialize requests to each domain 
All async tasks must wait 1 second since the last request to make
another to the same domain.
 2023-07-14 00:55:35 -04:00
thallada b7efc61cfc Add EntryCrawler that uses readability lib 
Actors delegating to actors baybeeee
 2023-07-11 01:52:38 -04:00
thallada f13c7e5e70 Add an async actor FeedCrawler for fetching feed details 
Currently, this allows the browser to subscribe to the response of the
asynchronous crawl after they add a new feed.

Eventually I will also use this in the main scheduled crawls. Right now,
it only upserts feed metadata based on the parsed feed.
 2023-07-09 21:30:23 -04:00
thallada b94555d346 Delete feed button 
And a non-functional edit button
 2023-07-08 01:52:15 -04:00
thallada 478e72d8f0 Add feed form 2023-07-08 01:05:49 -04:00
thallada f69d0f2752 Improve feed list and add feed page 2023-07-07 16:03:59 -04:00
thallada 3f028c3088 Store entry html content outside DB in file storage 
Since the HTML content can get quite big and can have embeded images.
 2023-07-05 23:45:49 -04:00
thallada 5169fac218 Reload frontend changes faster in dev mode 
Now frontend changes appear as fast as Bun can bundle them instead of
needing to wait for rust to compile too.
 2023-07-05 20:05:03 -04:00
thallada 74a205d51e Also bundle css with bun so it busts caches 
And updates the `build.rs` script so that it also builds a manifest.txt
for css files.
 2023-06-29 00:38:32 -04:00
thallada 76cc87631f Add frontend JS and improve post styling 
Frontend is built with Bun. It uses Stimulus to progressively enhance
the server-built HTML. Currently, it only replaces UTC timestamps from
the server with the time in the browser's timezone.
 2023-06-27 14:03:52 -04:00
thallada 7e06d23bba Replace argh with clap 
Mostly for the more concise Config parsing and error handling.
 2023-06-27 14:03:52 -04:00
thallada abd540d2ff Better database layout with uuid primary keys 
Serialize and deserialize the uuid ids as base62 strings in the URLs.
 2023-06-27 14:03:52 -04:00
thallada 4e41bbd6e1 Add domain to entry list 2023-06-27 14:03:52 -04:00
thallada 97c4ae73f0 Add static file server, CSS styles for home 
Also fix the livereloading.
 2023-06-09 01:06:19 -04:00
thallada 758e644173 Add published_at to entries, begin to support pagination 
Articles will be sorted by their published_at dates for now.
 2023-06-08 01:20:21 -04:00
thallada 3f29138bd1 Fetch and save entry HTML content with metadata 
And render the extracted HTML on the entry page in the frontend.
 2023-06-07 01:06:03 -04:00