-
e1e7dd2e5d
Rename drop_all.sql file
thallada
2021-07-29 01:01:27 -04:00
-
fc5a9d39c2
Add timestamp to mods for some resumability
thallada
2021-07-29 00:50:42 -04:00
-
a8424e830e
Adds download_at to files, skips files that are impossible to download
thallada
2021-07-28 23:54:39 -04:00
-
8a356ac7f5
Retry connect failures, write plugins to disk instead of zip archive
thallada
2021-07-26 19:31:25 -04:00
-
f62324d36c
Reset main.rs back w/ fix to listing RAR files
thallada
2021-07-26 17:02:14 -04:00
-
5d62fb7d61
Fix plugin listing for rar files, temporary backfil shim
thallada
2021-07-26 01:16:43 -04:00
-
87ca90be06
Fix plugin insert SQL binding
thallada
2021-07-25 18:33:45 -04:00
-
523f3eeb3f
Add retry logic to metadata fetch, don't crash on 500
thallada
2021-07-25 18:03:54 -04:00
-
39ae7703b0
Upgrade skyrim-cell-dump and handle
Cow<str>s
thallada
2021-07-23 23:40:05 -04:00
-
19126f4981
Add retry logic to API calls
thallada
2021-07-23 22:29:28 -04:00
-
81f12da99c
Upgrade unrar to master branch to fix Utf8Error
thallada
2021-07-23 19:27:21 -04:00
-
e69165fe5a
Set page size to 20 since nexus mods doesn't always obey 80
thallada
2021-07-22 21:55:31 -04:00
-
77562a7a05
Add CLI args with page, skip unreadable archives
thallada
2021-07-22 21:50:28 -04:00
-
e534447310
Simplify bulk_get_present_nexus_mod_ids query
thallada
2021-07-22 00:13:20 -04:00
-
f337917c99
Batch insert mods, add lifetimes to batch inserts
thallada
2021-07-21 23:20:51 -04:00
-
3fe09a8f8f
Upgrade skyrim-cell-dump to 0.2.1
thallada
2021-07-21 22:26:16 -04:00
-
04773954f9
Remove backfill, add bulk get game_mod method
thallada
2021-07-21 22:25:45 -04:00
-
4b333b3b99
Add world tables and columns, temporary backfill
thallada
2021-07-21 21:35:11 -04:00
-
0e0fdfd59d
Migration and models for worlds/plugin_worlds
thallada
2021-07-19 23:47:00 -04:00
-
7eba0c6adc
Upgrade skyrim-cell-dump to 0.2.0
thallada
2021-07-19 23:45:51 -04:00
-
1fd979b8e0
Skip archived files which cannot be downloaded
thallada
2021-07-18 22:11:03 -04:00
-
f6a7490214
Skip empty or invalid file archives
thallada
2021-07-18 16:06:49 -04:00
-
a0aa24f360
Log warning and skip invalid plugins
thallada
2021-07-18 15:09:23 -04:00
-
62c0f5295f
Attempt to download file 3 times before crashing
thallada
2021-07-18 01:45:08 -04:00
-
4e500f0b0b
Various fixes to errors that occurred during runs
thallada
2021-07-17 14:19:43 -04:00
-
5514fb5d29
Upgrade compress-tools to branch w/ fix for crash
thallada
2021-07-16 00:13:38 -04:00
-
621b98cd8c
Only process files (not dirs) extracted from rar
thallada
2021-07-15 13:35:59 -04:00
-
c7df9dec13
Fix file metadata parsing
thallada
2021-07-15 10:54:53 -04:00
-
ccac5b71de
Upgrade skyrim-cell-dump to 0.1.4
thallada
2021-07-15 10:39:06 -04:00
-
93d4bfeca2
Skip downloading files with no plugins
thallada
2021-07-12 19:33:43 -04:00
-
fd8015024f
Always uncompress entire rar archive
thallada
2021-07-12 13:49:02 -04:00
-
af5c11acc7
Tweak rate limit delay logic, log skipped files
thallada
2021-07-12 12:15:07 -04:00
-
81fcdfa381
Skip processing of invalid empty plugins
thallada
2021-07-12 10:43:03 -04:00
-
ced31f0eb6
Handle null value for size_in_bytes in API files
thallada
2021-07-12 00:43:54 -04:00
-
4612ffc3c4
Add some .env instructions to readme
thallada
2021-07-11 22:55:47 -04:00
-
b609e7059a
Log spans, track size & undownloadable files
thallada
2021-07-11 22:49:29 -04:00
-
792e78391c
Replace dbg macros with tracing events
thallada
2021-07-11 19:45:26 -04:00
-
22757bc475
Modularize scraping and api requests
thallada
2021-07-09 00:37:08 -04:00
-
19350081c3
Move SQL stuff into separate models module
thallada
2021-07-08 21:19:16 -04:00
-
d8080091f2
Skip already processed mods, fix pagination
thallada
2021-07-06 23:29:09 -04:00
-
184d240aac
Fix handling rar files
thallada
2021-07-04 00:01:59 -04:00
-
421f2b7071
WIP fix rar file extraction
Tyler Hallada
2021-07-03 16:00:18 -04:00
-
d6b8f4e74a
Mostly working download loop done
thallada
2021-06-13 22:30:40 -04:00
-
b132a94c64
Initital commit
thallada
2021-06-03 12:30:04 -04:00