Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• e1e7dd2e5d Rename drop_all.sql file Tyler Hallada 2021-07-29 01:01:27 -04:00
• fc5a9d39c2 Add timestamp to mods for some resumability Tyler Hallada 2021-07-29 00:50:42 -04:00
• a8424e830e Adds download_at to files, skips files that are impossible to download Tyler Hallada 2021-07-28 23:54:39 -04:00
• 8a356ac7f5 Retry connect failures, write plugins to disk instead of zip archive Tyler Hallada 2021-07-26 19:31:25 -04:00
• f62324d36c Reset main.rs back w/ fix to listing RAR files Tyler Hallada 2021-07-26 17:02:14 -04:00
• 5d62fb7d61 Fix plugin listing for rar files, temporary backfil shim Tyler Hallada 2021-07-26 01:16:43 -04:00
• 87ca90be06 Fix plugin insert SQL binding Tyler Hallada 2021-07-25 18:33:45 -04:00
• 523f3eeb3f Add retry logic to metadata fetch, don't crash on 500 Tyler Hallada 2021-07-25 18:03:54 -04:00
• 39ae7703b0 Upgrade skyrim-cell-dump and handle Cow<str>s Tyler Hallada 2021-07-23 23:40:05 -04:00
• 19126f4981 Add retry logic to API calls Tyler Hallada 2021-07-23 22:29:28 -04:00
• 81f12da99c Upgrade unrar to master branch to fix Utf8Error Tyler Hallada 2021-07-23 19:27:21 -04:00
• e69165fe5a Set page size to 20 since nexus mods doesn't always obey 80 Tyler Hallada 2021-07-22 21:55:31 -04:00
• 77562a7a05 Add CLI args with page, skip unreadable archives Tyler Hallada 2021-07-22 21:50:28 -04:00
• e534447310 Simplify bulk_get_present_nexus_mod_ids query Tyler Hallada 2021-07-22 00:13:20 -04:00
• f337917c99 Batch insert mods, add lifetimes to batch inserts Tyler Hallada 2021-07-21 23:20:51 -04:00
• 3fe09a8f8f Upgrade skyrim-cell-dump to 0.2.1 Tyler Hallada 2021-07-21 22:26:16 -04:00
• 04773954f9 Remove backfill, add bulk get game_mod method Tyler Hallada 2021-07-21 22:25:45 -04:00
• 4b333b3b99 Add world tables and columns, temporary backfill Tyler Hallada 2021-07-21 21:35:11 -04:00
• 0e0fdfd59d Migration and models for worlds/plugin_worlds Tyler Hallada 2021-07-19 23:47:00 -04:00
• 7eba0c6adc Upgrade skyrim-cell-dump to 0.2.0 Tyler Hallada 2021-07-19 23:45:51 -04:00
• 1fd979b8e0 Skip archived files which cannot be downloaded Tyler Hallada 2021-07-18 22:11:03 -04:00
• f6a7490214 Skip empty or invalid file archives Tyler Hallada 2021-07-18 16:06:49 -04:00
• a0aa24f360 Log warning and skip invalid plugins Tyler Hallada 2021-07-18 15:09:23 -04:00
• 62c0f5295f Attempt to download file 3 times before crashing Tyler Hallada 2021-07-18 01:45:08 -04:00
• 4e500f0b0b Various fixes to errors that occurred during runs Tyler Hallada 2021-07-17 14:19:43 -04:00
• 5514fb5d29 Upgrade compress-tools to branch w/ fix for crash Tyler Hallada 2021-07-16 00:13:38 -04:00
• 621b98cd8c Only process files (not dirs) extracted from rar Tyler Hallada 2021-07-15 13:35:59 -04:00
• c7df9dec13 Fix file metadata parsing Tyler Hallada 2021-07-15 10:54:53 -04:00
• ccac5b71de Upgrade skyrim-cell-dump to 0.1.4 Tyler Hallada 2021-07-15 10:39:06 -04:00
• 93d4bfeca2 Skip downloading files with no plugins Tyler Hallada 2021-07-12 19:33:43 -04:00
• fd8015024f Always uncompress entire rar archive Tyler Hallada 2021-07-12 13:49:02 -04:00
• af5c11acc7 Tweak rate limit delay logic, log skipped files Tyler Hallada 2021-07-12 12:15:07 -04:00
• 81fcdfa381 Skip processing of invalid empty plugins Tyler Hallada 2021-07-12 10:43:03 -04:00
• ced31f0eb6 Handle null value for size_in_bytes in API files Tyler Hallada 2021-07-12 00:43:54 -04:00
• 4612ffc3c4 Add some .env instructions to readme Tyler Hallada 2021-07-11 22:55:47 -04:00
• b609e7059a Log spans, track size & undownloadable files Tyler Hallada 2021-07-11 22:49:29 -04:00
• 792e78391c Replace dbg macros with tracing events Tyler Hallada 2021-07-11 19:45:26 -04:00
• 22757bc475 Modularize scraping and api requests Tyler Hallada 2021-07-09 00:37:08 -04:00
• 19350081c3 Move SQL stuff into separate models module Tyler Hallada 2021-07-08 21:19:16 -04:00
• d8080091f2 Skip already processed mods, fix pagination Tyler Hallada 2021-07-06 23:29:09 -04:00
• 184d240aac Fix handling rar files Tyler Hallada 2021-07-04 00:01:59 -04:00
• 421f2b7071 WIP fix rar file extraction Tyler Hallada 2021-07-03 16:00:18 -04:00
• d6b8f4e74a Mostly working download loop done Tyler Hallada 2021-06-13 22:30:40 -04:00
• b132a94c64 Initital commit Tyler Hallada 2021-06-03 12:30:04 -04:00