Tyler Hallada
5d55e78283
Dump plugins by hash instead of id
...
The same exact plugins are often included in many different files/mods. Grouping by hash in the sql query will show all files and mods a plugin appears in. Previously, the last file that was processed would overwrite the plugin file and only that single file/mod would appear in the file.
Also adds an index on plugins.hash since I'm now querying directly on it.
2022-03-12 13:05:21 -05:00
82996f237f
Add missing migration for cell lookup index
2022-02-06 23:30:20 -05:00
17facae842
Make ancestor cols not nullable, add fk constraints
...
Also removed the temporary backfill, now that it is complete.
Updates models so cols are not optional.
2022-01-24 21:41:45 -05:00
10c299dc45
Add indices for new ancestor id columns
2022-01-24 21:21:12 -05:00
e5936683fa
Add ancestor id columns to plugins and plugin_cells
...
With temp backfill queries. Second stage will to make the columns nullable and add foreign keys.
I'm hoping that this will improve query performance.
2022-01-24 21:14:59 -05:00
e7fe7abfb7
Set columns to not null, remove backfill code
2022-01-18 21:50:09 -05:00
b97689b7fd
Scrape additional fields to mod table
2022-01-17 23:37:58 -05:00
Tyler Hallada
6136ac18fa
Make mod category nullable
2021-08-08 19:14:23 -06:00
3ec7467571
Add more status columns to fix continuation of unprocessed files
2021-07-29 22:55:34 -04:00
fc5a9d39c2
Add timestamp to mods for some resumability
2021-07-29 00:50:42 -04:00
a8424e830e
Adds download_at to files, skips files that are impossible to download
2021-07-28 23:54:39 -04:00
4b333b3b99
Add world tables and columns, temporary backfill
...
Too lazy to make a new script, main() has a backfill using plugins.zip. Once I run it I will remove it.
2021-07-21 21:35:11 -04:00
0e0fdfd59d
Migration and models for worlds/plugin_worlds
2021-07-19 23:47:00 -04:00
b609e7059a
Log spans, track size & undownloadable files
2021-07-11 22:49:29 -04:00
b132a94c64
Initital commit
...
Not quite working yet. WIP.
2021-06-03 12:30:04 -04:00
