Commit Graph

13 Commits

Author SHA1 Message Date
d31f44a349 Add healthchecks.io badge img to README 2025-09-03 22:43:10 -04:00
68c5cfe8a6 Add monitoring of scripts through healthchecks.io
Updates the README with more cron script setup instructions.
2025-09-03 22:28:11 -04:00
7b8fbea7d4 Update nexus scraper to use new graphql api
Modmapper hasn't been pulling in new mods for a while because nexus mods changed their mod listing page to use a new graphql API and the old API I was using no longer works.
2025-05-30 15:07:53 -04:00
a677325c4d Prevent overwriting of base game cells is_base_game value
This fixes a bug that was causing cell data to not get written since
November 9th, 2023. When batch inserting cells while processing plugins,
I allowed overwriting cells that had `is_base_game = true`. Since I
always set `is_base_game = false` for cell upserts from plugins, this
was causing the base game cells to revert to `is_base_game = false`. All
it took was one mod to bundle `Skyrim.esm` for this to happen.

This broke writing cell data since `get_cell_data` depends on the
`is_base_game` value to find edits to the Skyrim base game cells.

To prevent this in the future, batch inserts when processing plugins is
no longer allowed to update cells which have `is_base_game = true`. The
only time we allow upserting these rows is when running the
`is_base_game` backfill which initially seeds the database with the base
game cells.
2025-03-02 14:15:05 -05:00
c7ead28cfa Fix cells unique index 2023-11-08 16:07:00 -05:00
e85edff30a Fix bad interior cell data in db
I had a bad unique index on the `cells` table. This commit does not fix
it but has two backfill scripts that will allow me to fixup the data
before creating the new unique index.

`is_base_game` identifies cells that are from the `Skyrim.esm` shipped
with the game (and not from some mod that happens to also have a
`Skyrim.esm` with a 0,0 cell with a different form id, see:
https://github.com/thallada/modmapper-web/issues/8).

`dump_cell_data` now only dumps cells from `is_base_game` cells.

`deduplicate_interior_cells` removes duplicate cell and plugin_cell rows
caused by the buggy index.

I will need to upgrade the database and create a new index in a later
commit.
2023-11-08 01:10:49 -05:00
03cf20029e Revert "Login before scraping mods to include adult mods"
This reverts commit bb02771b87.
2022-10-02 02:37:15 -04:00
bb02771b87 Login before scraping mods to include adult mods 2022-10-02 00:34:38 -04:00
7d229ccd1a Add support for scraping classic Skyrim mods
Not adding it to the update.sh script quite yet since I need to run the very long process of downloading every classic skyrim mod that currently exists on the nexus first.
2022-09-02 00:43:53 -04:00
1c1303e4f6 Separate buckets for each file type 2022-06-30 01:29:54 -04:00
4b7a52588c Use rclone for sync, add more instructions 2022-06-29 16:36:32 -04:00
4612ffc3c4 Add some .env instructions to readme 2021-07-11 22:59:53 -04:00
Tyler Hallada
421f2b7071 WIP fix rar file extraction
Also, starting to break up into smaller functions
2021-07-03 16:00:18 -04:00