diff --git a/_posts/2022-10-05-modmapper-putting-every-skyrim-mod-on-a-map-with-rust.md b/_posts/2022-10-05-modmapper-putting-every-skyrim-mod-on-a-map-with-rust.md
new file mode 100644
index 0000000..10de3e0
--- /dev/null
+++ b/_posts/2022-10-05-modmapper-putting-every-skyrim-mod-on-a-map-with-rust.md
@@ -0,0 +1,692 @@
+---
+title: "Modmapper: Putting every Skyrim mod on a map with Rust"
+layout: post
+image: /img/blog/modmapper.jpg
+hidden: true
+---
+
+[Modmapper](https://modmapper.com) is a website that I made that puts every mod
+for the game [Elder Scrolls V:
+Skyrim](https://en.wikipedia.org/wiki/The_Elder_Scrolls_V:_Skyrim) uploaded to
+[Nexus Mods](https://www.nexusmods.com/) on an interactive map.
+
+
+ 
+
+
+You can view the map at [https://modmapper.com](https://modmapper.com).
+
+Released in 2011, Skyrim is over a decade old now. But, its vast modding
+community has kept it alive and relevant to this day. [Skyrim is still in the
+top 50 games being played on Steam in 2022](https://steamcharts.com/top/p.2) and
+I think it's no coincidence that [it's also one of the most modded games
+ever](https://www.nexusmods.com/games?).
+
+
+
+The enormous and enduring modding community around the Elder Scrolls games is
+why I have a special fondness for the series. I was 13 when I first got
+interested in programming through [making mods for Elder Scrolls IV:
+Oblivion](https://www.nexusmods.com/users/512579?tab=user+files&BH=2). I quickly
+realized I got way more satisfaction out of modding the game than actually
+playing it. I was addicted to being able to create whatever my mind imagined in
+my favorite game.
+
+I was working on mod for Skyrim earlier in the year[^bazaarrealm] and was
+looking for the best places to put new buildings in the game world. I really
+wanted areas of the game world off the beaten (heavily-modded) path. After over
+a decade of modifications, there could be conflicts with hundreds of mods in any
+area I chose which could cause issues like multiple buildings overlapping or
+terrain changes causing floating rocks and trees.
+
+
+
+
+
+
+
+ Example of a conflict between two mods that both chose the same
+ spot to put a lamp post and sign post so they are clipping.
+ Screenshot by
+ AndreySG
+
+
+
+
+
+
+
+ Example of a conflict between two mods that both chose the same
+ spot to put a building and rock so they are clipping. Screenshot
+ by
+ LewdManoSaurus
+
+
+
+
+
+
+
+ Example of a conflict between two mods that both chose the same
+ spot to put a building and tree so they are clipping. Screenshot
+ by
+ Janquel
+
+
+
+
+
+
+
+ Example of a conflict between two mods that both chose the
+ same spot to put a woodcutting mill so they are clipping.
+ Screenshot by
+ Janquel
+
+
+
+
+
+
+Mod authors usually use a tool like
+[TES5Edit](https://www.nexusmods.com/skyrim/mods/25859) to analyze a group of
+mod plugins to find conflicts and create patches to resolve them on a
+case-by-case basis. But, I was unsatisfied with that. I wanted to be assured
+that there would be no conflicts, or at least know the set of all possible mods
+out there that could conflict so I could manually patch those few mods. There
+was no good solution for finding conflicts across all mods though. Mod authors
+would need to download every Skyrim mod ever and no one has time to download all
+85,000+ Skyrim mods, and no one has the computer memory to load all of those in
+TES5Edit at the same time.
+
+Through that frustration, Modmapper was born with the mission to create a
+database of all Skyrim mod exterior cell edits. With that database I can power
+the website which visualizes how popular cells are in aggregate as well as allow
+the user to drill down to individual cells, mods, or plugins to find potential
+conflicts without ever having to download files themselves.
+
+When I [released the website about 7 months
+ago](https://www.reddit.com/r/skyrimmods/comments/sr8k4d/modmapper_over_14_million_cell_edits_from_every/)
+it made a big splash in the Skyrim modding community. No one had ever visualized
+mods on a map like this before, and it gave everyone a new perspective on the
+vast library of Skyrim mods. It was even [featured on the front page of PC
+Gamer's
+website](https://www.pcgamer.com/skyrim-modmapper-is-a-weirdly-beautiful-way-to-manage-your-mods/).
+Thirteen-year-old me, who regularly read the monthly PC Gamer magazine, would
+have been astounded.
+
+
+ 
+
+
+The comments posted to the initial mod I posted on Nexus Mods[^takedown] for the
+project were very amusing. It seemed to be blowing their minds:
+
+> "Quite possibly this could be the best mod for
+Skyrim. This hands-down makes everyone's life easier to be able to see which of
+their mods might be conflicting." -- [Nexus Mods comment by
+lorddonk](/img/blog/modmapper-comment15.png)
+
+> "The 8th wonder of Skyrim. That's a Titan's work requiring a monk's
+> perserverance. Finally, a place to go check (in)compatibilities !!! Voted.
+> Endorsed." -- [Nexus Mods comment by
+> jfjb2005](/img/blog/modmapper-comment3.png)
+
+> "They shall sing songs of your greatness! Wow, just wow." -- [Nexus Mods
+> comment by
+LumenMystic](/img/blog/modmapper-comment7.png)
+
+> "Holy Batman Tits! Be honest..... You're a Govt Agent and made this mod during
+> your "Terrorist Watch Shift" using a CIA super computer.." -- [Nexus Mods
+comment by toddrizzle](/img/blog/modmapper-comment1.png)
+
+> "What drugs are you on and can I have some?" -- [Nexus Mods comment by
+> thappysnek](/img/blog/modmapper-comment11.png)
+
+> "This is madness! Author are some kind of overhuman?! GREAT work!"-- [Nexus
+> Mods comment by TeodorWild](/img/blog/modmapper-comment10.png)
+
+> "You are an absolute legend. Bards will sing tales of your exploits" -- [Nexus
+> Mods comment by burntwater](/img/blog/modmapper-comment2.png)
+
+> "I wanted to say something, but I'll just kneel before thee and worship. This
+> would have taken me a lifetime. Amazing." -- [Nexus Mods comment by
+> BlueGunk](/img/blog/modmapper-comment8.png)
+
+> "Finally found the real dragonborn" -- [Nexus Mods comment by
+> yag1z](/img/blog/modmapper-comment6.png)
+
+> "he is the messiah!" -- [Nexus Mods comment by
+> Cursedobjects](/img/blog/modmapper-comment12.png)
+
+> "A god amongst men." -- [Nexus Mods comment by
+> TheMotherRobbit](/img/blog/modmapper-comment13.png)
+
+Apparently knowing how to program is now a god-like ability! This is the type of
+feedback most programmers aspire to get from their users. I knew the tool was
+neat and fun to build, but I didn't realize it was *that* sorely needed by the
+community.
+
+Today, Modmapper has a sustained user-base of around 7.5k unique visitors a
+month[^analytics] and I still see it mentioned in reddit threads or discord
+servers whenever someone is asking about the places a mod edits or what mods
+might be conflicting in a particular cell.
+
+The rest of this blog post will delve into how I built the website and how I
+gathered all of the data necessary to display the visualization.
+
+### Downloading ALL THE MODS!
+
+
+
+In order for the project to work I needed to collect all the Skyrim mod plugin
+files.
+
+While there are a number of places people upload Skyrim mods, [Nexus
+Mods](https://nexusmods.com) is conveniently the most popular and has the vast
+majority of mods. So, I would only need to deal with this one source. Luckily,
+[they have a nice API
+handy](https://app.swaggerhub.com/apis-docs/NexusMods/nexus-mods_public_api_params_in_form_data/1.0).
+
+[modmapper](https://github.com/thallada/modmapper) is the project I created to
+do this. It is a Rust binary that:
+
+* Uses [reqwest](https://crates.io/crates/reqwest) to make requests to [Nexus
+ Mods](https://nexusmods.com) for pages of last updated mods.
+* Uses [scraper](https://crates.io/crates/scraper) to scrape the HTML for
+ individual mod metadata (since the Nexus API doesn't provide an endpoint to
+ list mods).
+* Makes requests to the Nexus Mods API to get file and download information for
+ each mod, using [serde](https://serde.rs/) to parse the
+ [JSON](https://en.wikipedia.org/wiki/JSON) responses.
+* Requests the content preview data for each file and walks through the list of
+ files in the archive looking for a Skyrim plugin file (`.esp`, `.esm`, or
+ `.esl`).
+* If it finds a plugin, it decides to download the mod. It hits the download API
+ to get a download link and downloads the mod file archive.
+* Then it extracts the archive using one of:
+ [compress_tools](https://crates.io/crates/compress-tools),
+ [unrar](https://crates.io/crates/unrar), or [7zip](https://www.7-zip.org/) via
+ [`std::process::Commmand`](https://doc.rust-lang.org/std/process/struct.Command.html)
+ (depending on what type of archive it is).
+* With the ESP files (Elder Scrolls Plugin files) extracted, I then use my
+ [skyrim-cell-dump](https://github.com/thallada/skyrim-cell-dump) library (more
+ on that later!) to extract all of the cell edits into structured data.
+* Uses [seahash](https://crates.io/crates/seahash) to create a fast unique hash
+ for plugin files.
+* It then saves all of this data to a [postgres](https://www.postgresql.org/)
+ database using the [sqlx crate](https://crates.io/crates/sqlx).
+* Uses extensive logging with the [tracing
+ crate](https://crates.io/crates/tracing) so I can monitor the output and have
+ a history of a run to debug later if I discover an issue.
+
+It is designed to be run as a nightly [cron](https://en.wikipedia.org/wiki/Cron)
+job which downloads mods that have updated on Nexus Mods since the last run.
+
+To keep costs for this project low, I decided to make the website entirely
+static. So, instead of creating an API server that would have to be constantly
+running to serve requests from the website by making queries directly to the
+database, I would dump all of the data that the website needed from the database
+to JSON files, then upload those files to [Amazon
+S3](https://aws.amazon.com/s3/) and serve them through the [Cloudflare
+CDN](https://www.cloudflare.com/cdn/) which has servers all over the world.
+
+So, for example, every mod in the database has a JSON file uploaded to
+`https://mods.modmapper.com/skyrimspecialedition/.json` and the
+website frontend will fetch that file when a user clicks a link to that mod in
+the UI.
+
+The cost for S3 is pretty reasonable to me ($~3.5/month), and Cloudflare has a
+[generous free tier](https://www.cloudflare.com/plans/#price-matrix) that allows
+me to host everything through it for free.
+
+The server that I actually run `modmapper` on to download all of the mods is a
+server I already have at home that I also use for other purposes. The output of
+each run is uploaded to S3, and I also make a full backup of the database and
+plugin files to [Dropbox](https://www.dropbox.com).
+
+A lot of people thought it was insane that I downloaded every mod[^adult-mods],
+but in reality it wasn't too bad once I got all the issues resolved in
+`modmapper`. I just let it run in the background all day and it would chug
+through the list of mods one-by-one. Most of the time ended up being spent
+waiting while the Nexus Mod's API hourly rate limit was reached on my
+account.[^rate-limit]
+
+As a result of this project I believe I now have the most complete set of all
+Skyrim plugins to date (extracted plugins only without other textures, models,
+etc.)[^plugin-collection]. Compressed, it totals around 99 GB, uncompressed: 191
+GB.
+
+[After I downloaded Skyrim Classic mods in addition to Skyrim Special
+Edition](#finishing-the-collection-by-adding-all-skyrim-classic-mods), here are
+some counts from the database:
+
+|:---|:---|
+| **Mods** | 113,028 |
+| **Files** | 330,487 |
+| **Plugins** | 534,831 |
+| **Plugin Cell Edits** | 33,464,556 |
+
+### Parsing Skyrim plugin files
+
+The Skyrim game engine has a concept of
+[worldspaces](https://en.uesp.net/wiki/Skyrim:Worldspaces) which are exterior
+areas where the player can travel to. The biggest of these being, of course,
+Skyrim itself (which, in the lore, is a province of the continent of
+[Tamriel](https://en.uesp.net/wiki/Lore:Tamriel) on the planet
+[Nirn](https://en.uesp.net/wiki/Lore:Nirn)). Worldspaces are recorded in a
+plugin file as [WRLD
+records](https://en.uesp.net/wiki/Skyrim_Mod:Mod_File_Format/WRLD).
+
+Worldspaces are then chunked up into a square grid of cells. The Skyrim
+worldspace consists of a little over 11,000 square cells. Mods that make a
+changes to the game world have a record in the plugin (a [CELL
+record](https://en.uesp.net/wiki/Skyrim_Mod:Mod_File_Format/CELL)) with the
+cell's X and Y coordinates and a list changes in that cell.
+
+There is some prior art ([esplugin](https://github.com/Ortham/esplugin),
+[TES5Edit](https://github.com/TES5Edit/TES5Edit),
+[zedit](https://github.com/z-edit/zedit)) of open-source programs that could
+parse Skyrim plugins and extract this data. However, all of these were too broad
+for my purpose or relied on the assumption of being run in the context of a load
+order where the master files of a plugin would also be available. I wanted a
+program that could take a single plugin in isolation and skip through all of the
+non-relevant parts of it and dump just the CELL and WRLD record data plus some
+metadata about the plugin from the header as fast as possible.
+
+After discovering [the wonderful documentation on the UESP wiki about the Skyrim
+mod file format](https://en.uesp.net/wiki/Skyrim_Mod:Mod_File_Format), I
+realized this would be something that would be possible to make myself.
+[skyrim-cell-dump](https://github.com/thallada/skyrim-cell-dump) is a Rust
+library/CLI program that accepts a Skyrim mod file and spits out the header
+metadata of the plugin, the worlds edited/created, and all of the cells it
+edits/creates.
+
+Under the hood, it uses the [nom crate](https://crates.io/crates/nom) to read
+through the plugin until it finds the relevant records, then uses
+[flate2](https://crates.io/crates/flate2) to decompress any compressed record
+data, and finally outputs the extracted data formatted to JSON with
+[serde](https://crates.io/crates/serde).
+
+Overall, I was pretty happy with this toolkit of tools and was able to quickly
+get the data I needed from plugins. My only gripe was that I never quite figured
+out how to properly do error handling with nom. If there was ever an error, I
+didn't get much data in the error about what failed besides what function it
+failed in. I often had to resort to peppering in a dozen `dbg!()` statements to
+figure out what went wrong.
+
+I built it as both a library and binary crate so that I could import it in other
+libraries and get the extracted data directly as Rust structs without needing to
+go through JSON. I'll go more into why this was useful later.
+
+### Building the website
+
+Since I wanted to keep server costs low and wanted the site to be as fast as
+possible for users, I decided pretty early on that the site would be purely
+static HTML and JavaScript with no backend server. I decided to use the [Next.js
+web framework](https://nextjs.org/) with
+[TypeScript](https://www.typescriptlang.org/) since it was what I was familiar
+with using in my day job. While it does have [server-side rendering
+support](https://nextjs.org/docs/basic-features/pages#server-side-rendering)
+which would require running a backend [Node.js](https://nodejs.org/en/) server,
+it also supports a limited feature-set that can be [exported as static
+HTML](https://nextjs.org/docs/advanced-features/static-html-export).
+
+I host the site on [Cloudflare pages](https://pages.cloudflare.com/) which is
+available on their free tier and made deploying from Github commits a
+breeze[^cloudflare]. The web code is in my [modmapper-web
+repo](https://github.com/thallada/modmapper-web).
+
+The most prominent feature of the website is the interactive satellite map of
+Skyrim. Two essential resources made this map possible: [the map tile images
+from the UESP skyrim map](https://srmap.uesp.net/) and
+[Mapbox](https://www.mapbox.com/).
+
+[Mapbox provides a JS library for its WebGL
+map](https://docs.mapbox.com/mapbox-gl-js/api/) which allows specifying a
+[raster tile
+source](https://docs.mapbox.com/mapbox-gl-js/example/map-tiles/)[^3d-terrain].
+
+The [UESP team painstakingly loaded every cell in the Skyrim worldspace in the
+Creation Kit and took a
+screenshot](https://en.uesp.net/wiki/UESPWiki:Skyrim_Map_Design). Once I figured
+out which image tiles mapped to which in-game cell it was relatively easy to put
+a map together by plugging them into the Mapbox map as a raster tile source.
+
+The heatmap overlaid on the map is created using a [Mapbox
+layer](https://docs.mapbox.com/help/glossary/layer/) that fills a cell with a
+color on a gradient from green to red depending on how many edits that cell has
+across the whole database of mods.
+
+
+
+The sidebar on the site is created using [React](https://reactjs.org/) and
+[Redux](https://redux.js.org/) and uses the
+[next/router](https://nextjs.org/docs/api-reference/next/router) to keep track
+of which page the user is on with URL parameters.
+
+
+
+
+
+
+
+
+The mod search is implemented using
+[MiniSearch](https://lucaong.github.io/minisearch/) that asynchronously loads
+the giant search indices for each game containing every mod name and id.
+
+
+
+One of the newest features of the site allows users to drill down to a
+particular plugin within a file of a mod and "Add" it to their list. All of the
+added plugins will be listed in the sidebar and the cells they edit displayed in
+purple outlines and conflicts between them displayed in red outlines.
+
+
+
+### Loading plugins client-side with WebAssembly
+
+A feature that many users requested after the initial release was being able to
+load a list of the mods currently installed on their game and see which ones of
+that set conflict with each other[^second-announcement]. Implementing this
+feature was one of the most interesting parts of the project. Choosing to use
+Rust made made it possible, since everything I was running server-side to
+extract the plugin data could also be done client-side in the browser with the
+same Rust code compiled to [WebAssembly](https://webassembly.org/).
+
+I used [wasm-pack](https://github.com/rustwasm/wasm-pack) to create
+[skyrim-cell-dump-wasm](https://github.com/thallada/skyrim-cell-dump-wasm/)
+which exported the `parse_plugin` function from my
+[skyrim-cell-dump](https://github.com/thallada/skyrim-cell-dump) Rust library
+compiled to WebAssembly. It also exports a `hash_plugin` function that creates a
+unique hash for a plugin file's slice of bytes using
+[seahash](https://crates.io/crates/seahash) so the site can link plugins a user
+has downloaded on their hard-drive to plugins that have been downloaded by
+modmapper and saved in the database.
+
+Dragging-and-dropping the Skyrim Data folder on to the webpage or selecting the
+folder in the "Open Skyrim Data directory" dialog kicks off a process that
+starts parsing all of the plugin files in that directory in parallel using [Web
+Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers).
+
+I developed my own
+[`WorkerPool`](https://github.com/thallada/modmapper-web/blob/4af628559030c3f24618b29b46d4a40af2f200a6/lib/WorkerPool.ts)
+that manages creating a pool of available workers and assigns them to plugins to
+process. The pool size is the number of cores on the user's device so that the
+site can process as many plugins in parallel as possible. After a plugin
+finishes processing a plugin and sends the output to the redux store, it gets
+added back to the pool and is then assigned a new plugin to process if there are
+any[^wasm-troubles].
+
+Once all plugins have been loaded, the map updates by displaying all of the
+cells edited in a purple box and any cells that are edited by more than one
+plugin in a red box.
+
+
+
+Users can also drag-and-drop or paste their `plugins.txt` file, which is the
+file that the game uses to define the load order of plugins and which plugins
+are enabled or disabled. Adding the `plugins.txt` sorts the list of loaded
+plugins in the sidebar in load order and enables or disables plugins as defined
+in the `plugins.txt`.
+
+
+
+Selecting a cell in the map will display all of the loaded cells that edit that
+cell in the sidebar.
+
+
+
+The ability to load plugins straight from a user's hard-drive allows users to
+map mods that haven't even been uploaded to Nexus Mods.
+
+### Vortex integration
+
+The initial mod I released on the Skyrim Special Edition page of Nexus Mods was
+[taken
+down](https://www.reddit.com/r/skyrimmods/comments/svnz4a/modmapper_got_removed/)
+by the site admins since it didn't contain an actual mod and they didn't agree
+that it qualified as a "Utility".
+
+Determined to have an actual mod page for Modmapper on Nexus Mods, I decided to
+make a [Vortex](https://www.nexusmods.com/about/vortex/) integration for
+modmapper. Vortex is a mod manager made by the developers of Nexus Mods and they
+allow creating extensions to the tool and have their own [mod section for Vortex
+extensions](https://www.nexusmods.com/site).
+
+With the help of [Pickysaurus](https://www.nexusmods.com/skyrim/users/31179975),
+one of the community managers for Nexus Mods, I created a [Vortex integration
+for Modmapper](https://www.nexusmods.com/site/mods/371). It adds a context menu
+option on mods to view the mod in Modmapper with all of the cells it edits
+selected in purple. It also adds a button next to every plugin file to view just
+that plugin in Modmapper (assuming it has been processed by Modmapper).
+
+
+
+
+
+
+
+
+To enable the latter part, I had to include `skyrim-cell-dump-wasm` in the
+extension so that I could hash the plugin contents with `seahash` to get the
+same hash that Modmapper would have generated. It only does this hashing when
+you click the "See on Modmapper" button to save from excessive CPU usage when
+viewing the plugin list.
+
+After releasing the Vortex plugin, Pickysaurus [published a news article about
+modmapper to the Skyrim Special Edition
+site](https://www.nexusmods.com/skyrimspecialedition/news/14678) which also got
+a lot of nice comments ❤️.
+
+### Finishing the collection by adding all Skyrim Classic mods
+
+Skyrim is very silly in that it has [many
+editions](https://ag.hyperxgaming.com/article/12043/every-skyrim-edition-released-over-the-last-decade).
+But there was only one that split the modding universe into two: [Skyrim Special
+Edition (SE)](https://en.uesp.net/wiki/Skyrim:Special_Edition).
+It was released in October 2016 with a revamped game engine that brought some
+sorely needed graphical upgrades. However, it also contained changes to how mods
+worked, requiring all mod authors to convert their mods to SE. This created big
+chasm in the library of mods, and Nexus Mods had to make a separate section for
+SE-only mods.
+
+When I started downloading mods in 2021, I started only with Skyrim SE mods,
+which, at the time of writing, totals at over [55,000 mods on Nexus
+Mods](https://www.nexusmods.com/skyrimspecialedition/mods/).
+
+After releasing with just SE mods, many users requested that all of the classic
+pre-SE Skyrim mods be added as well. This month, I finally finished downloading
+all Skyrim Classic mods, which, at the time of writing, totals at over [68,000
+mods on Nexus Mods](https://www.nexusmods.com/skyrim/mods/). That brings the
+total downloaded and processed mods for Modmapper at over 113,000
+mods[^adult-mods]!
+
+### The future
+
+A lot of users had great feedback and suggestions on what to add to the site. I
+could only implement so many of them, though. The rest I've been keeping track
+of on [this Trello board](https://trello.com/b/VdpTQ7ar/modmapper).
+
+Some of the headline items on it are:
+
+* Add [Solstheim map](https://dbmap.uesp.net/)
+
+ Since map tiles images are available for that worldspace and because I have
+ already recorded edits to the worldspace in my database, it shouldn't be too
+ terribly difficult.
+* Add [Mod Organizer 2](https://www.modorganizer.org/) plugin
+
+ Lots of people requested this since it's a very popular mod manager compared
+ to Vortex. MO2 supports python extensions so I created
+ [skyrim-cell-dump-py](https://github.com/thallada/skyrim-cell-dump-py) to
+ export the Rust plugin processing code to a Python library. I got a bit stuck
+ on actually creating the plugin though, so it might be a while until I get to
+ that.
+* Find a way to display interior cell edits on the map
+
+ The map is currently missing edits to interior cells. Since almost all
+ interior cells in Skyrim have a link to the exterior world through a door
+ teleporter, it should be possible to map an interior cell edit to an exterior
+ cell on the map based on which cell the door leads out to.
+
+ That will require digging much more into the plugin files for more data, and
+ city worldspaces will complicate things further. Then there's the question of
+ interiors with multiple doors to different exterior cells, or interior cells
+ nested recursively deep within many other interior cells.
+* Create a standalone [Electron](https://www.electronjs.org) app that can run
+ outside the browser
+
+ I think this would solve a lot of the issues I ran into while developing the
+ website. Since Electron has a Node.js process running on the user's computer
+ outside the sandboxed browser process, it gives me much more flexibility. It
+ could do things like automatically load a user's plugin files. Or just load
+ plugins at all wihtout having to deal with the annoying dialog that lies to
+ the user saying they're about to upload their entire Data folder hundreds of
+ gigabytes full of files to a server (I really wish the
+ [HTMLInputElement.webkitdirectory](https://developer.mozilla.org/en-US/docs/Web/API/HTMLInputElement/webkitdirectory)
+ API would use the same underlying code as the [HTML Drag and Drop
+ API](https://developer.mozilla.org/en-US/docs/Web/API/HTML_Drag_and_Drop_API)
+ which is a lot better).
+* Improving the search
+
+ The mod search feature struggles the most with the static generated nature of
+ the site. I found it very hard to pack all of the necessary info for the
+ search index for all 100k+ mods (index for both SE and LE is around 6 MB).
+ Asynchronously loading the indices with MiniSearch keeps it from freezing up
+ the browser, but it does take a very long time to fully load. I can't help
+ think that there's a better way to shard the indices somehow and only fetch
+ what I need based on what the user is typing into the search.
+
+To be clear, a lot of the Todos on the board are pipe-dreams. I may never get to
+them. This project is sustained purely by my motivation and self-interests and
+if something is too much of a pain to get working I'll just drop it.
+
+There will also be future Elder Scrolls games, and [future Bethesda games based
+on roughly the same game engine](https://bethesda.net/en/game/starfield). It
+would be neat to create similar database for those games as the modding
+community develops in realtime.
+
+Overall, I'm glad I made something of use to the modding community. I hope to
+keep the site running for as long as people are modding Skyrim (until the
+heat-death of the universe, probably).
+
+
+
+
+---
+
+#### Footnotes
+
+[^bazaarrealm]:
+ Unfortunately, I basically lost interest on the mod after working on
+ Modmapper. I might still write a blog post about it eventually since I did a
+ lot of interesting hacking on the Skyrim game engine to try to add some
+ asynchronous multiplayer aspects. [Project is here if anyone is curious in
+ the meantime](https://github.com/thallada/BazaarRealmPlugin).
+
+[^takedown]:
+ I sadly only have screenshots for some of the comments on that mod since it
+ was eventually taken down by the Nexus Mod admins. See explanation about
+ that in the [Vortex integration section](#vortex-integration).
+
+[^analytics]:
+ As recorded by Cloudflare's server side analytics, which may record a fair
+ amount of bot traffic. I suspect this is the most accurate number I can get
+ since most of my users probably use an ad blocker that blocks client-side
+ analytics.
+
+[^adult-mods]:
+ Every mod on Nexus Mods except for adult mods since the site restricts
+ viewing adult mods to only logged-in users and I wasn't able to get my
+ scraping bot to log in as a user.
+
+[^rate-limit]:
+ Apparently my mass-downloading did not go unnoticed by the Nexus Mod admins.
+ I think it's technically against their terms of service to automatically
+ download mods, but I somehow got on their good side and was spared the
+ ban-hammer. I don't recommend anyone else run modmapper themselves on the
+ entire site unless you talk to the admins beforehand and get the okay from
+ them.
+
+[^plugin-collection]:
+ If you would like access to this dataset of plugins to do some data-mining
+ please reach out to me at [tyler@hallada.net](mailto:tyler@hallada.net)
+ (Note: only contains plugins files, no models, textures, audio, etc.). I
+ don't plan on releasing it publicly since that would surely go against many
+ mod authors' wishes/licenses.
+
+[^cloudflare]:
+ I'm not sure I want to recommend anyone else use Cloudflare after [the whole
+ Kiwi Farms
+ debacle](https://www.theverge.com/2022/9/6/23339889/cloudflare-kiwi-farms-content-moderation-ddos).
+ I now regret having invested so much of the infrastructure in them. However,
+ I'm only using their free-tier, so at least I am a net-negative for their
+ business? I would recommend others look into
+ [Netlify](https://www.netlify.com/) or [fastly](https://www.fastly.com/) for
+ similar offerings to Cloudflare pages/CDN.
+
+[^3d-terrain]:
+ I also tried to add a [raster Terrain-DEM
+ source](https://docs.mapbox.com/data/tilesets/reference/mapbox-terrain-dem-v1/)
+ for rendering the terrain in 3D. I got fairly far [generating my own DEM RGB
+ tiles](https://github.com/syncpoint/terrain-rgb) from an upscaled [greyscale
+ heightmap](https://i.imgur.com/9RErBDo.png) [constructed from the LAND
+ records in Skyrim.esm](https://www.nexusmods.com/skyrim/mods/80692) (view it
+ [here](https://www.dropbox.com/s/56lffk021riil6h/heightmap-4x_foolhardy_Remacri_rgb.tif?dl=0)).
+ But, it came out all wrong: [giant cliffs in the middle of the
+ map](/img/blog/modmapper-terrain-cliff.jpg) and [tiny spiky lumps with big
+ jumps in elevation at cell boundaries](/img/blog/modmapper-bad-terrain.jpg).
+ Seemed like too much work to get right than it was worth it.
+
+[^second-announcement]:
+ [This was the announcement I posted to /r/skyrimmods for this feature](
+https://www.reddit.com/r/skyrimmods/comments/ti3gjh/modmapper_update_load_plugins_in_your_load_order/)
+
+[^wasm-troubles]:
+ At first, I noticed a strange issue with re-using the same worker on
+ different plugins multiple times. After a while (~30 reuses per worker), the
+ processing would slow to a crawl and eventually strange things started
+ happening (I was listening to music in my browser and it started to pop and
+ crack). It seemed like the speed of processing increased exponentially to
+ the number of times the worker was reused. So, to avoid this, I had to make
+ the worker pool terminate and recreate workers after every plugin processed.
+ This ended up not being as slow as it sounds and worked fine. However, I
+ recently discovered that [wee_alloc, the most suggested allocator to use
+ with rust in wasm, has a memory leak and is mostly unmaintained
+ now](https://www.reddit.com/r/rust/comments/x1cle0/dont_use_wee_alloc_in_production_code_targeting/).
+ I switched to the default allocator and I didn't run into the exponentially
+ slow re-use problem. For some reason, the first run on a fresh tab is always
+ much faster than the second run, but subsequent runs are still fairly stable
+ in processing time.
+
diff --git a/img/blog/allthemods.jpg b/img/blog/allthemods.jpg
new file mode 100755
index 0000000..c110b1a
Binary files /dev/null and b/img/blog/allthemods.jpg differ
diff --git a/img/blog/modmapper-added-plugins.jpg b/img/blog/modmapper-added-plugins.jpg
new file mode 100755
index 0000000..1a46dcd
Binary files /dev/null and b/img/blog/modmapper-added-plugins.jpg differ
diff --git a/img/blog/modmapper-bad-terrain.jpg b/img/blog/modmapper-bad-terrain.jpg
new file mode 100755
index 0000000..5a9ab8e
Binary files /dev/null and b/img/blog/modmapper-bad-terrain.jpg differ
diff --git a/img/blog/modmapper-cell-sidebar.jpg b/img/blog/modmapper-cell-sidebar.jpg
new file mode 100755
index 0000000..a81d1c0
Binary files /dev/null and b/img/blog/modmapper-cell-sidebar.jpg differ
diff --git a/img/blog/modmapper-clipping-example1.jpg b/img/blog/modmapper-clipping-example1.jpg
new file mode 100755
index 0000000..39af481
Binary files /dev/null and b/img/blog/modmapper-clipping-example1.jpg differ
diff --git a/img/blog/modmapper-clipping-example2.jpg b/img/blog/modmapper-clipping-example2.jpg
new file mode 100755
index 0000000..5e30efd
Binary files /dev/null and b/img/blog/modmapper-clipping-example2.jpg differ
diff --git a/img/blog/modmapper-clipping-example3.jpg b/img/blog/modmapper-clipping-example3.jpg
new file mode 100755
index 0000000..cbae0d4
Binary files /dev/null and b/img/blog/modmapper-clipping-example3.jpg differ
diff --git a/img/blog/modmapper-clipping-example4.jpg b/img/blog/modmapper-clipping-example4.jpg
new file mode 100755
index 0000000..a096485
Binary files /dev/null and b/img/blog/modmapper-clipping-example4.jpg differ
diff --git a/img/blog/modmapper-comment1.png b/img/blog/modmapper-comment1.png
new file mode 100755
index 0000000..8c9ab3e
Binary files /dev/null and b/img/blog/modmapper-comment1.png differ
diff --git a/img/blog/modmapper-comment10.png b/img/blog/modmapper-comment10.png
new file mode 100755
index 0000000..c934cc4
Binary files /dev/null and b/img/blog/modmapper-comment10.png differ
diff --git a/img/blog/modmapper-comment11.png b/img/blog/modmapper-comment11.png
new file mode 100755
index 0000000..48a2c55
Binary files /dev/null and b/img/blog/modmapper-comment11.png differ
diff --git a/img/blog/modmapper-comment12.png b/img/blog/modmapper-comment12.png
new file mode 100755
index 0000000..87e1267
Binary files /dev/null and b/img/blog/modmapper-comment12.png differ
diff --git a/img/blog/modmapper-comment13.png b/img/blog/modmapper-comment13.png
new file mode 100755
index 0000000..9feb53d
Binary files /dev/null and b/img/blog/modmapper-comment13.png differ
diff --git a/img/blog/modmapper-comment15.png b/img/blog/modmapper-comment15.png
new file mode 100755
index 0000000..f96f542
Binary files /dev/null and b/img/blog/modmapper-comment15.png differ
diff --git a/img/blog/modmapper-comment16.png b/img/blog/modmapper-comment16.png
new file mode 100755
index 0000000..5b1867e
Binary files /dev/null and b/img/blog/modmapper-comment16.png differ
diff --git a/img/blog/modmapper-comment2.png b/img/blog/modmapper-comment2.png
new file mode 100755
index 0000000..a3644b8
Binary files /dev/null and b/img/blog/modmapper-comment2.png differ
diff --git a/img/blog/modmapper-comment3.png b/img/blog/modmapper-comment3.png
new file mode 100755
index 0000000..7122438
Binary files /dev/null and b/img/blog/modmapper-comment3.png differ
diff --git a/img/blog/modmapper-comment4.png b/img/blog/modmapper-comment4.png
new file mode 100755
index 0000000..d77f41a
Binary files /dev/null and b/img/blog/modmapper-comment4.png differ
diff --git a/img/blog/modmapper-comment5.png b/img/blog/modmapper-comment5.png
new file mode 100755
index 0000000..f83f49c
Binary files /dev/null and b/img/blog/modmapper-comment5.png differ
diff --git a/img/blog/modmapper-comment6.png b/img/blog/modmapper-comment6.png
new file mode 100755
index 0000000..a795b29
Binary files /dev/null and b/img/blog/modmapper-comment6.png differ
diff --git a/img/blog/modmapper-comment7.png b/img/blog/modmapper-comment7.png
new file mode 100755
index 0000000..fd8054d
Binary files /dev/null and b/img/blog/modmapper-comment7.png differ
diff --git a/img/blog/modmapper-comment8.png b/img/blog/modmapper-comment8.png
new file mode 100755
index 0000000..be3bdd0
Binary files /dev/null and b/img/blog/modmapper-comment8.png differ
diff --git a/img/blog/modmapper-conflicted-cell.jpg b/img/blog/modmapper-conflicted-cell.jpg
new file mode 100755
index 0000000..8395f19
Binary files /dev/null and b/img/blog/modmapper-conflicted-cell.jpg differ
diff --git a/img/blog/modmapper-heatmap-closeup.jpg b/img/blog/modmapper-heatmap-closeup.jpg
new file mode 100755
index 0000000..f43dc12
Binary files /dev/null and b/img/blog/modmapper-heatmap-closeup.jpg differ
diff --git a/img/blog/modmapper-loaded-plugins.jpg b/img/blog/modmapper-loaded-plugins.jpg
new file mode 100755
index 0000000..b9920af
Binary files /dev/null and b/img/blog/modmapper-loaded-plugins.jpg differ
diff --git a/img/blog/modmapper-mod-sidebar.jpg b/img/blog/modmapper-mod-sidebar.jpg
new file mode 100755
index 0000000..91e54bf
Binary files /dev/null and b/img/blog/modmapper-mod-sidebar.jpg differ
diff --git a/img/blog/modmapper-pcgamer.jpg b/img/blog/modmapper-pcgamer.jpg
new file mode 100755
index 0000000..60841b2
Binary files /dev/null and b/img/blog/modmapper-pcgamer.jpg differ
diff --git a/img/blog/modmapper-pluginstxt-dialog.jpg b/img/blog/modmapper-pluginstxt-dialog.jpg
new file mode 100755
index 0000000..06f7aef
Binary files /dev/null and b/img/blog/modmapper-pluginstxt-dialog.jpg differ
diff --git a/img/blog/modmapper-search.jpg b/img/blog/modmapper-search.jpg
new file mode 100755
index 0000000..8f8c3de
Binary files /dev/null and b/img/blog/modmapper-search.jpg differ
diff --git a/img/blog/modmapper-terrain-cliff.jpg b/img/blog/modmapper-terrain-cliff.jpg
new file mode 100755
index 0000000..55320df
Binary files /dev/null and b/img/blog/modmapper-terrain-cliff.jpg differ
diff --git a/img/blog/modmapper-vortex-mod-menu.jpg b/img/blog/modmapper-vortex-mod-menu.jpg
new file mode 100755
index 0000000..6d93c0a
Binary files /dev/null and b/img/blog/modmapper-vortex-mod-menu.jpg differ
diff --git a/img/blog/modmapper-vortex-plugin-button.jpg b/img/blog/modmapper-vortex-plugin-button.jpg
new file mode 100755
index 0000000..98c0b28
Binary files /dev/null and b/img/blog/modmapper-vortex-plugin-button.jpg differ
diff --git a/img/blog/modmapper.jpg b/img/blog/modmapper.jpg
new file mode 100755
index 0000000..0abeea0
Binary files /dev/null and b/img/blog/modmapper.jpg differ
+ 
+ 
+ 
+ 
+ 
+ 
+ 
+