165 changed files with 178 additions and 4223 deletions
--- a/.ruby-version
+++ b/.ruby-version
@ -1 +0,0 @@
 3.1.2
--- a/.well-known/keybase.txt
+++ b/.well-known/keybase.txt
@ -1,54 +0,0 @@
 ==================================================================
 https://keybase.io/thallada
 --------------------------------------------------------------------
 I hereby claim:
  * I am an admin of http://www.hallada.net
  * I am thallada (https://keybase.io/thallada) on keybase.
  * I have a public key ASDOaZj16vBUa4vjFa4dhC7map8qIX5MUSCqjgWeX1CfbQo
 To do so, I am signing this object:
 {
    "body": {
        "key": {
            "eldest_kid": "0120ce6998f5eaf0546b8be315ae1d842ee66a9f2a217e4c5120aa8e059e5f509f6d0a",
            "host": "keybase.io",
            "kid": "0120ce6998f5eaf0546b8be315ae1d842ee66a9f2a217e4c5120aa8e059e5f509f6d0a",
            "uid": "bf2238122821bbc309d5bf1ed2421d19",
            "username": "thallada"
        },
        "service": {
            "hostname": "www.hallada.net",
            "protocol": "http:"
        },
        "type": "web_service_binding",
        "version": 1
    },
    "client": {
        "name": "keybase.io go client",
        "version": "1.0.18"
    },
    "ctime": 1486587579,
    "expire_in": 504576000,
    "merkle_root": {
        "ctime": 1486587569,
        "hash": "099473519e50871bbe05a57da09e5ba8fa575e8a51e259dd6ad6e8e4ead07fdb63895424659bc01aef600d5caa2ddaf32bd194fec6e08b54f623c0bf381df30c",
        "seqno": 846152
    },
    "prev": "7ec1319be81dde8362b777b669904944eaebd592206c17953c628749b41cf992",
    "seqno": 9,
    "tag": "signature"
 }
 which yields the signature:
 hKRib2R5hqhkZXRhY2hlZMOpaGFzaF90eXBlCqNrZXnEIwEgzmmY9erwVGuL4xWuHYQu5mqfKiF+TFEgqo4Fnl9Qn20Kp3BheWxvYWTFAvZ7ImJvZHkiOnsia2V5Ijp7ImVsZGVzdF9raWQiOiIwMTIwY2U2OTk4ZjVlYWYwNTQ2YjhiZTMxNWFlMWQ4NDJlZTY2YTlmMmEyMTdlNGM1MTIwYWE4ZTA1OWU1ZjUwOWY2ZDBhIiwiaG9zdCI6ImtleWJhc2UuaW8iLCJraWQiOiIwMTIwY2U2OTk4ZjVlYWYwNTQ2YjhiZTMxNWFlMWQ4NDJlZTY2YTlmMmEyMTdlNGM1MTIwYWE4ZTA1OWU1ZjUwOWY2ZDBhIiwidWlkIjoiYmYyMjM4MTIyODIxYmJjMzA5ZDViZjFlZDI0MjFkMTkiLCJ1c2VybmFtZSI6InRoYWxsYWRhIn0sInNlcnZpY2UiOnsiaG9zdG5hbWUiOiJ3d3cuaGFsbGFkYS5uZXQiLCJwcm90b2NvbCI6Imh0dHA6In0sInR5cGUiOiJ3ZWJfc2VydmljZV9iaW5kaW5nIiwidmVyc2lvbiI6MX0sImNsaWVudCI6eyJuYW1lIjoia2V5YmFzZS5pbyBnbyBjbGllbnQiLCJ2ZXJzaW9uIjoiMS4wLjE4In0sImN0aW1lIjoxNDg2NTg3NTc5LCJleHBpcmVfaW4iOjUwNDU3NjAwMCwibWVya2xlX3Jvb3QiOnsiY3RpbWUiOjE0ODY1ODc1NjksImhhc2giOiIwOTk0NzM1MTllNTA4NzFiYmUwNWE1N2RhMDllNWJhOGZhNTc1ZThhNTFlMjU5ZGQ2YWQ2ZThlNGVhZDA3ZmRiNjM4OTU0MjQ2NTliYzAxYWVmNjAwZDVjYWEyZGRhZjMyYmQxOTRmZWM2ZTA4YjU0ZjYyM2MwYmYzODFkZjMwYyIsInNlcW5vIjo4NDYxNTJ9LCJwcmV2IjoiN2VjMTMxOWJlODFkZGU4MzYyYjc3N2I2Njk5MDQ5NDRlYWViZDU5MjIwNmMxNzk1M2M2Mjg3NDliNDFjZjk5MiIsInNlcW5vIjo5LCJ0YWciOiJzaWduYXR1cmUifaNzaWfEQIiv6L2Js0MEXpgiHcIhP9B3MmBl+81QA0z32QYT5XWXsH/6rsylYYQCLWjrIXAILrOIoH5Jyd2GFfpU+O8A/w2oc2lnX3R5cGUgpGhhc2iCpHR5cGUIpXZhbHVlxCDXYP2O217NRz/lX6Q6G54D0G6/EIfX2OJY/hduqsrYNqN0YWfNAgKndmVyc2lvbgE=
 And finally, I am proving ownership of this host by posting or
 appending to this document.
 View my publicly-auditable identity here: https://keybase.io/thallada
 ==================================================================
--- a/.well-known/matrix/server
+++ b/.well-known/matrix/server
@ -1,3 +0,0 @@
 {
    "m.server": "synapse.hallada.net:443"
 }
--- a/2
+++ b/2
@ -1 +1 @@
-www.hallada.net
+www.hallada.net
--- a/4
+++ b/4
@ -1,4 +1,2 @@
 source 'http://rubygems.org'
-
+gem 'github-pages'
 gem "webrick"
 gem 'github-pages', group: :jekyll_plugins
--- a/README.md
+++ b/README.md
@ -4,21 +4,9 @@ thallada.github.io
 This is the latest version of my personal website. It is a static website built
 with [Jekyll](http://jekyllrb.com/).
-See it at [https://www.hallada.net/](https://www.hallada.net/).
+See it at [http://www.hallada.net/](http://www.hallada.net/).
 ## Build Locally
 To run a version of the site in development locally. Checkout this repo and 
 then:
 1. `cd thallada.github.io`
 2. [Install Jekyll](https://jekyllrb.com/docs/installation/)
 3. Run `bundle install`
 3. Run `bundle exec jekyll serve`
 4. Visit `http://localhost:4000` to view the website
 ## Magic
 ##Magic##
 Most of the development work of this website went into creating what I like to
 call "magic", or the dynamic background to my homepage. A few seconds after
 loading the page, a branching web of colored tendrils will grow in a random
@ -41,14 +29,11 @@ random, colorful, and more CPU efficient.
 It was really fun to tweak various variables in the script and see how the
 animation reacted. It didn't take much tweaking to get the lines to appear like
 lightning flashing in the distant background, or like cracks splitting the
-screen, or like growing forest of sprouting trees.
+screen, or like growing forest of sprouting trees. A future project may involve
-
+putting the magic up on its own webpage and add UI dials to allow anyone to
-You can play around with these variables yourself on the [/magic
+change these variables in realtime.
 page](https://www.hallada.net/magic) which has sliders for tweaking the
 animations in realtime.
 ## Layout & CSS
 ##Layout & CSS##
 I use a [grid system devised by Adam Kaplan](http://www.adamkaplan.me/grid/) and
 with some pieces from [Jorden Lev](http://jordanlev.github.io/grid/). It is
 set-up by scratch in my `main.css`. I decided on this so that I would not have
@ -120,37 +105,11 @@ desktop, use `hide-desktop` instead.
 </div>
 ```
 I had an issue with displaying elements on desktop that had the class
 "hide-mobile", so you can add the following classes to make sure they redisplay
 in the right display type correctly:
 * `hide-mobile-block`
 * `hide-mobile-inline-block`
 * `hide-mobile-inline`
 * `hide-deskop-block`
 * `hide-desktop-inline-block`
 * `hide-desktop-inline`
 I could add more for each `display` property, but I'm trying to find a better
 way of fixing this without adding these second classes.
 Another note: I use [box-sizing (as suggested by Paul
 Irish)](http://www.paulirish.com/2012/box-sizing-border-box-ftw/), which I think
 makes dealing with sizing elements a lot more sane.
-### Light & Dark Themes
+##Attributions##
 In 2020, I created a dark theme for the website. The dark theme is used if it 
 detects that the user's OS is set to prefer a dark theme [using the 
 `prefers-color-scheme` `@media` 
 query](https://css-tricks.com/dark-modes-with-css/).
 To allow the user to select a theme separate from their OS's theme, I have also 
 included [a switch that can toggle between the two 
 themes](https://github.com/GoogleChromeLabs/dark-mode-toggle).
 ## Attributions
 [Book](http://thenounproject.com/term/book/23611/) designed by [Nherwin
 Ardoña](http://thenounproject.com/nherwinma) from the Noun Project.
--- a/_config.yml
+++ b/_config.yml
@ -1,32 +1,9 @@
 title: Tyler Hallada
 name: Tyler Hallada - Blog
 description: Musings on technology, literature, and interesting topics
-author: thallada
+url: http://hallada.net/blog
-url: https://www.hallada.net
+markdown: redcarpet
-blog_url: https://www.hallada.net/blog
+pygments: true
 assets: https://hallada.net/assets/
 logo: /img/profile_icon_128x128.jpg
 social:
    name: Tyler Hallada
    links:
        - https://twitter.com/tyhallada
        - https://www.facebook.com/tyhallada
        - https://www.linkedin.com/in/thallada/
        - https://github.com/thallada
 defaults:
    -
        scope:
            path: ""
        values:
            image: /img/profile_icon_300x200.jpg
 markdown: kramdown
 kramdown:
    syntax_highlighter: rouge
 excerpt_separator: "<!--excerpt-->"
 paginate: 10
 paginate_path: "blog/page:num"
 gems:
    - jekyll-redirect-from
    - jekyll-paginate
    - jekyll-seo-tag
 include: [".well-known"]
--- a/_data/authors.yml
+++ b/_data/authors.yml
@ -1,3 +0,0 @@
 thallada:
    picture: /img/profile_icon_128x128.jpg
    twitter: tyhallada
--- a/_includes/comments.html
+++ b/_includes/comments.html
@ -1,12 +0,0 @@
 <div class="card">
  <div class="row clearfix">
    <div class="column full">
      <script data-isso="https://comments.hallada.net/"
              src="https://comments.hallada.net/js/embed.min.js"></script>
      <section id="isso-thread">
        <noscript>Javascript needs to be activated to view comments.</noscript>
      </section>
    </div>
  </div>
 </div>
--- a/_includes/mail-form.html
+++ b/_includes/mail-form.html
@ -1,36 +0,0 @@
 <div class="card">
  <div class="subscribe-form">
    <div class="row clearfix">
      <div class="column full">
        <h3>Subscribe to my future posts</h3>
      </div>
    </div>
    <form action="https://list.hallada.net/subscribe" method="POST" accept-charset="utf-8">
      <div class="row clearfix">
        <div class="column half">
          <label for="name">Name (optional)</label><br />
          <input type="text" name="name" id="name" />
        </div>
        <div class="column half">
          <label for="email">Email</label><br />
          <input type="email" name="email" id="email" />
        </div>
      </div>
      <div class="row clearfix">
        <div style="display:none;">
          <label for="hp">HP</label><br />
          <input type="text" name="hp" id="hp" />
        </div>
        <input type="hidden" name="list" value="Q7JrUBzeCeftZqwDtxsQ9w" />
        <div class="column half">
          <input type="submit" name="submit" id="submit" value="Submit" />
        </div>
        <div class="column half">
          <span class="form-rss">Or subscribe to my <a href="/feed.xml">RSS feed</a></span>
        </div>
      </div>
    </form>
    <div class="row clearfix">
    </div>
  </div>
 </div>
--- a/_layouts/default.html
+++ b/_layouts/default.html
@ -1,5 +1,5 @@
 <!DOCTYPE html>
-<html lang="en">
+<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
@ -10,62 +10,54 @@
    <link rel="stylesheet" href="/css/normalize.css">
    <!-- Fix IE -->
    <!--[if lt IE 9]>
-      <script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7/html5shiv.js"></script>
+      <script src="http://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7/html5shiv.js"></script>
-      <script src="https://cdnjs.cloudflare.com/ajax/libs/respond.js/1.4.2/respond.js"></script>
+      <script src="http://cdnjs.cloudflare.com/ajax/libs/respond.js/1.4.2/respond.js"></script>
        <![endif]-->
    <!-- syntax highlighting CSS -->
    <link rel="stylesheet" href="/css/syntax.css">
    <link rel="stylesheet" href="/css/syntax_dark.css" media="(prefers-color-scheme: dark)">
    <!-- Custom CSS -->
    <link rel="stylesheet" href="/css/main.css">
    <link rel="stylesheet" href="/css/main_dark.css" media="(prefers-color-scheme: dark)">
    <!-- Web Fonts -->
-    <link href="https://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,700italic,400,300,700" rel="stylesheet" type="text/css">
+    <link href='http://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,700italic,400,300,700' rel='stylesheet' type='text/css'>
    <!-- RSS Feed -->
-    <link href="/feed.xml" rel="alternate" type="application/atom+xml">
+    <link href='/feed.xml' rel='alternate' type='application/atom+xml'>
    <!-- Favicon -->
-    <link rel="shortcut icon" href="/favicon.png">
+    <link rel="shortcut icon" href="/favicon.png" />
-    <!-- Cloudflare Web Analytics -->
+    <!-- Google Analytics -->
-    <script defer src='https://static.cloudflareinsights.com/beacon.min.js' data-cf-beacon='{"token": "54df1dc81a2d4cb7920b456212bbd437"}'></script>
+    <script>
-    <!-- End Cloudflare Web Analytics -->
+    (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
-    <script type="module" src="https://unpkg.com/dark-mode-toggle"></script>
+    ga('create', 'UA-39880341-1', 'auto');
    ga('send', 'pageview');
-    {% seo %}
+    </script>
    <!-- End Google Analytics -->
  </head>
  <body>
-    <div class="root">
+    <div class="container">
-      <div class="container">
+      <div class="row clearfix header">
-        <div class="row clearfix header">
+        <h1 class="title"><a href="/blog">{{ site.name }}</a></h1>
-          <h1 class="title"><a href="/blog/">{{ site.name }}</a></h1>
+        <a class="extra" href="/">home</a>
          <a class="extra" href="/">home</a>
        </div>
        {{ content }}
        <div class="row clearfix rss">
          <a class="rss" href="/feed.xml"><img src="/img/rss.png" alt="RSS" class="icon" /></a>
        </div>
        <div class="row clearfix footer">
          <div class="column full contact">
            <p class="contact-info">
              <a href="mailto:tyler@hallada.net">tyler@hallada.net</a>
            </p>
          </div>
        </div>
      </div>
-      <div class="theme-toggle">
+      {{ content }}
-        <dark-mode-toggle
+      <div class="row clearfix rss">
-          id="dark-mode-toggle-1"
+        <a class="rss" href="/feed.xml"><img src="/img/rss.png" alt="RSS"/></a>
-          appearance="toggle"
+      </div>
-          light="Dark"
+      <div class="row clearfix footer">
-          dark="Light"
+        <div class="column full contact">
-          permanent
+          <p class="contact-info">
-        ></dark-mode-toggle>
+            <a href="mailto:tyler@hallada.net">tyler@hallada.net</a>
          </p>
        </div>
      </div>
    </div>
  </body>
--- a/_layouts/home.html
+++ b/_layouts/home.html
@ -10,57 +10,49 @@
    <link rel="stylesheet" href="/css/normalize.css">
    <!-- Fix IE -->
    <!--[if lt IE 9]>
-      <script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7/html5shiv.js"></script>
+      <script src="http://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7/html5shiv.js"></script>
-      <script src="https://cdnjs.cloudflare.com/ajax/libs/respond.js/1.4.2/respond.js"></script>
+      <script src="http://cdnjs.cloudflare.com/ajax/libs/respond.js/1.4.2/respond.js"></script>
    <![endif]-->
    <!-- syntax highlighting CSS -->
    <link rel="stylesheet" href="/css/syntax.css">
    <link rel="stylesheet" href="/css/syntax_dark.css" media="(prefers-color-scheme: dark)">
    <!-- Custom CSS -->
    <link rel="stylesheet" href="/css/main.css">
    <link rel="stylesheet" href="/css/main_dark.css" media="(prefers-color-scheme: dark)">
    <!-- RSS Feed -->
-    <link href="/feed.xml" rel="alternate" type="application/atom+xml">
+    <link href='/feed.xml' rel='alternate' type='application/atom+xml'>
    <!-- Web Fonts -->
-    <link href="https://fonts.googleapis.com/css?family=Open+Sans:400,400italics,300,300italics,200" rel="stylesheet" type="text/css">
+    <link href='http://fonts.googleapis.com/css?family=Open+Sans:400,400italics,300,300italics,200' rel='stylesheet' type='text/css'>
    <!-- Icon Fonts -->
    <link rel="stylesheet" href="/css/ionicons.min.css">
    <!-- Favicon -->
-    <link rel="shortcut icon" href="/favicon.png">
+    <link rel="shortcut icon" href="/favicon.png" />
    <!-- Scripts -->
    <script src="/js/AnimationFrame.min.js"></script>
    <script async src="/js/magic.js"></script>
-    <!-- Cloudflare Web Analytics -->
+    <!-- Google Analytics -->
-    <script defer src='https://static.cloudflareinsights.com/beacon.min.js' data-cf-beacon='{"token": "54df1dc81a2d4cb7920b456212bbd437"}'></script>
+    <script>
-    <!-- End Cloudflare Web Analytics -->
+    (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
-    <script type="module" src="https://unpkg.com/dark-mode-toggle"></script>
+    ga('create', 'UA-39880341-1', 'auto');
    ga('send', 'pageview');
-    {% seo %}
+    </script>
    <!-- End Google Analytics -->
  </head>
  <body>
-    <div class="root">
+  <canvas id="magic"></canvas>
-      <canvas id="magic"></canvas>
+    <div class="container">
-      <div class="container">
+      {{ content }}
        {{ content }}
      </div>
      <div class="theme-toggle">
        <dark-mode-toggle
          id="dark-mode-toggle-1"
          appearance="toggle"
          light="Dark"
          dark="Light"
          permanent
        ></dark-mode-toggle>
      </div>
    </div>
  </body>
 </html>
--- a/_layouts/post.html
+++ b/_layouts/post.html
@ -1,20 +1,18 @@
 ---
 layout: default
 ---
 <div class="card">
-  <div class="row clearfix">
+  <div class="row clearfix post-header">
-    <div class="column full post-header">
+    <div class="column three-fourths">
-      <h2 class="post-title"><a href="{{ post.url }}">{{ page.title }}</a></h2>
+      <a href="{{ post.url }}"><h2 class="post-title">{{ page.title }}</h2></a>
-      <span class="timestamp">{{ page.date | date_to_string }}</span>
+    </div>
    <div class="column fourth">
      <span class="timestamp">{{ page.date | date_to_string }}</span> 
    </div>
  </div>
  <div class="row clearfix">
-    <div class="column full post">{{ content }}</div>
+    <div class="column full post">
      {{ content }}
    </div>
  </div>
 </div>
 {% include comments.html %}
 <!-- disabling until I fix the mail form -->
 <!-- {% include mail-form.html %} -->
--- a/_posts/2012-12-03-hello-world.md
+++ b/_posts/2012-12-03-hello-world.md
@ -13,7 +13,6 @@ and was pretty familiar with it and I was beginning to get familiar with
 was what I was working with at [Valti](https://www.valti.com), and I was really
 liking making websites with it. It took what made Python awesome and applied
 that to web development.
 <!--excerpt-->
 I started from a blank Django project and built it up from there. Django's
 Object-Relational Mapper (ORM) can be boiled down to this: python classes
--- a/_posts/2013-04-02-blog-or-not-blog.md
+++ b/_posts/2013-04-02-blog-or-not-blog.md
@ -11,7 +11,6 @@ you can tell, there hasn't been any posts since my first ["Hello,
 World!"](/2012/12/03/hello-world) post. Sure, I've been working on projects, but I
 just haven't gotten to the point in any of those projects where I felt like I
 could blog in detail about it.
 <!--excerpt-->
 Then I watched this great talk that [Brian
 Jones](http://pyvideo.org/speaker/352/brian-k-jones) gave at
@ -19,7 +18,7 @@ Jones](http://pyvideo.org/speaker/352/brian-k-jones) gave at
 pointed out to me:
 <div class="video-container"><iframe width="640" height="360"
-src="https://www.youtube.com/embed/BBfW3m3TK0w?feature=player_embedded"
+src="http://www.youtube.com/embed/BBfW3m3TK0w?feature=player_embedded"
 frameborder="0" allowfullscreen></iframe></div>
 One point that he makes that really resonates with me is how I should write
--- a/_posts/2013-04-09-visualizing-laundry-usage.md
+++ b/_posts/2013-04-09-visualizing-laundry-usage.md
@ -11,7 +11,6 @@ keeps track of the status of every machine and displays it on a
 [website](http://gmu.esuds.net/) so students can check how full the machines
 are before making the trek down to the laundry rooms. The system emails each
 student when their laundry is finished as well.
 <!--excerpt-->
 The only problem is that their user interface is pretty atrocious. I wrote up a
 [usability analysis](https://gist.github.com/thallada/5351114) of the site for
@ -27,7 +26,7 @@ which made it easy for me to dive right in. I'll probably try out
 [d3js](http://d3js.org/) for my next visualization project though, it looks a
 whole lot more advanced.
-### Current laundry usage charts
+###Current laundry usage charts###
 I created an [app](/laundry) in [Django](https://www.djangoproject.com/) to
 display current laundry machine usage charts for all of the laundry rooms on
@ -48,7 +47,7 @@ folder).
 The point was to make this as dead simple and easy to use as possible. Do you
 think I succeeded?
-### Weekly laundry usage chart
+###Weekly laundry usage chart###
 Knowing the *current* laundry machine usage is nice for saving a wasted trip
 down to the laundry room, but what if you wanted to plan ahead and do your
--- a/_posts/2013-04-18-how-download-rtmp-video.md
+++ b/_posts/2013-04-18-how-download-rtmp-video.md
@ -12,7 +12,6 @@ streamed to the user's Flash player (in their browser) bit-by-bit, the full
 video file is never given to the user for them to keep. This is desirable to a
 lot of media companies because then they can force you to watch through ads to
 see their content and can charge you to download the full video.
 <!--excerpt-->
 However, [RTMPDump](http://rtmpdump.mplayerhq.hu/), an open-source tool
 designed to intercept RTMP streams, can download the full video. 
@ -28,7 +27,7 @@ Since this is questionably legal, make sure you understand any Terms of
 Services you accepted or laws in your locality regarding this before you follow
 the steps below ;).
-### Have Linux
+###Have Linux###
 Most of these instructions will assume you have Ubuntu, but
 most distributions will work.
@ -37,18 +36,18 @@ While RTMPDump works on a variety of operating systems, I've only researched
 how to do this on Linux. Feel free to comment if you know how to do this in
 Windows or OSX.
-### Install RTMPDump
+###Install RTMPDump###
 This open source goodness can be found at
 [http://rtmpdump.mplayerhq.hu/](http://rtmpdump.mplayerhq.hu/) or you can just
 intall it using your Linux distro's package manager. For Ubuntu, that would be
 typing the following into your terminal:
-~~~ bash
+```bash
 sudo apt-get install rtmpdump
-~~~
+```
-### Redirect ALL the RTMP!
+###Redirect ALL the RTMP!###
 Now we need to configure your firewall to redirect
 all RTMP traffic to a local port on your computer (Note: this will screw up any
@ -56,11 +55,11 @@ RTMP streaming video you try to watch on your computer, so make sure you run
 the undo command in one of the later steps to return things to normal). Type
 the following into your terminal, there should be no output from the command:
-~~~ bash
+```bash
 sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
-~~~
+```
-### Run rtmpsrv
+###Run rtmpsrv###
 When you install `rtmpdump`, a program called `rtmpsrv`
 should have been bundled with it and installed as well. We will want to run
@ -74,7 +73,7 @@ This should output something that looks like this:
    Streaming on rtmp://0.0.0.0:1935
-### Feed rtmpsrv the Precious Video
+###Feed rtmpsrv the Precious Video###
 Now go to your browser and open/refresh
 the page with the desired video. Try playing the video. If nothing happens and
@ -88,24 +87,24 @@ will need it later.
 You can CTRL+C out of rtmpsrv now that we have what we need.
-### Undo the Redirection
+###Undo the Redirection###
 You must undo the iptables redirection command we
 performed earlier before you can do anything else, so run this in your
 terminal:
-~~~ bash
+```bash
 sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT
-~~~
+```
-### Finally, Download the Precious Video
+###Finally, Download the Precious Video###
 Now paste that command you copied
 from the rtmpsrv output in the step before last into your terminal prompt and
 hit enter. You should now see a torrent of `INFO` printout along with a
 percentage as the video is being downloaded.
-### Feast Eyes on Precious Video
+###Feast Eyes on Precious Video###
 Once downloaded, the video file, which has a
 `flv` extension and was named by the `-o` parameter in the command you copied
--- a/_posts/2013-04-26-hackers.md
+++ b/_posts/2013-04-26-hackers.md
@ -11,7 +11,6 @@ met since the past two internships I've had at [Valti](https:/www.valti.com/)
 and [Humbug](https://humbughq.com/) in Cambridge, Massachusetts. Seeing as it
 encapsulated what I've learned culturally since then, I decided to post it here
 as well.*
 <!--excerpt-->
 Hackers -- not your malicious meddling Hollywood-style speed-typists -- but the
 type who sees a toaster and turns it into a computer capable of etching emails
--- a/_posts/2013-05-19-gmu-bookstore-homepage-concept.md
+++ b/_posts/2013-05-19-gmu-bookstore-homepage-concept.md
@ -12,7 +12,6 @@ to create a homepage for the University's bookstore website, applying all of the
 usability principles we had learned over the semester. I ended up working on it
 when I wanted to procrastinate on assignments in my other classes, so I put
 quite a bit of effort into it.
 <!--excerpt-->
 See it here: [swe205.hallada.net](http://swe205.hallada.net)
 <div style="text-align: center">
--- a/_posts/2013-06-04-w3m-reddit.md
+++ b/_posts/2013-06-04-w3m-reddit.md
@ -10,7 +10,6 @@ includes redditing. I probably spend far too much time on
 way to view reddit through the command-line.  [w3m](http://w3m.sourceforge.net/)
 could render reddit okay, but I couldn't view my personal front-page because
 that required me to login to my profile.
 <!--excerpt-->
 The solution was [cortex](http://cortex.glacicle.org/), a CLI app for viewing
 reddit.
@ -18,19 +17,19 @@ reddit.
 However, I kind of got tired of viewing reddit through w3m, the header alone is
 a few pages long to scroll through, and the CSS for the comments doesn't load so
 there isn't any sense of threading. But, then I discovered reddit's mobile
-website: [http://reddit.com/.mobile](http://reddit.com/.mobile), and it looks absolutely
+website: [http://m.reddit.com](http://m.reddit.com), and it looks absolutely
 beautiful in w3m. In fact, I think I prefer it to the normal website in any
 modern browser; there are no distractions, just pure content.
 <a href="/img/blog/w3m_mobile_reddit.png"><img src="/img/blog/w3m_mobile_reddit.png" alt="m.reddit.com rendered in w3m"></a>
 In order to get cortex to open the mobile version of reddit, I made a bash
-script wrapper around w3m that takes urls and appends `".mobile"` to the end of
+script wrapper around w3m that takes urls and replaces `"http://reddit.com"` and
-reddit urls before passing them to w3m (as well as fixing a double forward slash
+`"http://www.reddit.com"` with `"http://m.reddit.com"` before passing them to
-error in the comment uri cortex outputs that desktop reddit accepts but mobile
+w3m (as well as fixing a double forward slash error in the comment uri cortex
-reddit 404s on). The script:
+outputs that desktop reddit accepts but mobile reddit 404s on). The script:
-~~~ bash
+```bash
 #!/bin/bash
 args=()
@ -46,17 +45,8 @@ done
 args+=("$@")
 for arg in "${args[@]}" ; do
    # Switch to mobile reddit
-    url=$arg
+    url=${arg/http:\/\/reddit.com/http:\/\/m.reddit.com}
-    mobile='.mobile'
+    url=${url/http:\/\/www.reddit.com/http:\/\/m.reddit.com}
    if [[ $url =~ http:\/\/www.reddit.com || $url =~ http:\/\/reddit.com ]]
    then
        if [[ $url =~ \/$ ]]
        then
            url=$url$mobile
        else
            url=$url'/'$mobile
        fi
    fi
    # Fix double backslash error in comment uri for mobile reddit
    url=${url/\/\/comments/\/comments}
    if [[ $t == "1" ]]; then
@ -65,7 +55,7 @@ for arg in "${args[@]}" ; do
        w3m "${url}"
    fi
 done
-~~~
+```
 Since I regurally use [Tmux](http://tmux.sourceforge.net/) (with
 [Byobu](http://byobu.co/)), I also added an optional `-t`/`--tmux` switch that
@ -74,10 +64,10 @@ will open w3m in a temporary new tmux window that will close when w3m is closed.
 I saved the script as `w3m-reddit` and made it an executable command. In Ubuntu
 that's done with the following commands:
-~~~ bash
+```bash
 $ sudo mv w3m-reddit /usr/bin/
 $ sudo chmod +x /usr/bin/w3m-reddit
-~~~
+```
 Now cortex needs to be configured to use `w3m-reddit`, and that's done by
 setting `browser-command` in the cortex config at `~/.cortex/config` to
@ -103,9 +93,3 @@ scrapping the whole thing and starting over in Python instead.
 Stay tuned for more posts on how I view images and videos efficiently from the
 command-line.
 EDIT 04/25/2015: Reddit seems to have gotten rid of their old mobile reddit site
 and replaced it with a more modern version that unfortunately doesn't look as
 good in w3m. However, the old mobile site is still accessable by adding a
 ".mobile" to the end of urls. The script above has been edited to reflect this
 change.
--- a/_posts/2013-07-10-quick-command-line-search-search-pane.md
+++ b/_posts/2013-07-10-quick-command-line-search-search-pane.md
@ -17,7 +17,6 @@ customizability and compatibility with other programs. There's nothing more
 powerful than being able to whip up a small python or bash script that interacts
 with a couple of other programs to achieve something instantly that optimizes my
 work flow.
 <!--excerpt-->
 I use the [Awesome](http://awesome.naquadah.org/) window manager, which works
 great for tiling up terminal windows right up next to browser windows. However,
@ -54,7 +53,7 @@ This is how I got it setup (on any Ubuntu machine with sudo privileges):
 Save the following python file in `/usr/bin/` as `search-pane` (no extension):
-~~~ python
+```python
 #!/usr/bin/python
 from subprocess import call, check_output
 from threading import Thread
@ -107,27 +106,27 @@ except Exception, errtxt:
    print errtxt
 call(['w3m', url]) # pass url off to w3m
-~~~
+```
 Make the directory and file for search history:
-~~~ bash
+```bash
 mkdir ~/.search-pane
 touch ~/.search-pane/history
-~~~
+```
 Allow anyone to execute the python script (make it into a program):
-~~~ bash
+```bash
 chmod a+x /usr/bin/search-pane
-~~~
+```
 To get quick access to the program from the command-line edit `~/.bashrc` to
 add:
-~~~ bash
+```bash
 alias s='search-pane'
-~~~
+```
 To add byobu key bindings edit `~/.byobu/keybindings.tmux` (or `/usr/share/byobu/keybindings/f-keys.tmux`):
--- a/_posts/2013-08-24-on-chromebooks.md
+++ b/_posts/2013-08-24-on-chromebooks.md
@ -9,7 +9,6 @@ just glorified browsers, right? What if I wanted to do anything outside of the
 browser?  Why would you spend [$1299 or $1449 for a
 computer](https://www.google.com/intl/en/chrome/devices/chromebooks.html#pixel)
 that can only run a browser?
 <!--excerpt-->
 While I know a lot of people who buy expensive MacBooks only to just use a web
 browser and iTunes, I’m a bit more of a power user and I need things like
@ -83,7 +82,7 @@ of tweaking. If anyone has read my past posts, they know that I am obsessed
 with configuring things. Here is what I came up with for everything I would
 ever need to do on my Chromebook:
-### Writing
+###Writing###
 I spent a lot of time downloading
 [various](https://chrome.google.com/webstore/detail/write-space/aimodnlfiikjjnmdchihablmkdeobhad)
@ -116,7 +115,7 @@ hassle though, so I often just stick to the default style. It’s a sign that I
 am procrastinating if I’m trying to look for the “perfect template” to write in
 anyways.
-### Programming
+###Programming###
 I’ve gotten so used to [vim](http://www.vim.org/) in a Linux
 terminal that I don’t think I could ever use any other editor. There are a few
@ -151,7 +150,7 @@ have all of the great chrome apps and extensions right at my fingertips.
 Especially when some apps can be opened up in small panels in the corner of the
 screen temporarily.
-### Panels
+###Panels###
 Chrome recently released a new concept for opening new windows
 called “Panels”, and once I discovered them I couldn’t get enough of them. The
@ -188,7 +187,7 @@ Panel](https://chrome.google.com/webstore/detail/improved-google-tasks-pan/kgnap
 I’m still lacking Facebook Messenger and Google Voice panel view apps, so I
 might try my hand at creating one myself soon.
-### Web Browsing
+###Web Browsing###
 And, of course, being a laptop dedicated to chrome, it
 obviously has a great web browsing experience. 
--- a/_posts/2013-10-03-publishing-draft-docs-to-my-blog.md
+++ b/_posts/2013-10-03-publishing-draft-docs-to-my-blog.md
@ -13,7 +13,6 @@ features, one of its best being a version control system that allows you to
 send a draft to other people and accept or reject any changes they suggest. It
 also has a minamilistic iA Writer type interface, which focuses on the actual
 writing and nothing more.
 <!--excerpt-->
 One of my most favorite features that I have just discovered, though, is that
 it allows publishing any Draft document to any arbitrary
--- a/_posts/2014-07-30-new-website.md
+++ b/_posts/2014-07-30-new-website.md
@ -9,7 +9,6 @@ development knowledge had exceeded what it was showing off. The main thing that
 annoyed me about my last website was that I was hosting what essentially was a
 static website on a web framework meant for dynamic websites. It was time for a
 update.
 <!--excerpt-->
 I decided to go with [Jekyll](http://jekyllrb.com/) which had everything I
 wanted:
@ -48,5 +47,5 @@ under 20% (on my machine), which was actually better than a few chrome
 extensions I was running anyways.
 Hopefully this new blog will also inspire me to write more posts as [my last
-post](/2013/10/03/publishing-draft-docs-to-my-blog.html)
+post](http://thallada.github.io/2013/10/03/publishing-draft-docs-to-my-blog.html)
 was almost a year ago now.
--- a/_posts/2015-06-03-midnight-desktop.md
+++ b/_posts/2015-06-03-midnight-desktop.md
@ -1,268 +0,0 @@
 ---
 title: Midnight Desktop
 layout: post
 ---
 I tend to use Linux (Ubuntu) on my desktop late at night in a dark room. To
 protect my eyes from the blinding light of my monitors I've tooled my desktop
 environment over the course of a few months to be as dark as possible. It has
 gotten complex enough that I thought it would be worth sharing now.
 <!--excerpt-->
 ### dotfiles
 Before I begin, I want to note that all the configuration for the setup I'm
 describing is stored in a [dotfiles repo on my github
 profile](https://github.com/thallada/dotfiles). If you would like to replicate
 any of this setup, I would go there. Just note that I will probably be updating
 the master branch fairly often, but the
 [midnight](https://github.com/thallada/dotfiles/tree/midnight) branch will
 always contain the setup described here.
 ### bspwm
 Inspired by [/r/unixporn](http://www.reddit.com/r/unixporn), I decided to switch
 from gnome to bspwm, a minimal tiling window manager that positions windows like
 leaves on a binary tree.
 I don't really use the tiling features that often, though. I often do most of my
 work in the terminal and [tmux](http://tmux.sourceforge.net/) does the terminal
 pane management. But, when I do open another application, it's nice that bspwm
 forces it to use the maximum available space.
 I also like how hackable the whole manager is. There is a terminal command
 `bspc` that controls the entire desktop environment and a separate program
 `sxhkd` (probably the hardest program name ever to remember) handles all of the
 hotkeys for the environment. All of them are stored in a
 [`sxhkdrc`](https://github.com/thallada/dotfiles/blob/master/sxhkd/.config/sxhkd/sxhkdrc)
 under the home directory and it's super easy to add my own. The hotkeys make
 this superior to gnome for me because I never have to touch my mouse to move
 around the desktop.
 ### gnome and gtk
 I still love some of the features from gnome. Especially the text hinting, which
 is why I still run `gnome-settings-daemon` in my [bspwm startup
 script](https://github.com/thallada/dotfiles/blob/master/bspwm/bin/bspwm-session).
 To make gtk applications universally dark (and also to tune text hinting)
 install the `gnome-tweak-tool`. There should be a "Global Dark Theme" setting
 under the "Appearance" tab that can be enabled. I use the
 [Numix](https://numixproject.org/) gtk theme which seems to behave fine with
 this setting.
 ### Gnome Terminal
 I've tried using a few other lighter-weight terminals like xterm, but I still
 like the features of gnome-terminal more. I created a "bspwm" profile and set
 the background to be transparent with opacity at about half-way on the slider.
 My background, set in the [bspwm startup
 script](https://github.com/thallada/dotfiles/blob/master/bspwm/bin/bspwm-session)
 is a subtle [dark tiling pattern](https://www.toptal.com/designers/subtlepatterns/mosaic/)
 so this effectively makes the background of the terminal dark.
 In my
 [sxhkdrc](https://github.com/thallada/dotfiles/blob/master/sxhkd/.config/sxhkd/sxhkdrc)
 I can then map my hotkeys for starting a new terminal to the command
 `gnome-terminal --window-with-profile=bspwm`.
 ### vim
 Making vim dark is pretty easy. Just put this in the
 [`.vimrc`](https://github.com/thallada/dotfiles/blob/master/vim/.vimrc):
 ~~~ vim
 set background=dark
 ~~~
 I use the colorscheme
 [distinguished](https://github.com/Lokaltog/vim-distinguished) which is
 installed by putting the `distinguished.vim` file under
 [`~/.vim/colors/`](https://github.com/thallada/dotfiles/tree/master/vim/.vim/colors)
 and adding this to the `.vimrc`:
 ~~~ vim
 colorscheme distinguished
 ~~~
 ### tmux/byobu
 I like the abstraction that [byobu](http://byobu.co/) puts ontop of tmux, so
 that's what I use in the terminal. Colors can be configured by editing the
 [`~/.byobu/color.tmux`](https://github.com/thallada/dotfiles/blob/master/byobu/.byobu/color.tmux)
 file. This is what I have in mine:
    BYOBU_DARK="\#333333"
    BYOBU_LIGHT="\#EEEEEE"
    BYOBU_ACCENT="\#4D2100"
    BYOBU_HIGHLIGHT="\#303030"
    MONOCHROME=0
 ### evince
 I tell my browser, firefox, to open pdfs in evince (aka. Document Viewer)
 because evince can darken pdfs.
 Select View > Invert Colors and then Edit > Save Current Settings as Default and
 now most pdfs will be displayed as white text on black background.
 ### gimp
 Gimp allows you to change themes easily. [Gimp GTK2 Photoshop CS6
 Theme](http://gnome-look.org/content/show.php?content=160952) is my favorite
 dark theme. Put that in `~/.gimp-2.8/themes/` (or whichever gimp version is
 installed) and, in Gimp, change the theme at Edit > Preferences > Theme.
 ### Firefox
 I had to hack firefox a lot to get it to be universally dark since the web
 (unfortunately!) doesn't have a night mode switch. I'm using firefox instead of
 chrome because firefox has better customization for doing something this
 extreme.
 #### Userstyles
 Firefox has a really neat addon called
 [Stylish](https://addons.mozilla.org/en-us/firefox/addon/stylish/) that allows
 you to install and edit user CSS files to change the style of any website you
 visit. A lot of popular websites have dark themes on
 [userstyles.org](https://userstyles.org/), but the rest of the internet still
 mostly has a white background by default.
 Luckily there's a few global dark themes. [Midnight Surfing
 Alternative](https://userstyles.org/styles/47391/midnight-surfing-alternative)
 seemed to work the best for me.
 However, since the theme is global, it overwrites the custom tailored dark
 themes that I had installed for specific popular sites (listed below) making the
 sites ugly. The Midnight Surfing Alternative theme can be edited through the
 Stylish extension to exclude the websites that I already have dark themes for.
 [This superuser question explains what to
 edit](http://superuser.com/questions/463153/disable-stylish-on-certain-sites-in-firefox).
 Now, whenever I add a new dark theme to Stylish, I edit the regex to add the
 domains it covers to the parenthesized list that is delimited by pipes.
 ~~~ css
@-moz-document regexp("(https?|liberator|file)://(?!([^.]+\\.)?(maps\\.google\\.com|...other domains....)[/:]).*"){
 ~~~
 Here is the list of dark themes I'm currently using with Stylish in addition to
 Midnight Surfing Alternative:
 * [Amazon Dark -
    VisualPlastik](https://userstyles.org/styles/52294/amazon-dark-visualplastik)
 * [Dark Feedly
    (Hauschild's)](https://userstyles.org/styles/89622/dark-feedly-hauschild-s)
 * [Dark Gmail mod by
    Karsonito](https://userstyles.org/styles/107544/dark-gmail-mod-by-karsonito)
    (this one is a bit buggy right now, though)
 * [Dark Netflix
    [GRiMiNTENT]](https://userstyles.org/styles/102627/dark-netflix-grimintent)
 * [dark-facebook 2 [a dark facebook
    theme]](https://userstyles.org/styles/95359/facebook-dark-facebook-2-a-dark-facebook-theme)
 * [Forecast.io - hide
    map](https://userstyles.org/styles/104812/forecast-io-hide-map)
 * [GitHub Dark](https://userstyles.org/styles/37035/github-dark) (this one is
    really well done, I love it)
 * [Google Play (Music) Dark \*Updated
    5-15\*](https://userstyles.org/styles/107643/google-play-music-dark-updated-5-15)
 * [Messenger.com Dark](https://userstyles.org/styles/112722/messenger-com-dark)
 * [Telegram web dark / custom
    color](https://userstyles.org/styles/109612/telegram-web-dark-custom-color)
 * [Youtube - Lights Out - A Dark Youtube
    Theme](https://userstyles.org/styles/92164/youtube-lights-out-a-dark-youtube-theme)
 #### UI Themes
 Most of my firefox UI is styled dark with the [FT
 DeepDark](https://addons.mozilla.org/en-US/firefox/addon/ft-deepdark/) theme.
 The firefox developer tools can be [themed dark in its
 settings](http://soledadpenades.com/2014/11/20/using-the-firefox-developer-edition-dark-theme-with-nightly/).
 #### Addons
 For reddit, I use the [RES](http://redditenhancementsuite.com/) addon which has
 a night mode option.
 I also use [Custom New
 Tab](https://addons.mozilla.org/en-US/firefox/addon/custom-new-tab/) combined
 with [homepage.py](https://github.com/ok100/homepage.py) to display a list of my
 favorite websites when I start a new tab.
 [Vimperator](https://addons.mozilla.org/en-US/firefox/addon/vimperator/) allows
 me to control firefox completely with my keyboard which is really useful when I
 am switching back and forth between firefox and vim. By default, the vimperator
 window has a white background, so I had to [set it to a dark
 theme](https://github.com/vimpr/vimperator-colors). Also, in order to make all
 of the vimperator help pages dark, I had to add the protocol `liberator://` to
 the regex for Midnight Surfing Alternative (exact syntax for that above).
 ### Redshift
 At night, it's also useful to filter out blue light to help with sleep.
 [Redshift](http://jonls.dk/redshift/) is a utility that does this automatically
 while running in the background.
 ![Midnight in action with redshift](/assets/midnight_screenshot_redshift.png)
 ### Invert it all!
 I noticed that with the dark colors and my monitor brightness turned low, it was
 hard to see the screen during the day because of glares. An easy solution to
 this is to simply invert the colors on the output of the monitor into an instant
 day theme.
 I would have used the command `xcalib -a -i` but that would only work on one
 monitor and I have two. Luckily, someone made a utility that would invert colors
 on more than one monitor called
 [xrandr-invert-colors](https://github.com/zoltanp/xrandr-invert-colors).
 The only problem was that this utility seemed to interfere with redshift, so I
 made [a script that would disable redshift before
 inverting](https://github.com/thallada/dotfiles/blob/master/invert/bin/invert).
 ~~~ bash
 #!/bin/bash
 inverted=$(xcalib -a -p | head -c 1)
 if [ "$inverted" == "W" ]
  then
    if [ -z "$(pgrep redshift)" ]
      then
        xrandr-invert-colors
        redshift &
    fi
  else
    if [ -z "$(pgrep redshift)" ]
      then
        xrandr-invert-colors
      else
        killall redshift
        sleep 3
        xrandr-invert-colors
    fi
 fi
 ~~~
 And, now I have [a
 shortcut](https://github.com/thallada/dotfiles/commit/e5153a90fa7c89a0e2ca16e5943f0fa20d4a9512)
 to invert the screen.
 However, images and videos look pretty ugly inverted. VLC has a setting under
 Tools > Effects and Filters > Video Effects > Colors called Negate colors that
 can fix that.
 For firefox, I made a global userstyle to invert images and videos.
 ~~~ css
@-moz-document regexp("(https?|liberator|file)://(?!([^.]+\\.)?[/:]).*"){
  img, video, div.html5-video-container, div.player-api, span.emoji, i.emoji, span.emoticon, object[type="application/x-shockwave-flash"], embed[type="application/x-shockwave-flash"] {
      filter: invert(100%);
    }
 }
 ~~~
 Whenever I invert the colors, I enable that global theme on firefox.
 ![Midnight inverted into a day theme](/assets/midnight_screenshot_inverted.png)
--- a/_posts/2016-01-06-neural-style.md
+++ b/_posts/2016-01-06-neural-style.md
@ -1,276 +0,0 @@
 ---
 title: Generating Realistic Satellite Imagery with Deep Neural Networks
 layout: post
 ---
 I've been doing a lot of experimenting with [neural-style](https://github.com/jcjohnson/neural-style)
 the last month. I think I've discovered a few exciting applications of the
 technique that I haven't seen anyone else do yet. The true power of this
 algorithm really shines when you can see concrete examples.
 <!--excerpt-->
 Skip to the **Applications** part of this post to see the outputs from my
 experimentation if you are already familiar with DeepDream, Deep Style, and all
 the other latest happenings in generating images with deep neural networks.
 ### Background and History
 On [May 18, 2015 at 2 a.m., Alexander
 Mordvintsev](https://medium.com/backchannel/inside-deep-dreams-how-google-made-its-computers-go-crazy-83b9d24e66df#.g4t69y8wy),
 an engineer at Google, did something with deep neural networks that no one had
 done before. He took a net designed for *recognizing* objects in images and used
 it to *generate* objects in images. In a sense, he was telling these systems
 that mimic the human visual cortex to hallucinate things that weren't really
 there. The [results](https://i.imgur.com/6ocuQsZ.jpg) looked remarkably like LSD
 trips or what a [schizophrenic person sees on a blank
 wall](https://www.reddit.com/r/deepdream/comments/3cewgn/an_artist_suffering_from_schizophrenia_was_told/).
 Mordvintsev's discovery quickly gathered attention at Google once he posted
 images from his experimentation on the company's internal network. On June 17,
 2015, [Google posted a blog post about the
 technique](http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html)
 (dubbed "Inceptionism") and how it was useful for opening up the notoriously
 black-boxed neural networks using visualizations that researchers could examine.
 These machine hallucinations were key for identifying the features of objects
 that neural networks used to tell one object from another (like a dog from a
 cat). But the post also revealed the [beautiful
 results](https://goo.gl/photos/fFcivHZ2CDhqCkZdA) of applying the algorithm
 iteratively on it's own outputs and zooming out at each step.
 The internet exploded in response to this post. And once [Google posted the code
 for performing the
 technique](http://googleresearch.blogspot.com/2015/07/deepdream-code-example-for-visualizing.html?m=1),
 people began experimenting and sharing [their fantastic and creepy
 images](https://www.reddit.com/r/deepdream) with the world.
 Then, on August, 26, 2015, a paper titled ["A Neural Algorithm of Artistic
 Style"](http://arxiv.org/abs/1508.06576) was published. It showed how one could
 identify which layers of deep neural networks recognized stylistic information
 of an image (and not the content) and then use this stylistic information in
 Google's Inceptionism technique to paint other images in the style of any
 artist. A [few](https://github.com/jcjohnson/neural-style)
 [implementations](https://github.com/kaishengtai/neuralart) of the paper were
 put up on Github. This exploded the internet again in a frenzy. This time, the
 images produced were less like psychedelic-induced nightmares but more like the
 next generation of Instagram filters ([reddit
 how-to](https://www.reddit.com/r/deepdream/comments/3jwl76/how_anyone_can_create_deep_style_images/)). 
 People began to wonder [what all of this
 meant](http://www.hopesandfears.com/hopes/culture/is-this-art/215039-deep-dream-google-art)
 to [the future of
 art](http://kajsotala.fi/2015/07/deepdream-today-psychedelic-images-tomorrow-unemployed-artists/).
 Some of the results produced where [indistinguishable from the style of dead
 artists'
 works](https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_starry.png).
 Was this a demonstration of creativity in computers or just a neat trick?
 On November, 19, 2015, [another paper](http://arxiv.org/abs/1511.06434) was
 released that demonstrated a technique for generating scenes from convolutional
 neural nets ([implementation on Github](https://github.com/Newmu/dcgan_code)).
 The program could generate random (and very realistic) [bedroom
 images](https://github.com/Newmu/dcgan_code/raw/master/images/lsun_bedrooms_five_epoch_samples.png)
 from a neural net trained on bedroom images. Amazingly, it could also generate
 [the same bedroom from any
 angle](https://github.com/Newmu/dcgan_code/blob/master/images/lsun_bedrooms_five_epochs_interps.png).
 It could also [produce images of the same procedurally generated face from any
 angle](https://github.com/Newmu/dcgan_code/blob/master/images/turn_vector.png).
 Theoretically, we could use this technology to create *procedurally generated
 game art*.
 The main thing holding this technology back from revolutionizing procedurally
 generated video games is that it is not real-time. Using
 [neural-style](https://github.com/jcjohnson/neural-style) to apply artistic
 style to a 512 by 512 pixel content image could take minutes even on the
 top-of-the-line GTX Titan X graphics card. Still, I believe this technology has
 a lot of potential for generating game art even if it can't act as a real-time
 filter.
 ### Applications: Generating Satellite Images for Procedural World Maps
 I personally know very little machine learning, but I have been able to produce
 a lot of interesting results by using the tool provided by
 [neural-style](https://github.com/jcjohnson/neural-style).
 Inspired by [Kaelan's procedurally generated world
 maps](http://blog.kaelan.org/randomly-generated-world-map/), I wanted to extend
 the idea by generating realistic satellite images of the terrain maps. The
 procedure is simple: take a [generated terrain map](/assets/kaelan_terrain1.png)
 and apply the style of a [real-world satellite image](/assets/uk_satellite.jpg)
 on it using neural-style.
 ![Output of generated map plus real-world satellite
 imagery](/assets/satellite_terrain1_process.png)
 The generated output takes on whatever terrain is in the satellite image. Here
 is an output processing one of Kaelan's maps with a [arctic satellite
 image](/assets/svalbard_satellite.jpg):
 ![Kaelan's terrain map](/assets/kaelan_terrain2.jpg)
 ![Output of terrain map plus arctic satellite imagery](/assets/satellite_terrain2.png)
 And again, with one of Kaelan's desert maps and a [satellite image of a
 desert](/assets/desert_satellite.jpg):
 ![Kaelan's desert terrain map](/assets/kaelan_terrain3.jpg)
 ![Output of terrain map plus desert satellite imagery](/assets/satellite_terrain3.png)
 It even works with [Kaelan's generated hexagon
 maps](http://blog.kaelan.org/hexagon-world-map-generation/). Here's an island
 hexagon map plus a [satellite image of a volcanic
 island](/assets/volcano_satellite.jpg):
 ![Kaelan's island hexagon map](/assets/kaelan_hex_terrain.jpg)
 ![Output of hexagon map plus island satellite
 imagery](/assets/satellite_hex_terrain.png)
 This image even produced an interesting three-dimensional effect because of the
 volcano in the satellite image.
 By the way, this also works with minecraft maps. Here's a minecraft map I found
 on the internet plus a [satellite image from Google
 Earth](/assets/river_satellite.png):
 ![Minecraft map](/assets/minecraft_map.jpg)
 ![Output of minecraft map plus river satellite
 imagery](/assets/satellite_minecraft_map.png)
 No fancy texture packs or 3-D rendering needed :).
 Here is the Fallout 4 grayscale map plus a
 [satellite image of Boston](/assets/boston_aerial.jpg):
 ![Fallout 4 grayscale map](/assets/fallout4_map.png)
 ![Output of Fallout 4 map plus Boston satellite
 imagery](/assets/satellite_fallout4_map.png)
 Unfortunately, it puts the built-up dense part of the city in the wrong part of
 the geographic area. But, this is understandable since we gave the algorithm no
 information on where that is on the map.
 We can also make the generated terrain maps look like old hand-drawn maps using
 neural-style. With Kaelan's terrain map as the
 content and [the in-game Elder Scrolls IV Oblivion map of
 Cyrodiil](/assets/cyrodiil_ingame.jpg) as the style we get this:
 ![Kaelan's terrain map](/assets/kaelan_terrain1.png)
 ![Output of terrain map plus map of Cyrodiil](/assets/cyrodiil_terrain1.png)
 It looks cool, but the water isn't conveyed very clearly (e.g. makes deep water
 look like land). Neural-style seems to work better when there is lots of color
 in both images.
 Here is the output of the hex terrain plus satellite map above and the Cyrodiil
 map which looks a little cleaner:
 ![Satellite-like hex terrain map](/assets/satellite_hex_terrain.png)
 ![Output of hex terrain plus satellite and map of
 Cyrodiil](/assets/cyrodiil_satellite_hex_terrain.png)
 I was interested to see what neural-style could generate from random noise, so I
 rendered some clouds in GIMP and ran it with a satellite image of [Mexico City
 from Google Earth](/assets/mexico_city.jpg) (by the way, I've been getting high
 quality Google Earth shots from
 [earthview.withgoogle.com](https://earthview.withgoogle.com)).
 ![Random clouds](/assets/blurry_clouds.png)
 ![Output of random clouds and Mexico City](/assets/random_mexico_city.png)
 Not bad for a neural net without a degree in urban planning.
 I also tried generating on random noise with a satellite image of [a water
 treatment plant in Peru](/assets/treatment_plant.jpg)
 ![Random clouds](/assets/blurry_clouds2.png)
 ![Output of random clouds and water treatment
 plant](/assets/random_treatment_plant.png)
 ### Applications: More Fun
 For fun, here are some other outputs that I liked.
 [My photo of Boston's skyline as the content](/assets/boston_skyline.jpg) and
 [Vincent van Gogh's The Starry Night as the style](/assets/starry_night.jpg):
 ![Output of Boston skyline and starry night](/assets/starry_boston.png)
 [A photo of me](/assets/standing_forest.jpg) (by Aidan Bevacqua) and [Forrest in
 the end of Autumn by Caspar David Friedrich](/assets/forrest_autumn.jpg):
 ![Output of me and Forrest in the end of
 Autumn](/assets/dead_forest_standing.png)
 [Another photo of me by Aidan](/assets/sitting_forest.jpg) in the same style:
 ![Output of me and Forrest in the end of Autumn](/assets/dead_forest_sitting.png)
 [A photo of me on a mountain](/assets/mountain_view.jpg) (by Aidan Bevacqua) and
 [pixel art by Paul Robertson](/assets/pixels.png)
 ![Output of me on a mountain and pixel art](/assets/mountain_view_pixels.png)
 [A photo of a park in Copenhagen I took](/assets/copenhagen_park.jpg) and a
 painting similar in composition, [Avenue of Poplars at Sunset by Vincent van
 Gogh](/assets/avenue_poplars.jpg):
 ![Output of park in Copenhagen and Avenue of Poplars at
 Sunset](/assets/poplars.png)
 [My photo of the Shenandoah National Park](/assets/shenandoah_mountains.jpg) and
 [this halo graphic from GMUNK](/assets/halo_ring_mountains.jpg)
 ([GMUNK](http://www.gmunk.com/filter/Interactive/ORA-Summoners-HALO)):
 ![Output of Shenandoah mountains and halo ring
 mountains](/assets/halo_shenandoah.png)
 [A photo of me by Aidan](/assets/me.png) and a [stained glass
 fractal](/assets/stained_glass.jpg):
 ![Output of me and a stained glass fractal](/assets/stained_glass_portrait.png)
 Same photo of me and some [psychedelic art by GMUNK](/assets/pockets.jpg)
 ![Output of me and psychedelic art](/assets/pockets_portrait.png)
 [New York City](/assets/nyc.jpg) and [a rainforest](/assets/rainforest.jpg):
 ![Output of New York City and a rainforest](/assets/jungle_nyc.png)
 [Kowloon Walled City](/assets/kowloon.jpg) and [a National Geographic
 Map](/assets/ngs_map.jpg):
 ![Output of Kowloon and NGS map](/assets/kowloon_ngs.png)
 [A photo of me by Aidan](/assets/side_portrait.jpg) and [Head of Lioness by
 Theodore Gericault](/assets/head_lioness.jpg):
 ![Output of photo of me and ](/assets/lion_portrait.png)
 [Photo I took of a Norwegian forest](/assets/forest_hill.jpg) and [The Mountain
 Brook by Albert Bierstadt](/assets/mountain_brook.jpg):
 ![Output of Norwegian forest and The Mountain
 Brook](/assets/mountain_brook_hill.png)
 ### Limitations
 I don't have infinite money for a GTX Titan X, so I'm stuck with using OpenCL on
 my more-than-a-few-generations-old AMD card. It takes about a half-hour to
 generate one 512x512 px image in my set-up (which makes the feedback loop for
 correcting mistakes *very* long). And sometimes the neural-style refuses to run
 on my GPU (I suspect it runs out of VRAM), so I have to run it on my CPU which
 takes even longer...
 I am unable to generate bigger images (though
 [the author has been able to generate up to 1920x1010
 px](https://github.com/jcjohnson/neural-style/issues/36#issuecomment-142994812)).
 As the size of the output increases the amount of memory and time to generate
 also increases. And, it's not practical to just generate thumbnails to test
 parameters, because increasing the image size will probably generate a very
 different image since all the other parameters stay the same even though they
 are dependent on the image size.
 Some people have had success running these neural nets on GPU spot instances in
 AWS. It would be certainly cheaper than buying a new GPU in the short-term.
 So, I have a few more ideas for what to run, but it will take me quite a while
 to get through the queue.
--- a/_posts/2017-06-20-how-to-install-tensorflow-on-ubuntu-16-04.md
+++ b/_posts/2017-06-20-how-to-install-tensorflow-on-ubuntu-16-04.md
@ -1,197 +0,0 @@
 ---
 title: How to Install TensorFlow on Ubuntu 16.04 with GPU Support
 layout: post
 ---
 I found the [tensorflow
 documentation](https://www.tensorflow.org/install/install_linux) rather lacking
 for installation instructions, especially in regards to getting GPU support.
 I'm going to write down my notes from wrangling with the installation here for
 future reference and hopefully this helps someone else too.
 <!--excerpt-->
 This will invariably go out-of-date at some point, so be mindful of the publish
 date of this post. Make sure to cross-reference other documentation that has
 more up-to-date information.
 ## Assumptions
 These instructions are very specific to my environment, so this is what I am
 assuming:
 1. You are running Ubuntu 16.04. (I have 16.04.1)
    - You can check this in the output of `uname -a`
 2. You have a 64 bit machine.
    - You can check this with `uname -m`. (should say `x86_64`)
 2. You have an NVIDIA GPU that has CUDA Compute Capability 3.0 or higher.
 [NVIDIA documentation](https://developer.nvidia.com/cuda-gpus) has a full table
 of cards and their Compute Capabilities.  (I have a GeForce GTX 980 Ti)
    - You can check what card you have in Settings > Details under the label
      "Graphics"
    - You can also check by verifying there is any output when you run `lspci |
      grep -i nvidia`
 3. You have a linux kernel version 4.4.0 or higher. (I have 4.8.0)
    - You can check this by running `uname -r`
 4. You have gcc version 5.3.1 or higher installed. (I have 5.4.0)
    - You can check this by running `gcc --version`
 5. You have the latest [proprietary](https://i.imgur.com/8osspXj.jpg) NVIDIA
 drivers installed.
    - You can check this and install it if you haven't in the "Additional
      Drivers" tab in the "Software & Updates" application (`update-manager`).
      (I have version 375.66 installed)
 6. You have the kernel headers installed.
    - Just run `sudo apt-get install linux-headers-$(uname -r)` to install them
      if you don't have them installed already.
 7. You have Python installed. The exact version shouldn't matter, but for the
 rest of this post I'm going to assume you have `python3` installed.
    - You can install `python3` by running `sudo apt-get install python3`. This
      will install Python 3.5.
    - Bonus points: you can install Python 3.6 by following [this
      answer](https://askubuntu.com/a/865569), but Python 3.5 should be fine.
 ## Install the CUDA Toolkit 8.0
 NVIDIA has [a big scary documentation
 page](http://docs.nvidia.com/cuda/cuda-installation-guide-linux/) on this, but I
 will summarize the only the parts you need to know here.
 Go to the [CUDA Toolkit Download](https://developer.nvidia.com/cuda-downloads)
 page. Click Linux > x86_64 > Ubuntu > 16.04 > deb (network).
 Click download and then follow the instructions, copied here:
 1. `sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb`
 2. `sudo apt-get update`
 3. `sudo apt-get install cuda`
 This will install CUDA 8.0. It installed it to the directory
 `/usr/local/cuda-8.0/` on my machine.
 There are some [post-install
 actions](http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions)
 we must follow:
 1. Edit your `~/.bashrc`
    - Use your favorite editor `gedit ~/.bashrc`, `nano ~/.bashrc`, `vim
      ~/.bashrc`, whatever.
 2. Add the following lines to the end of the file:
 ```bash
 # CUDA 8.0 (nvidia) paths
 export CUDA_HOME=/usr/local/cuda-8.0
 export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
 export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
 ```
 3. Save and exit.
 4. Run `source ~/.bashrc`.
 5. Install writable samples by running the script `cuda-install-samples-8.0.sh
 ~/`.
   - If the script cannot be found, the above steps didn't work :(
   - I don't actually know if the samples are absolutely required for what I'm
     using CUDA for, but it's recommended according to NVIDIA, and compiling
     them will output a nifty `deviceQuery` binary which can be ran to test if
     everything is working properly.
 6. Make sure `nvcc -V` outputs something.
   - If an error, the above steps 1-4 didn't work :(
 7. `cd ~/NVIDIA_CUDA-8.0_Samples`, cross your fingers, and run `make`
   - The compile will take a while
   - My compile actually errored near the end with an error about `/usr/bin/ld:
     cannot find -lnvcuvid` I *think* that doesn't really matter because the
 binary files were still output.
 8. Try running `~/NVIDIA_CUDA-8.0_Samples/bin/x86_64/linux/release/deviceQuery`
 to see if you get any output. Hopefully you will see your GPU listed.
 ## Install cuDNN v5.1
 [This AskUbuntu answer](https://askubuntu.com/a/767270) has good instructions.
 Here are the instructions specific to this set-up:
 1. Visit the [NVIDIA cuDNN page](https://developer.nvidia.com/cudnn) and click
 "Download".
 2. Join the program and fill out the survey.
 3. Agree to the terms of service.
 4. Click the link for "Download cuDNN v5.1 (Jan 20, 2017), for CUDA 8.0"
 5. Download the "cuDNN v5.1 Library for Linux" (3rd link from the top).
 6. Untar the downloaded file. E.g.:
 ```bash
 cd ~/Downloads
 tar -xvf cudnn-8.0-linux-x64-v5.1.tgz
 ```
 7. Install the cuDNN files to the CUDA folder:
 ```bash
 cd cuda
 sudo cp -P include/* /usr/local/cuda-8.0/include/
 sudo cp -P lib64/* /usr/local/cuda-8.0/lib64/
 sudo chmod a+r /usr/local/cuda-8.0/lib64/libcudnn*
 ```
 ## Install libcupti-dev
 This one is simple. Just run:
 ```bash
 sudo apt-get install libcupti-dev
 ```
 ## Create a Virtualenv
 I recommend using
 [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/index.html)
 to create the tensorflow virtualenv, but the TensorFlow docs still have
 [instructions to create the virtualenv
 manually](https://www.tensorflow.org/install/install_linux#InstallingVirtualenv).
 1. [Install
 virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/install.html).
 Make sure to add [the required
 lines](https://virtualenvwrapper.readthedocs.io/en/latest/install.html#shell-startup-file)
 to your `~/.bashrc`.
 2. Create the virtualenv:
 ```bash
 mkvirtualenv --python=python3 tensorflow
 ```
 ## Install the TensorFlow with GPU support
 If you just run `pip install tensorflow` you will not get GPU support. To
 install the correct version you will have to install from a [particular
 url](https://www.tensorflow.org/install/install_linux#python_35). Here is the
 install command you will have to run to install TensorFlow 1.2 for Python 3.5
 with GPU support:
 ```bash
 pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.2.0-cp35-cp35m-linux_x86_64.whl
 ```
 If you need a different version of TensorFlow, you can edit the version number
 in the URL. Same with the Python version (change `cp35` to `cp36` to install for
 Python 3.6 instead, for example).
 ## Test that the installation worked
 Save this script from [the TensorFlow
 tutorials](https://www.tensorflow.org/tutorials/using_gpu#logging_device_placement)
 to a file called `test_gpu.py`:
 ```python
 # Creates a graph.
 with tf.device('/cpu:0'):
  a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
  b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
 c = tf.matmul(a, b)
 # Creates a session with log_device_placement set to True.
 sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
 # Runs the op.
 print(sess.run(c))
 ```
 And then run it:
 ```bash
 python test_gpu.py
 ```
 You should see your GPU card listed under "Device mapping:" and that each task
 in the compute graph is assigned to `gpu:0`.
 If you see "Device mapping: no known devices" then something went wrong and
 TensorFlow cannot access your GPU.
--- a/_posts/2017-07-11-generating-random-poems-with-python.md
+++ b/_posts/2017-07-11-generating-random-poems-with-python.md
@ -1,516 +0,0 @@
 ---
 title: Generating Random Poems with Python
 layout: post
 image: /img/blog/buzzfeed.jpg
 ---
 In this post, I will demonstrate how to generate random text using a few lines
 of standard python and then progressively refine the output until it looks
 poem-like.
 If you would like to follow along with this post and run the code snippets
 yourself, you can clone [my NLP repository](https://github.com/thallada/nlp/)
 and run [the Jupyter
 notebook](https://github.com/thallada/nlp/blob/master/edX%20Lightning%20Talk.ipynb).
 You might not realize it, but you probably use an app everyday that can generate
 random text that sounds like you: your phone keyboard.
 <!--excerpt-->
 ![Suggested next words UI feature on the iOS
 keyboard](/img/blog/phone_keyboard.jpg)
 Just by tapping the next suggested word over and over, you can generate text. So how does it work?
 ## Corpus
 First, we need a **corpus**: the text our generator will recombine into new
 sentences. In the case of your phone keyboard, this is all the text you've ever
 typed into your keyboard. For our example, let's just start with one sentence:
 ```python
 corpus = 'The quick brown fox jumps over the lazy dog'
 ```
 ## Tokenization
 Now we need to split this corpus into individual **tokens** that we can operate
 on. Since our objective is to eventually predict the next word from the previous
 word, we will want our tokens to be individual words. This process is called
 **tokenization**. The simplest way to tokenize a sentence into words is to split
 on spaces:
 ```python
 words = corpus.split(' ')
 words
 ```
 ```python
 ['The', 'quick', 'brown', 'fox', 'jumps', 'over', 'the', 'lazy', 'dog']
 ```
 ## Bigrams
 Now, we will want to create **bigrams**. A bigram is a pair of two words that
 are in the order they appear in the corpus. To create bigrams, we will iterate
 through the list of the words with two indices, one of which is offset by one:
 ```python
 bigrams = [b for b in zip(words[:-1], words[1:])]
 bigrams
 ```
 ```python
 [('The', 'quick'),
 ('quick', 'brown'),
 ('brown', 'fox'),
 ('fox', 'jumps'),
 ('jumps', 'over'),
 ('over', 'the'),
 ('the', 'lazy'),
 ('lazy', 'dog')]
 ```
 How do we use the bigrams to predict the next word given the first word?
 Return every second element where the first element matches the **condition**:
 ```python
 condition = 'the'
 next_words = [bigram[1] for bigram in bigrams
          if bigram[0].lower() == condition]
 next_words
 ```
 ```python
 ['quick', 'lazy']
 ```
 We have now found all of the possible words that can follow the condition "the"
 according to our corpus: "quick" and "lazy".
 <pre>
 (<span style="color:blue">The</span> <span style="color:red">quick</span>) (quick brown) ... (<span style="color:blue">the</span> <span style="color:red">lazy</span>) (lazy dog)
 </pre>
 Either "<span style="color:red">quick</span>" or "<span
 style="color:red">lazy</span>" could be the next word.
 ## Trigrams and N-grams
 We can partition our corpus into groups of threes too:
 <pre>
 (<span style="color:blue">The</span> <span style="color:red">quick brown</span>) (quick brown fox) ... (<span style="color:blue">the</span> <span style="color:red">lazy dog</span>)
 </pre>
 Or, the condition can be two words (`condition = 'the lazy'`):
 <pre>
 (The quick brown) (quick brown fox) ... (<span style="color:blue">the lazy</span> <span style="color:red">dog</span>)
 </pre>
 These are called **trigrams**.
 We can partition any **N** number of words together as **n-grams**.
 ## Conditional Frequency Distributions
 Earlier, we were able to compute the list of possible words to follow a
 condition:
 ```python
 next_words
 ```
 ```python
 ['quick', 'lazy']
 ```
 But, in order to predict the next word, what we really want to compute is what
 is the most likely next word out of all of the possible next words. In other
 words, find the word that occurred the most often after the condition in the
 corpus.
 We can use a **Conditional Frequency Distribution (CFD)** to figure that out! A
 **CFD** can tell us: given a **condition**, what is **likelihood** of each
 possible outcome.
 This is an example of a CFD with two conditions, displayed in table form. It is
 counting words appearing in a text collection (source: nltk.org).
 ![Two tables, one for each condition: "News" and "Romance". The first column of
 each table is 5 words: "the", "cute", "Monday", "could", and "will". The second
 column is a tally of how often the word at the start of the row appears in the
 corpus.](http://www.nltk.org/images/tally2.png)
 Let's change up our corpus a little to better demonstrate the CFD:
 ```python
 words = ('The quick brown fox jumped over the '
         'lazy dog and the quick cat').split(' ')
 print words
 ```
 ```python
 ['The', 'quick', 'brown', 'fox', 'jumped', 'over', 'the', 'lazy', 'dog', 'and', 'the', 'quick', 'cat']
 ```
 Now, let's build the CFD. I use
 [`defaultdicts`](https://docs.python.org/2/library/collections.html#defaultdict-objects)
 to avoid having to initialize every new dict.
 ```python
 from collections import defaultdict
 cfd = defaultdict(lambda: defaultdict(lambda: 0))
 for i in range(len(words) - 2):  # loop to the next-to-last word
    cfd[words[i].lower()][words[i+1].lower()] += 1
 # pretty print the defaultdict
 {k: dict(v) for k, v in dict(cfd).items()}
 ```
 ```python
 {'and': {'the': 1},
 'brown': {'fox': 1},
 'dog': {'and': 1},
 'fox': {'jumped': 1},
 'jumped': {'over': 1},
 'lazy': {'dog': 1},
 'over': {'the': 1},
 'quick': {'brown': 1},
 'the': {'lazy': 1, 'quick': 2}}
 ```
 So, what's the most likely word to follow `'the'`?
 ```python
 max(cfd['the'])
 ```
 ```python
 'quick'
 ```
 Whole sentences can be the conditions and values too. Which is basically the way
 [cleverbot](http://www.cleverbot.com/) works.
 ![An example of a conversation with Cleverbot](/img/blog/cleverbot.jpg)
 ## Random Text
 Lets put this all together, and with a little help from
 [nltk](http://www.nltk.org/) generate some random text.
 ```python
 import nltk
 import random
 TEXT = nltk.corpus.gutenberg.words('austen-emma.txt')
 # NLTK shortcuts :)
 bigrams = nltk.bigrams(TEXT)
 cfd = nltk.ConditionalFreqDist(bigrams)
 # pick a random word from the corpus to start with
 word = random.choice(TEXT)
 # generate 15 more words
 for i in range(15):
    print word,
    if word in cfd:
        word = random.choice(cfd[word].keys())
    else:
        break
 ```
 Which outputs something like:
 ```
 her reserve and concealment towards some feelings in moving slowly together .
 You will shew
 ```
 Great! This is basically what the phone keyboard suggestions are doing. Now how
 do we take this to the next level and generate text that looks like a poem?
 ## Random Poems
 Generating random poems is accomplished by limiting the choice of the next word
 by some constraint:
 * words that rhyme with the previous line
 * words that match a certain syllable count
 * words that alliterate with words on the same line
 * etc.
 ## Rhyming
 ### Written English != Spoken English
 English has a highly **nonphonemic orthography**, meaning that the letters often
 have no correspondence to the pronunciation. E.g.:
 > "meet" vs. "meat"
 The vowels are spelled differently, yet they rhyme [^1].
 So if the spelling of the words is useless in telling us if two words rhyme,
 what can we use instead?
 ### International Phonetic Alphabet (IPA)
 The IPA is an alphabet that can represent all varieties of human pronunciation.
 * meet: /mit/
 * meat: /mit/
 Note that this is only the IPA transcription for only one **accent** of English.
 Some English speakers may pronounce these words differently which could be
 represented by a different IPA transcription.
 ## Syllables
 How can we determine the number of syllables in a word? Let's consider the two
 words "poet" and "does":
 * "poet" = 2 syllables
 * "does" = 1 syllable
 The vowels in these two words are written the same, but are pronounced
 differently with a different number of syllables.
 Can the IPA tell us the number of syllables in a word too? 
 * poet: /ˈpoʊət/
 * does: /ˈdʌz/
 Not really... We cannot easily identify the number of syllables from those
 transcriptions. Sometimes the transcriber denotes syllable breaks with a `.` or
 a `'`, but sometimes they don't.
 ### Arpabet
 The Arpabet is a phonetic alphabet developed by ARPA in the 70s that:
 * Encodes phonemes specific to American English.
 * Meant to be a machine readable code. It is ASCII only.
 * Denotes how stressed every vowel is from 0-2.
 This is perfect! Because of that third bullet, a word's syllable count equals
 the number of digits in the Arpabet encoding.
 ### CMU Pronouncing Dictionary (CMUdict)
 A large open source dictionary of English words to North American pronunciations
 in Arpanet encoding. Conveniently, it is also in NLTK...
 ### Counting Syllables
 ```python
 import string
 from nltk.corpus import cmudict
 cmu = cmudict.dict()
 def count_syllables(word):
    lower_word = word.lower()
    if lower_word in cmu:
        return max([len([y for y in x if y[-1] in string.digits])
                    for x in cmu[lower_word]])
 print("poet: {}\ndoes: {}".format(count_syllables("poet"),
                                  count_syllables("does")))
 ```
 Results in:
 ```
 poet: 2
 does: 1
 ```
 ## Buzzfeed Haiku Generator
 To see this in action, try out a haiku generator I created that uses Buzzfeed
 article titles as a corpus. It does not incorporate rhyming, it just counts the
 syllables to make sure it's [5-7-5](https://en.wikipedia.org/wiki/Haiku). You can view the full code
 [here](https://github.com/thallada/nlp/blob/master/generate_poem.py).
 ![Buzzfeed Haiku Generator](/img/blog/buzzfeed.jpg)
 Run it live at:
 [http://mule.hallada.net/nlp/buzzfeed-haiku-generator/](http://mule.hallada.net/nlp/buzzfeed-haiku-generator/)
 ## Syntax-aware Generation
 Remember these?
 ![Example Mad Libs: "A Visit to the Dentist"](/img/blog/madlibs.jpg)
 Mad Libs worked so well because they forced the random words (chosen by the
 players) to fit into the syntactical structure and parts-of-speech of an
 existing sentence.
 You end up with **syntactically** correct sentences that are **semantically**
 random. We can do the same thing!
 ### NLTK Syntax Trees!
 NLTK can parse any sentence into a [syntax
 tree](http://www.nltk.org/book/ch08.html). We can utilize this syntax tree
 during poetry generation.
 ```python
 from stat_parser import Parser
 parsed = Parser().parse('The quick brown fox jumps over the lazy dog.')
 print parsed
 ```
 Syntax tree output as an
 [s-expression](https://en.wikipedia.org/wiki/S-expression):
 ```
 (S
  (NP (DT the) (NN quick))
  (VP
    (VB brown)
    (NP
      (NP (JJ fox) (NN jumps))
      (PP (IN over) (NP (DT the) (JJ lazy) (NN dog)))))
  (. .))
 ```
 ```python
 parsed.pretty_print()
 ```
 And the same tree visually pretty printed in ASCII:
 ```
                              S                            
      ________________________|__________________________   
     |               VP                                  | 
     |           ____|_____________                      |  
     |          |                  NP                    | 
     |          |         _________|________             |  
     |          |        |                  PP           | 
     |          |        |          ________|___         |  
     NP         |        NP        |            NP       | 
  ___|____      |     ___|____     |     _______|____    |  
 DT       NN    VB   JJ       NN   IN   DT      JJ   NN  . 
 |        |     |    |        |    |    |       |    |   |  
 the     quick brown fox     jumps over the     lazy dog  . 
 ```
 NLTK also performs [part-of-speech tagging](http://www.nltk.org/book/ch05.html)
 on the input sentence and outputs the tag at each node in the tree. Here's what
 each of those mean:
 |**S**  | Sentence |
 |**VP** | Verb Phrase |
 |**NP** | Noun Phrase |
 |**DT** | Determiner |
 |**NN** | Noun (common, singular) |
 |**VB** | Verb (base form) |
 |**JJ** | Adjective (or numeral, ordinal) |
 |**.**  | Punctuation |
 Now, let's use this information to swap matching syntax sub-trees between two
 corpora ([source for the generate
 function](https://github.com/thallada/nlp/blob/master/syntax_aware_generate.py)).
 ```python
 from syntax_aware_generate import generate
 # inserts matching syntax subtrees from trump.txt into
 # trees from austen-emma.txt
 generate('trump.txt', word_limit=10)
 ```
 ```
 (SBARQ
  (SQ
    (NP (PRP I))
    (VP (VBP do) (RB not) (VB advise) (NP (DT the) (NN custard))))
  (. .))
 I do not advise the custard .
 ==============================
 I do n't want the drone !
 (SBARQ
  (SQ
    (NP (PRP I))
    (VP (VBP do) (RB n't) (VB want) (NP (DT the) (NN drone))))
  (. !))
 ```
 Above the line is a sentence selected from a corpus of Jane Austen's *Emma*.
 Below it is a sentence generated by walking down the syntax tree and finding
 sub-trees from a corpus of Trump's tweets that match the same syntactical
 structure and then swapping the words in.
 The result can sometimes be amusing, but more often than not, this approach
 doesn't fare much better than the n-gram based generation.
 ### spaCy
 I'm only beginning to experiment with the [spaCy](https://spacy.io/) Python
 library, but I like it a lot. For one, it is much, much faster than NLTK:
 ![spaCy speed comparison](/img/blog/spacy_speed.jpg)
 [https://spacy.io/docs/api/#speed-comparison](https://spacy.io/docs/api/#speed-comparison)
 The [API](https://spacy.io/docs/api/) takes a little getting used to coming from
 NLTK. It doesn't seem to have any sort of out-of-the-box solution to printing
 out syntax trees like above, but it does do [part-of-speech
 tagging](https://spacy.io/docs/api/tagger) and [dependency relation
 mapping](https://spacy.io/docs/api/dependencyparser) which should accomplish
 about the same. You can see both of these visually with
 [displaCy](https://demos.explosion.ai/displacy/).
 ## Neural Network Based Generation
 If you haven't heard all the buzz about [neural
 networks](https://en.wikipedia.org/wiki/Artificial_neural_network), they are a
 particular technique for [machine
 learning](https://en.wikipedia.org/wiki/Machine_learning) that's inspired by our
 understanding of the human brain. They are structured into layers of nodes which
 have connections to other nodes in other layers of the network. These
 connections have weights which each node multiplies by the corresponding input
 and enters into a particular [activation
 function](https://en.wikipedia.org/wiki/Activation_function) to output a single
 number. The optimal weights for solving a particular problem with the network
 are learned by training the network using
 [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) to perform
 [gradient descent](https://en.wikipedia.org/wiki/Gradient_descent) on a
 particular [cost function](https://en.wikipedia.org/wiki/Loss_function) that
 tries to balance getting the correct answer while also
 [generalizing](https://en.wikipedia.org/wiki/Regularization_(mathematics)) the
 network enough to perform well on data the network hasn't seen before.
 [Long short-term memory
 (LSTM)](https://en.wikipedia.org/wiki/Long_short-term_memory) is a type of
 [recurrent neural network
 (RNN)](https://en.wikipedia.org/wiki/Recurrent_neural_network) (a network with
 cycles) that can remember previous values for a short or long period of time.
 This property makes them remarkably effective at a multitude of tasks, one of
 which is predicting text that will follow a given sequence. We can use this to
 continually generate text by inputting a seed, appending the generated output to
 the end of the seed, removing the first element from the beginning of the seed,
 and then inputting the seed again, following the same process until we've
 generated enough text from the network ([paper on using RNNs to generate
 text](http://www.cs.utoronto.ca/~ilya/pubs/2011/LANG-RNN.pdf)).
 Luckily, a lot of smart people have done most of the legwork so you can just
 download their neural network architecture and train it yourself. There's
 [char-rnn](https://github.com/karpathy/char-rnn) which has some [really exciting
 results for generating texts (e.g. fake
 Shakespeare)](http://karpathy.github.io/2015/05/21/rnn-effectiveness/). There's
 also [word-rnn](https://github.com/larspars/word-rnn) which is a modified
 version of char-rnn that operates on words as a unit instead of characters.
 Follow [my last blog post on how to install TensorFlow on Ubuntu
 16.04](/2017/06/20/how-to-install-tensorflow-on-ubuntu-16-04.html) and
 you'll be almost ready to run a TensorFlow port of word-rnn:
 [word-rnn-tensorflow](https://github.com/hunkim/word-rnn-tensorflow).
 I plan on playing around with NNs a lot more to see what kind of poetry-looking
 text I can generate from them.
 ---
 [^1]:
    Fun fact: They used to be pronounced differently in Middle English during
    the invention of the printing press and standardized spelling. The [Great
    Vowel Shift](https://en.wikipedia.org/wiki/Great_Vowel_Shift) happened
    after, and is why they are now pronounced the same.
--- a/_posts/2017-08-07-proximity-structures.md
+++ b/_posts/2017-08-07-proximity-structures.md
@ -1,76 +0,0 @@
 ---
 title: "Proximity Structures: Playing around with PixiJS"
 layout: post
 image: /img/blog/proximity-structures.png
 ---
 I've been messing around with a library called [PixiJS](http://www.pixijs.com/)
 which allows you to create WebGL animations which will fall back to HTML5 canvas
 if WebGL is not available in the browser. I mostly like it because the API is
 similar to HTML5 canvas which [I was already familiar
 with](https://github.com/thallada/thallada.github.io/blob/master/js/magic.js). I
 can't say that I like the PixiJS API and documentation that much, though. For
 this project, I mostly just used a small portion of it to create [WebGL (GPU
 accelerated) primitive
 shapes](http://www.goodboydigital.com/pixi-webgl-primitives/) (lines and
 circles).
 <!--excerpt-->
 **Play with it here**: [http://proximity.hallada.net](http://proximity.hallada.net)
 **Read/clone the code here**: [https://github.com/thallada/proximity-structures](https://github.com/thallada/proximity-structures)
 [![The animation in
 action](/img/blog/proximity-structures.gif)](http://proximity.hallada.net)
 The idea was inspired by
 [all](https://thumb9.shutterstock.com/display_pic_with_logo/3217643/418838422/stock-vector-abstract-technology-futuristic-network-418838422.jpg)
 [those](https://ak5.picdn.net/shutterstock/videos/27007555/thumb/10.jpg)
 [countless](https://ak9.picdn.net/shutterstock/videos/10477484/thumb/1.jpg)
 [node](https://ak3.picdn.net/shutterstock/videos/25825727/thumb/1.jpg)
 [network](https://t4.ftcdn.net/jpg/00/93/24/21/500_F_93242102_mqtDljufY7CNY0wMxunSbyDi23yNs1DU.jpg)
 [graphics](https://ak6.picdn.net/shutterstock/videos/12997085/thumb/1.jpg) that
 I see all the time as stock graphics on generic tech articles.
 This was really fun to program. I didn't care much about perfect code, I just
 kept hacking one thing onto another while watching the instantaneous feedback of
 the points and lines responding to my changes until I had something worth
 sharing.
 ### Details
 The majority of the animation you see is based on
 [tweening](https://en.wikipedia.org/wiki/Inbetweening). Each point has an origin
 and destination stored in memory. Every clock tick (orchestrated by the almighty
 [requestAnimationFrame](https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame)),
 the main loop calculates where each point should be in the path between its
 origin and destination based on how long until it completes its "cycle".  There
 is a global `cycleDuration`, defaulted to 60. Every frame increments the cycle
 counter by 1 until it reaches 60, at which point it folds over back to 0. Every
 point is assigned a number between 1 and 60. This is its start cycle. When the
 global cycle counter equals a point's start cycle number, the point has reached
 its destination and a new target destination is randomly chosen.
 Each point is also randomly assigned a color. When a point is within
 `connectionDistance` of another point in the canvas, a line is drawn between the
 two points, their colors are averaged, and the points' colors become the average
 color weighted by the distance between the points. You can see clusters of
 points converging on a color in the animation.
 Click interaction is implemented by modifying point target destinations within a
 radius around the click. Initially, a mouse hover will push points away.
 Clicking and holding will draw points in, progressively growing the effect
 radius in the process to capture more and more points.
 I thought it was really neat that without integrating any physics engine
 whatsoever, I ended up with something that looked sort of physics based thanks
 to the tweening functions. Changing the tweening functions that the points use
 seems to change the physical properties and interactions of the points. The
 elastic tweening function makes the connections between the points snap like
 rubber bands. And, while I am not drawing any explicit polygons, just points and
 lines based on proximity, it sometimes looks like the points are coalescing into
 some three-dimensional structure.
 I'll probably make another procedural animation like this in the future since it
 was so fun. Next time, I'll probably start from the get-go in ES2015 (or ES7,
 or ES8??) and proper data structures.
--- a/_posts/2017-08-30-making-mailing-list-jekyll-blog-using-sendy.md
+++ b/_posts/2017-08-30-making-mailing-list-jekyll-blog-using-sendy.md
@ -1,185 +0,0 @@
 ---
 title: "Making a Mailing List for a Jekyll Blog Using Sendy"
 layout: post
 ---
 When my beloved [Google Reader](https://en.wikipedia.org/wiki/Google_Reader) was
 discontinued in 2013, I stopped regularly checking RSS feeds. Apparently, [I am
 not alone](https://trends.google.com/trends/explore?date=all&q=rss). It seems
 like there's a new article every month arguing either that [RSS is dead or RSS
 is not dead
 yet](https://hn.algolia.com/?q=&query=rss%20dead&sort=byPopularity&prefix&page=0&dateRange=all&type=story).
 Maybe RSS will stick around to serve as a cross-site communication backbone, but
 I don't think anyone will refute that RSS feeds are declining in consumer use.
 Facebook, Twitter, and other aggregators are where people really go. However, I
 noticed that I still follow some small infrequent blogs through mailing lists
 that they offer. I'm really happy to see an email sign up on blogs I like,
 because it means I'll know when they post new content in the future. I check my
 email regularly unlike my RSS feeds.
 <!--excerpt-->
 Even though I'm sure my blog is still too uninteresting and unheard of to get
 many signups, I still wanted to know what it took to make a blog mailing list.
 RSS is super simple for website owners, because all they need to do is dump all
 of their content into a specially formatted XML file, host it, and let RSS
 readers deal with all the complexity. In my blog, [I didn't even need a
 Jekyll
 plugin](https://github.com/thallada/thallada.github.io/blob/master/feed.xml).
 Email is significantly more difficult. With email, the website owner owns more
 of the complexity. And, spam filters make it unfeasible to roll your own email
 server. A couple people can mark you as spam, and BAM: now you are blacklisted
 and you have to move to a new IP address. This is why most people turn to a
 hosted service like [Mailchimp](https://mailchimp.com/). Though, I was
 dissatisfied with that because of the [high costs and measly free
 tier](https://mailchimp.com/pricing/).
 [Amazon Simple Email Service (SES)](https://aws.amazon.com/ses/) deals with all
 the complexity of email for you and is also
 [cheap](https://aws.amazon.com/ses/pricing/). In fact, it's free unless you have
 more than 62,000 subscribers or post way more than around once a month, and even
 after that it's a dime for every 1,000 emails sent. Frankly, no one can really
 compete with what Amazon is offering here.
 Okay, so that covers sending the emails, but what about collecting and storing
 subscriptions? SES doesn't handle any of that. I searched around a long time for
 something simple and free that wouldn't require me setting up a server [^1]. I
 eventually ended up going with [Sendy](https://sendy.co/) because it looked like
 a well-designed product exactly for this use case that also handled drafting
 emails, email templates, confirmation emails, and analytics. It costs a one-time
 fee of $59 and I was willing to fork that over for quality software. Especially
 since most other email newsletter services require some sort of monthly
 subscription that scales with the number of emails you are sending.
 Unfortunately, since Sendy is self-hosted, I had to add a dynamic server to my
 otherwise completely static Jekyll website hosted for free on [Github
 Pages](https://pages.github.com/). You can put Sendy on pretty much anything
 that runs PHP and MySQL including the cheap [t2.micro Amazon EC2 instance
 type](https://aws.amazon.com/ec2/instance-types/). If you are clever, you might
 find a cheaper way. I already had a t2.medium for general development,
 tinkering, and hosting, so I just used that.
 There are many guides out there for setting up MySQL and Apache, so I won't go
 over that. But, I do want to mention how I got Sendy to integrate with
 [nginx](https://nginx.org/en/) which is the server engine I was already using. I
 like to put separate services I'm running under different subdomains of
 my domain hallada.net even though they are running on the same server and IP
 address. For Sendy, I chose [list.hallada.net](http://list.hallada.net) [^2].
 Setting up another subdomain in nginx requires [creating a new server
 block](https://askubuntu.com/a/766369). There's [a great Gist of a config for
 powering Sendy using nginx and
 FastCGI](https://gist.github.com/refringe/6545132), but I ran into so many
 issues with the subdomain that I decided to use nginx as a proxy to the Apache
 mod_php site running Sendy. I'll just post my config here:
 ```nginx
 server {
    listen 80;
    listen [::]:80;
    server_name list.hallada.net;
    root /var/www/html/sendy;
    index index.php;
    location /l/ {
        rewrite ^/l/([a-zA-Z0-9/]+)$ /l.php?i=$1 last;
    }
    location /t/ {
        rewrite ^/t/([a-zA-Z0-9/]+)$ /t.php?i=$1 last;
    }
    location /w/ {
        rewrite ^/w/([a-zA-Z0-9/]+)$ /w.php?i=$1 last;
    }
    location /unsubscribe/ {
        rewrite ^/unsubscribe/(.*)$ /unsubscribe.php?i=$1 last;
    }
    location /subscribe/ {
        rewrite ^/subscribe/(.*)$ /subscribe.php?i=$1 last;
    }
    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:8080/sendy/;
    }
 }
 ```
 Basically, this proxies all of the requests through to Apache which I configured
 to run on port 8080 by changing the `Listen` directive in
 `/etc/apache2/ports.conf`.
 I also had to add `RewriteBase /sendy` to the end of the `.htcaccess` file in
 the sendy directory (which, for me, was in `/var/www/html/sendy`). This
 basically forces Sendy to use urls that start with `http://list.hallada.net`
 instead of `http://list.hallada.net/sendy` which I thought was redundant since I
 am dedicating the whole subdomain to sendy.
 A perplexing issue I ran into was that Gmail accounts were completely dropping
 (not even bouncing!) any emails I sent to them if I used my personal email
 `tyler@hallada.net` as the from address. I switched to `tyhallada@gmail.com` for
 the from address and emails went through fine after that [^4]. [The issue seems
 unresolved](https://forums.aws.amazon.com/thread.jspa?messageID=802461&#802461)
 as of this post.
 Lastly, I needed to create a form on my website for readers to sign up for the
 mailing list. Sendy provides the HTML in the UI to create the form, which I
 [tweaked a
 little](https://github.com/thallada/thallada.github.io/blob/master/_includes/mail-form.html)
 and placed in a [Jekyll includes template
 partial](https://jekyllrb.com/docs/includes/) that I could include on both the
 post layout and the blog index template. I refuse to pollute the internet with
 yet another annoying email newsletter form that pops up while you are trying to
 read the article, so you can find my current version at the bottom of this
 article where it belongs [^5].
 All in all, setting up a mailing list this way wasn't too bad except for the part
 where I spent way too much time fiddling with nginx configs. But, I always do
 that, so I guess that's expected.
 As for the content of the newsletter, I haven't figured out how to post the
 entirety of a blog post into the HTML format of an email as soon as I commit a
 new post yet. So, I think for now I will just manually create a new email
 campaign in Sendy (from an email template) that will have a link to the new
 post, and send that.
 ---
 [^1]:
    It would be interesting to look into creating a [Google
    Form](https://www.google.com/forms/about/) that submits rows to a [Google
    Sheet](https://www.google.com/sheets/about/) and then triggering a [AWS
    Lambda](https://aws.amazon.com/lambda/) service that iterates over the rows
    using something like [the Google Sheets Python
    API](https://developers.google.com/sheets/api/quickstart/python) and sending
    an email for every user using the [Amazon SES
    API](http://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-email-api.html)
    ([python-amazon-ses-api](https://github.com/pankratiev/python-amazon-ses-api)
    might also be useful there).
 [^2]:
    I ran into a hiccup [verifying this domain for Amazon
    SES](http://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domain-procedure.html)
    using the [Namecheap](https://www.namecheap.com/) advanced DNS settings
    because it only allowed me to set up one MX record, but I already had one
    for my root hallada.net domain that I needed. So, I moved to [Amazon's Route
    53](https://aws.amazon.com/route53/) instead [^3] which made setting up the
    [DKIM
    verification](http://docs.aws.amazon.com/ses/latest/DeveloperGuide/easy-dkim.html)
    really easy since Amazon SES gave a button to create the necessary DNS
    records directly in my Route 53 account.
 [^3]:
    As [Amazon continues its plan for world
    domination](https://www.washingtonpost.com/business/is-amazon-getting-too-big/2017/07/28/ff38b9ca-722e-11e7-9eac-d56bd5568db8_story.html)
    it appears I'm moving more and more of my personal infrastructure over to
    Amazon as well...
 [^4]: Obviously a conspiracy by Google to force domination of Gmail.
 [^5]: Yes, I really hate those pop-ups.
--- a/_posts/2017-11-15-isso-comments.md
+++ b/_posts/2017-11-15-isso-comments.md
@ -1,259 +0,0 @@
 ---
 title: "Isso Comments"
 layout: post
 ---
 I've been meaning to add a commenting system to this blog for a while, but I
 couldn't think of a good way to do it. I implemented my own commenting system on
 my [old Django personal site](https://github.com/thallada/personalsite). While I
 enjoyed working on it at the time, it was a lot of work, especially to fight the
 spam. Now that my blog is hosted statically on Github's servers, I have no way
 to host something dynamic like comments.
 <!--excerpt-->
 [Disqus](http://disqus.com/) seems to be the popular solution to this problem
 for other people that host static blogs. The way it works is that you serve a
 javascript client script on the static site you own. The script will make AJAX
 requests to a separate server that Disqus owns to retrieve comments and post new
 ones.
 The price you pay for using Disqus, however, is that [they get to sell all of
 the data that you and your commenters give
 them](https://replyable.com/2017/03/disqus-is-your-data-worth-trading-for-convenience/).
 That reason, plus the fact that I wanted something more DIY, meant this blog has
 gone without comments for a few years.
 Then I discovered [Isso](https://github.com/posativ/isso). Isso calls itself a
 lightweight alternative to [Disqus](http://disqus.com/). Isso allows you to
 install the server code on your own server so that the comment data never goes
 to a third party. Also, it does not require logging into some social media
 account just to comment. Today, I installed it on my personal AWS EC2 instance
 and added the Isso javascript client script on this blog. So far, my experience
 with it has been great and it performs exactly the way I expect.
 I hit a few snags while installing it, however.
 ## Debian Package
 **I don't recommend using the Debian package anymore as it frequently goes out 
 of date and breaks on distribution upgrades. See bottom edit.**
 There is a very handy [Debian package](https://github.com/jgraichen/debian-isso)
 that someone has made for Isso. Since my server runs Ubuntu 16.04, and Ubuntu is
 based off of Debian, this is a package I can install with my normal ubuntu
 package manager utilities. There is no PPA to install since the package is in
 the [main Ubuntu package archive](https://packages.ubuntu.com/xenial/isso). Just
 run `sudo apt-get install isso`.
 I got a bit confused after that point, though. There seems to be no
 documentation I could find about how to actually configure and start the server
 once you have installed it. This is what I did:
 ```bash
 sudo cp /etc/default/isso /etc/isso.d/available/isso.cfg
 sudo ln -s /etc/isso.d/available/isso.cfg /etc/isso.d/enabled/isso.cfg
 ```
 Then you can edit `/etc/isso.d/available/isso.cfg` with your editor of choice to
 [configure the Isso server for your
 needs](https://posativ.org/isso/docs/configuration/server/). Make sure to set
 the `host` variable to the URL for your static site.
 Once you're done, you can run `sudo service isso restart` to reload the server
 with the new configuration. `sudo service isso status` should report `Active
 (running)`.
 Right now, there should be a [gunicorn](http://gunicorn.org/) process running
 the isso server. You can check that with `top` or running `ps aux | grep
 gunicorn`, which should return something about "isso".
 ## Nginx Reverse Proxy
 In order to map the URL "comments.hallada.net" to this new gunicorn server, I
 need an [nginx reverse
 proxy](https://www.nginx.com/resources/admin-guide/reverse-proxy/).
 To do that, I made a new server block: `sudo vim
 /etc/nginx/sites-available/isso` which I added:
 ```nginx
 server {
    listen 80;
    listen [::]:80;
    server_name comments.hallada.net;
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Script-Name /isso;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://localhost:8000;
    }
 }
 ```
 Then I enabled this new server block with:
 ```bash
 sudo ln -s /etc/nginx/sites-available/isso /etc/nginx/sites-enabled/isso
 sudo systemctl restart nginx
 ```
 ## DNS Configuration
 I added a new A record for "comments.hallada.net" that pointed to my server's IP
 address to the DNS configuration for my domain (which I recently switched to
 [Amazon Route 53](https://aws.amazon.com/route53/)).
 After the DNS caches had time to refresh, visiting `http://comments.hallada.net`
 would hit the new `isso` nginx server block, which would then pass the request
 on to the gunicorn process.
 You can verify if nginx is getting the request by looking at
 `/var/log/nginx/access.log`.
 ## Adding the Isso Script to my Jekyll Site
 I created a file called `_includes/comments.html` with the contents that [the
 Isso documentation](https://posativ.org/isso/docs/quickstart/#integration)
 provides. Then, in my post template, I simply included that on the page where I
 wanted the comments to go:
 ```html
 {% include comments.html %}
 ```
 Another thing that was not immediately obvious to me is that the value of the
 `name` variable in the Isso server configuration is the URL path that you will
 need to point the Isso JavaScript client to. For example, I chose `name = blog`,
 so the `data-isso` attribute on the script tag needed to be
 `http://comments.hallada.net/blog/`.
 ## The Uncaught ReferenceError
 **You won't need to fix this if you install Isso from PIP! See bottom edit.**
 There's [an issue](https://github.com/posativ/isso/issues/318) with that Debian
 package that causes a JavaScript error in the console when trying to load the
 Isso script in the browser. I solved this by uploading the latest version of the
 Isso `embeded.min.js` file to my server, which I put at
 `/var/www/html/isso/embeded.min.js`. Then I modified the nginx server block to
 serve that file when the path matches `/isso`:
 ```nginx
 server {
    listen 80;
    listen [::]:80;
    server_name comments.hallada.net;
    root /var/www/html;
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Script-Name /isso;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://localhost:8000;
    }
    location /isso {
        try_files $uri $uri/ $uri.php?$args =404;
    }
 }
 ```
 Now requesting `http://comments.hallada.net/isso/embeded.min.js` would return
 the newer script without the bug.
 ## Sending Emails Through Amazon Simple Email Service
 I already set up [Amazon's SES](https://aws.amazon.com/ses/) in my [last
 blog
 post](http://www.hallada.net/2017/08/30/making-mailing-list-jekyll-blog-using-sendy.html).
 To get Isso to use SES to send notifications about new comments, create a new
 credential in the SES UI, and then set the `user` and `password` fields in the
 `isso.cfg` to what get's generated for the IAM user. The SES page also has
 information for what `host` and `port` to use. I used `security = starttls` and
 `port = 587`. Make sure whatever email you use for `from` is a verified email in
 SES. Also, don't forget to add your email as the `to` value.
 ## Enabling HTTPS with Let's Encrypt
 [Let's Encrypt](https://letsencrypt.org/) allows you to get SSL certificates for
 free! I had already installed the certbot/letsencrypt client before, so I just
 ran this to generate a new certificate for my new sub-domain
 "comments.hallada.net":
 ```bash
 sudo letsencrypt certonly --nginx -d comments.hallada.net
 ```
 Once that successfully completed, I added a new nginx server block for the https
 version at `/etc/nginx/sites-available/isso-https`:
 ```nginx
 server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name comments.hallada.net;
    root /var/www/html;
    ssl_certificate /etc/letsencrypt/live/comments.hallada.net/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/comments.hallada.net/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/comments.hallada.net/fullchain.pem;
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Script-Name /isso;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://localhost:8000;
    }
    location /isso {
        try_files $uri $uri/ $uri.php?$args =404;
    }
 }
 ```
 And, I changed the old http server block so that it just permanently redirects
 to the https version:
 ```nginx
 server {
    listen 80;
    listen [::]:80;
    server_name comments.hallada.net;
    root /var/www/html;
    location / {
        return 301 https://comments.hallada.net$request_uri;
    }
 }
 ```
 Then I enabled the https version:
 ```bash
 sudo ln -s /etc/nginx/sites-available/isso-https /etc/nginx/sites-enabled/isso-https
 sudo systemctl restart nginx
 ```
 I checked that I didn't get any errors visiting `https://comments.hallada.net/`,
 and then changed my Jekyll include snippet so that it pointed at the `https`
 site instead of `http`.
 Now you can securely leave a comment if you want to yell at me for writing the
 wrong thing!
 ## EDIT 5/28/2019:
 I don't recommend using the Debian package anymore since it frequently goes out 
 of date and breaks when upgrading your Linux distribution.
 Instead, follow the [Isso docs](https://posativ.org/isso/docs/install/) by 
 creating a [virtualenv](https://virtualenv.pypa.io/en/latest/) and then run `pip 
 install isso` and `pip install gunicorn` from within the virtualenv.  Then, when 
 creating [a systemd 
 service](https://github.com/jgraichen/debian-isso/blob/master/debian/isso.service), 
 make sure to point to the gunicorn executable in that virtualenv (e.g.  
 `/opt/isso/bin/gunicorn`). It should load and run Isso from the same virtualenv.
--- a/_posts/2018-04-26-studio-frontend.md
+++ b/_posts/2018-04-26-studio-frontend.md
@ -1,365 +0,0 @@
 ---
 title: "Studio-Frontend: Developing Frontend Separate from edX Platform"
 layout: post
 ---
 *This is a blog post that I originally wrote for the [edX engineering 
 blog](https://engineering.edx.org/).*
 At the core of edX is the [edx-platform](https://github.com/edx/edx-platform), a
 monolithic Django code-base 2.7 times the size of Django itself.
 <!--excerpt-->
 ```
 -------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
 -------------------------------------------------------------------------------
 ActionScript            1          118           74           23           21
 Autoconf               10          425          237          163           25
 CSS                    55        17106        14636         1104         1366
 HTML                  668        72567        36865        30306         5396
 JavaScript           1500       463147       352306        55882        54959
 JSON                   91        14583        14583            0            0
 JSX                    33         2595         2209           62          324
 LESS                    1          949          606          232          111
 Makefile                1           65           49            8            8
 Markdown               23          287          287            0            0
 Mustache                1            1            1            0            0
 Python               3277       559255       442756        29254        87245
 ReStructuredText       48         4252         4252            0            0
 Sass                  424        75559        55569         4555        15435
 Shell                  15          929          505          292          132
 SQL                     4         6283         5081         1186           16
 Plain Text            148         3521         3521            0            0
 TypeScript             20        88506        76800        11381          325
 XML                   364         5283         4757          231          295
 YAML                   36         1630         1361          119          150
 -------------------------------------------------------------------------------
 Total                6720      1317061      1016455       134798       165808
 -------------------------------------------------------------------------------
 ```
 35% of the edx-platform is JavaScript. While it has served edX well since its
 inception in 2012, reaching over 11 million learners in thousands of courses on
 [edX.org](https://www.edx.org/) and many more millions on all of the [Open edX
 instances across the
 world](https://openedx.atlassian.net/wiki/spaces/COMM/pages/162245773/Sites+powered+by+Open+edX),
 it is starting to show its age. Most of it comes in the form of [Backbone.js
 apps](http://backbonejs.org/) loaded by [RequireJS](http://requirejs.org/) in
 Django [Mako templates](http://www.makotemplates.org/), with
 [jQuery](https://jquery.com/) peppered throughout.
 Many valiant efforts are underway to modernize the frontend of edx-platform
 including replacing RequireJS with Webpack, Backbone.js with
 [React](https://reactjs.org/), and ES5 JavaScript and CoffeeScript with ES6
 JavaScript. Many of these efforts [were covered in detail at the last Open edX
 conference](https://www.youtube.com/watch?v=xicBnbDX4AY) and in [Open edX
 Proposal 11: Front End Technology
 Standards](https://open-edx-proposals.readthedocs.io/en/latest/oep-0011-bp-FED-technology.html).
 However, the size and complexity of the edx-platform means that these kind of
 efforts are hard to prioritize, and, in the meantime, frontend developers are
 forced to [wait over 10
 minutes](https://openedx.atlassian.net/wiki/spaces/FEDX/pages/264700138/Asset+Compilation+Audit+2017-11-01)
 for our home-grown asset pipeline to build before they can view changes.
 There have also been efforts to incrementally modularize and extract parts of
 the edx-platform into separate python packages that could be installed as
 [Django apps](https://docs.djangoproject.com/en/2.0/ref/applications/), or even
 as separately deployed
 [microservices](https://en.wikipedia.org/wiki/Microservices). This allows
 developers to work independently from the rest of the organization inside of a
 repository that they own, manage, and is small enough that they could feasibly
 understand it entirely.
 When my team was tasked with improving the user experience of pages in
 [Studio](https://studio.edx.org/), the tool that course authors use to create
 course content, we opted to take a similar architectural approach with the
 frontend and create a new repository where we could develop new pages in
 isolation and then integrate them back into the edx-platform as a plugin. We
 named this new independent repository
 [studio-frontend](https://github.com/edx/studio-frontend). With this approach,
 our team owns the entire studio-frontend code-base and can make the best
 architectural changes required for its features without having to consult with
 and contend with all of the other teams at edX that contribute to the
 edx-platform. Developers of studio-frontend can also avoid the platform’s slow
 asset pipeline by doing all development within the studio-frontend repository
 and then later integrating the changes into platform.
 ## React and Paragon
 When edX recently started to conform our platform to the [Web Content
 Accessibility Guidelines 2.0 AA (WCAG 2.0
 AA)](https://www.w3.org/WAI/intro/wcag), we faced many challenges in
 retrofitting our existing frontend code to be accessible.  Rebuilding Studio
 pages from scratch in studio-frontend allows us to not only follow the latest
 industry standards for building robust and performant frontend applications, but
 to also build with accessibility in mind from the beginning.
 The Javascript community has made great strides recently to [address
 accessibility issues in modern web
 apps](https://reactjs.org/docs/accessibility.html). However, we had trouble
 finding an open-source React component library that fully conformed to WCAG 2.0
 AA and met all of edX’s needs, so we decided to build our own:
 [Paragon](https://github.com/edx/paragon).
 Paragon is a library of building-block components like buttons, inputs, icons,
 and tables which were built from scratch in React to be accessible. The
 components are styled using the [Open edX theme of Bootstrap
 v4](https://github.com/edx/edx-bootstrap) (edX’s decision to adopt Bootstrap is
 covered in
 [OEP-16](https://open-edx-proposals.readthedocs.io/en/latest/oep-0016-bp-adopt-bootstrap.html)).
 Users of Paragon may also choose to use the
 [themeable](https://github.com/edx/paragon#export-targets) unstyled target and
 provide their own Bootstrap theme.
 ![Paragon's modal component displayed in
 Storybook](/img/blog/paragon-modal-storybook.jpg)
 Studio-frontend composes together Paragon components into higher-level
 components like [an accessibility
 form](https://github.com/edx/studio-frontend/blob/master/src/accessibilityIndex.jsx)
 or [a table for course assets with searching, filtering, sorting, pagination,
 and upload](https://github.com/edx/studio-frontend/blob/master/src/index.jsx).
 While we developed these components in studio-frontend, we were able to improve
 the base Paragon components. Other teams at edX using the same components were
 able to receive the same improvements with a single package update.
 ![Screenshot of the studio-frontend assets table inside of
 Studio](/img/blog/studio-frontend-assets-table.jpg)
 ## Integration with Studio
 We were able to follow the typical best practices for developing a React/Redux
 application inside studio-frontend, but at the end of the day, we still had to
 somehow get our components inside of existing Studio pages and this is where
 most of the challenges arose.
 ## Webpack
 The aforementioned move from RequireJS to Webpack in the edx-platform made it
 possible for us to build our studio-frontend components from source with Webpack
 within edx-platform. However, this approach tied us to the edx-platform’s slow
 asset pipeline. If we wanted rapid development, we had to duplicate the
 necessary Webpack config between both studio-frontend and edx-platform.
 Instead, studio-frontend handles building the development and production Webpack
 builds itself. In development mode, the incremental rebuild that happens
 automatically when a file is changed takes under a second. The production
 JavaScript and CSS bundles, which take about 25 seconds to build, are published
 with every new release to
 [NPM](https://www.npmjs.com/package/@edx%2Fstudio-frontend). The edx-platform
 `npm install`s studio-frontend and then copies the built production files from
 `node_modules` into its Django static files directory where the rest of the
 asset pipeline will pick it up.
 To actually use the built JavaScript and CSS, edx-platform still needs to
 include it in its Mako templates. We made a [Mako template
 tag](https://github.com/edx/edx-platform/blob/master/common/djangoapps/pipeline_mako/templates/static_content.html#L93-L122)
 that takes a Webpack entry point name in studio-frontend and generates script
 tags that include the necessary files from the studio-frontend package. It also
 dumps all of the initial context that studio-frontend needs from the
 edx-platform Django app into [a JSON
 object](https://github.com/edx/edx-platform/blob/master/cms/templates/asset_index.html#L36-L56)
 in a script tag on the page that studio-frontend components can access via a
 shared id. This is how studio-frontend components get initial data from Studio,
 like which course it’s embedded in.
 For performance, modules that are shared across all studio-frontend components
 are extracted into `common.min.js` and `common.min.css` files that are included
 on every Studio template that has a studio-frontend component. User's browsers
 should cache these files so that they do not have to re-download libraries like
 React and Redux every time they visit a new page that contains a studio-frontend
 component.
 ## CSS Isolation
 Since the move to Bootstrap had not yet reached the Studio part of the
 edx-platform, most of the styling clashed with the Bootstrap CSS that
 studio-frontend components introduced. And, the Bootstrap styles were also
 leaking outside of the studio-frontend embedded component `div` and affecting
 the rest of the Studio page around it.
 ![Diagram of a studio-frontend component embedded inside of
 Studio](/img/blog/studio-frontend-style-isolation.jpg)
 We were able to prevent styles leaking outside of the studio-frontend component
 by scoping all CSS to only the `div` that wraps the component. Thanks to the
 Webpack [postcss-loader](https://github.com/postcss/postcss-loader) and the
 [postcss-prepend-selector](https://github.com/ledniy/postcss-prepend-selector)
 we were able to automatically scope all of our CSS selectors to that `div` in
 our build process.
 Preventing the Studio styles from affecting our studio-frontend component was a
 much harder problem because it means avoiding the inherently cascading nature of
 CSS. A common solution to this issue is to place the 3rd-party component inside
 of an
 [`iframe`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe)
 element, which essentially creates a completely separate sub-page where both CSS
 and JavaScript are isolated from the containing page. Because `iframe`s
 introduce many other performance and styling issues, we wanted to find a
 different solution to isolating CSS.
 The CSS style [`all:
 initial`](https://developer.mozilla.org/en-US/docs/Web/CSS/all) allows
 resetting all properties on an element to their initial values as defined in the
 CSS spec. Placing this style under a wildcard selector in studio-frontend
 allowed us to reset all inherited props from the legacy Studio styles without
 having to enumerate them all by hand.
 ```css
 * {
  all: initial;
 }
 ```
 While this CSS property doesn’t have broad browser support yet, we were able to
 polyfill it thanks to postcss with the
 [postcss-initial](https://github.com/maximkoretskiy/postcss-initial) plugin.
 However, this resets the styles to *nothing*. For example, all `div`s are
 displayed inline. To return the styles back to to some sane browser default we
 had to re-apply a browser default stylesheet. You can read more about this
 technique at
 [default-stylesheet](https://github.com/thallada/default-stylesheet).
 From there, Bootstrap’s
 [reboot](https://getbootstrap.com/docs/4.0/content/reboot/) normalizes the
 browser-specific styling to a common baseline and then applies the Bootstrap
 styles conflict-free from the surrounding CSS cascade.
 There's a candidate recommendation in CSS for a [`contains`
 property](https://www.w3.org/TR/css-contain-1/), which will "allow strong,
 predictable isolation of a subtree from the rest of the page". I hope that it
 will provide a much more elegant solution to this problem once browsers support
 it.
 ## Internationalization
 Another major challenge with separating out the frontend from edx-platform was
 that most of our internationalization tooling was instrumented inside the
 edx-platform. So, in order to display text in studio-frontend components in the
 correct language we either had to pass already-translated strings from the
 edx-platform into studio-frontend, or set-up translations inside
 studio-frontend.
 We opted for the latter because it kept the content close to the code that used
 it. Every display string in a component is stored in a
 [displayMessages.jsx](https://github.com/edx/studio-frontend/blob/master/src/components/AssetsTable/displayMessages.jsx)
 file and then imported and referenced by an id within the component. A periodic
 job extracts these strings from the project, pushes them up to our translations
 service [Transifex](https://www.transifex.com/), and pulls any new translations
 to store them in our NPM package.
 Because Transifex’s `KEYVALUEJSON` file format does not allow for including
 comments in the strings for translation, [Eric](https://github.com/efischer19)
 created a library called [reactifex](https://github.com/efischer19/reactifex)
 that will send the comments in separate API calls.
 Studio includes the user’s language in the context that it sends a
 studio-frontend component for initialization. Using this, the component can
 display the message for that language if it exists. If it does not, then it will
 display the original message in English and [wrap it in a `span` with `lang="en"`
 as an
 attribute](https://github.com/edx/studio-frontend/blob/master/src/utils/i18n/formattedMessageWrapper.jsx)
 so that screen-readers know to read it in English even if their default is some
 other language.
 Read more about studio-frontend’s internationalization process in [the
 documentation that Eric
 wrote](https://github.com/edx/studio-frontend/blob/master/src/data/i18n/README.md).
 ## Developing with Docker
 To normalize the development environment across the whole studio-frontend team,
 development is done in a Docker container. This is a minimal Ubuntu 16.04
 container with specific version of Node 8 installed and its only purpose is to
 run Webpack. This follows the pattern established in [OEP-5: Pre-built
 Development
 Environments](https://open-edx-proposals.readthedocs.io/en/latest/oep-0005-arch-containerize-devstack.html)
 for running a single Docker container per process that developers can easily
 start without installing dependencies.
 Similar to edX’s [devstack](https://github.com/edx/devstack) there is a Makefile
 with commands to start and stop the docker container. The docker container then
 immediately runs [`npm run
 start`](https://github.com/edx/studio-frontend/blob/master/package.json#L12),
 which runs Webpack with the
 [webpack-dev-server](https://github.com/webpack/webpack-dev-server). The
 webpack-dev-server is a node server that serves assets built by Webpack.
 [Studio-frontend's Webpack
 config](https://github.com/edx/studio-frontend/blob/master/config/webpack.dev.config.js#L94)
 makes this server available to the developer's host machine
 at `http://localhost:18011`.
 With [hot-reload](https://webpack.js.org/concepts/hot-module-replacement/)
 enabled, developers can now visit that URL in their browser, edit source files
 in studio-frontend, and then see changes reflected instantly in their browser
 once Webpack finishes its incremental rebuild.
 However, many studio-frontend components need to be able to talk to the
 edx-platform Studio backend Django server. Using [docker’s network connect
 feature](https://docs.docker.com/compose/networking/#use-a-pre-existing-network)
 the studio-frontend container can join the developer’s existing docker devstack
 network so that the studio-frontend container can make requests to the docker
 devstack Studio container at `http://edx.devstack.studio:18010/` and Studio can
 access studio-frontend at `http://dahlia.studio-fronend:18011/`.
 The webpack-dev-server can now [proxy all
 requests](https://github.com/edx/studio-frontend/blob/master/config/webpack.dev.config.js#L101)
 to Studio API endpoints (like `http://localhost:18011/assets`)
 to `http://edx.devstack.studio:18010/`.
 ## Developing within Docker Devstack Studio
 Since studio-frontend components will be embedded inside of an existing Studio
 page shell, it’s often useful to develop on studio-frontend containers inside of
 this set-up. [This can be
 done](https://github.com/edx/studio-frontend#development-inside-devstack-studio)
 by setting a variable in the devstack's `cms/envs/private.py`:
 ```python
 STUDIO_FRONTEND_CONTAINER_URL = 'http://localhost:18011'
 ```
 This setting is checked in the Studio Mako templates wherever studio-frontend
 components are embedded. If it is set to a value other than `None`, then the
 templates will request assets from that URL instead of the Studio's own static
 assets directory. When a developer loads a Studio page with an embedded
 studio-frontend component, their studio-frontend webpack-dev-server will be
 requested at that URL. Similarly to developing on studio-frontend in isolation,
 edits to source files will trigger a Webpack compilation and the Studio page
 will be hot-reloaded or reloaded to reflect the changes automatically.
 Since the studio-frontend JS loaded on `localhost:18010` is now requesting the
 webpack-dev-server on `localhost:18011`,
 an [`Access-Control-Allow-Origin` header](https://github.com/edx/studio-frontend/blob/master/config/webpack.dev.config.js#L98)
 has to be configured on the webpack-dev-server to get around CORS violations.
 ![Diagram of studio-frontend's docker container communicating to Studio inside
 of the devstack_default docker 
 network](/img/blog/studio-frontend-docker-devstack.jpg)
 ## Deploying to Production
 [Each release of
 studio-frontend](https://github.com/edx/studio-frontend#releases) will upload
 the `/dist` files built by Webpack in production mode to
 [NPM](https://www.npmjs.com/package/@edx/studio-frontend). edx-platform
 requires a particular version of studio-frontend in its
 [`package.json`](https://github.com/edx/edx-platform/blob/master/package.json#L7).
 When a new release of edx-platform is made, `paver update_assets` will run
 which will copy all of the files in the
 `node_modules/@edx/studio-frontend/dist/` to the Studio static folder.
 Because `STUDIO_FRONTEND_CONTAINER_URL` will be `None` in production, it will be
 ignored, and Studio pages will request studio-frontend assets from that static
 folder.
 ## Future
 Instead of “bringing the new into the old”, we’d eventually like to move to a
 model where we “work in the new and bring in the old if necessary”. We could
 host studio-frontend statically on a completely separate server which talks to
 Studio via a REST (or [GraphQL](https://graphql.org/)) API. This approach would
 eliminate the complexity around CSS isolation and bring big performance wins for
 our users, but it would require us to rewrite more of Studio.
--- a/_posts/2020-02-01-generating-icosahedrons-and-hexspheres-in-rust.md
+++ b/_posts/2020-02-01-generating-icosahedrons-and-hexspheres-in-rust.md
@ -1,758 +0,0 @@
 ---
 title: "Generating icosahedrons and hexspheres in Rust"
 layout: post
 image: /img/blog/hexsphere_colored_7.png
 ---
 I've been trying to learn [Rust](https://www.rust-lang.org/) lately, the hot new 
 systems programming language. One of the projects I wanted to tackle with the 
 speed of Rust was generating 3D polyhedron shapes. Specifically, I wanted to 
 implement something like the [Three.js 
 `IcosahedronGeometry`](https://threejs.org/docs/#api/en/geometries/IcosahedronGeometry) 
 in Rust. If you try to generate 
 [icosahedron](https://en.wikipedia.org/wiki/Icosahedron)s in Three.js over any 
 detail level over 5 the whole browser will slow to a crawl. I think we can do 
 better in Rust!
 Furthermore, I wanted to generate a hexsphere: a sphere composed of hexagon 
 faces and 12 pentagon faces, otherwise known as a truncated icosahedron or the 
 [Goldberg polyhedron](https://en.wikipedia.org/wiki/Goldberg_polyhedron). The 
 shape would be ideal for a game since (almost) every tile would have the same 
 area and six sides to defend or attack from. There's a few [Javascript projects 
 for generating hexspheres](https://www.robscanlon.com/hexasphere/). Most of them 
 generate the shape by starting with a subdivided icosahedron and then truncating 
 the sides into hexagons. Though, there [exist other methods for generating the 
 hexsphere 
 shape](https://stackoverflow.com/questions/46777626/mathematically-producing-sphere-shaped-hexagonal-grid).
 **Play around with all of these shapes in your browser at: 
 [https://www.hallada.net/planet/](https://www.hallada.net/planet/).**
 So, how would we go about generating a hexsphere from scratch?
 <!--excerpt-->
 ### The Icosahedron Seed
 To start our sculpture, we need our ball of clay. The most basic shape that we 
 start with can be defined by its 20 triangle faces and 12 vertices: the regular 
 icosahedron. If you've ever played Dungeons and Dragons, this is the 20-sided 
 die.
 To define this basic shape in Rust, we first need to define a few structs. The 
 most basic unit we need is a 3D vector which describes a single point in 3D 
 space with a X, Y, and Z float values. I could have defined this myself, but to 
 avoid having to implement a bunch of vector operations (like add, subtract, 
 multiply, etc.) I chose to import 
 [`Vector3`](https://docs.rs/cgmath/0.17.0/cgmath/struct.Vector3.html) from the 
 [cgmath crate](https://crates.io/crates/cgmath).
 The next struct we need is `Triangle`. This will define a face between three 
 vertices:
 ```rust
 #[derive(Debug)]
 pub struct Triangle {
    pub a: usize,
    pub b: usize,
    pub c: usize,
 }
 impl Triangle {
    fn new(a: usize, b: usize, c: usize) -> Triangle {
        Triangle { a, b, c }
    }
 }
 ```
 We use `usize` for the three points of the triangle because they are indices 
 into a [`Vec`](https://doc.rust-lang.org/std/vec/struct.Vec.html) of `Vector3`s.
 To keep these all together, I'll define a `Polyhedron` struct:
 ```rust
 #[derive(Debug)]
 pub struct Polyhedron {
    pub positions: Vec<Vector3>,
    pub cells: Vec<Triangle>,
 }
 ```
 With this, we can define the regular icosahedron:
 ```rust
 impl Polyhedron {
    pub fn regular_isocahedron() -> Polyhedron {
        let t = (1.0 + (5.0 as f32).sqrt()) / 2.0;
        Polyhedron {
            positions: vec![
                Vector3::new(-1.0, t, 0.0),
                Vector3::new(1.0, t, 0.0),
                Vector3::new(-1.0, -t, 0.0),
                Vector3::new(1.0, -t, 0.0),
                Vector3::new(0.0, -1.0, t),
                Vector3::new(0.0, 1.0, t),
                Vector3::new(0.0, -1.0, -t),
                Vector3::new(0.0, 1.0, -t),
                Vector3::new(t, 0.0, -1.0),
                Vector3::new(t, 0.0, 1.0),
                Vector3::new(-t, 0.0, -1.0),
                Vector3::new(-t, 0.0, 1.0),
            ],
            cells: vec![
                Triangle::new(0, 11, 5),
                Triangle::new(0, 5, 1),
                Triangle::new(0, 1, 7),
                Triangle::new(0, 7, 10),
                Triangle::new(0, 10, 11),
                Triangle::new(1, 5, 9),
                Triangle::new(5, 11, 4),
                Triangle::new(11, 10, 2),
                Triangle::new(10, 7, 6),
                Triangle::new(7, 1, 8),
                Triangle::new(3, 9, 4),
                Triangle::new(3, 4, 2),
                Triangle::new(3, 2, 6),
                Triangle::new(3, 6, 8),
                Triangle::new(3, 8, 9),
                Triangle::new(4, 9, 5),
                Triangle::new(2, 4, 11),
                Triangle::new(6, 2, 10),
                Triangle::new(8, 6, 7),
                Triangle::new(9, 8, 1),
            ],
        }
    }
 }
 ```
 ### JSON Serialization
 To prove this works, we need to be able to output our shape to some format that 
 will be able to be rendered. Coming from a JS background, I'm only familiar with 
 rendering shapes with WebGL. So, I need to be able to serialize the shape to 
 JSON so I can load it in JS.
 There's an amazing library in Rust called 
 [serde](https://crates.io/crates/serde) that will make this very 
 straightforward. We just need to import it and `impl Serialize` for all of our 
 structs.
 The JSON structure we want will look like this. This is what Three.js expects 
 when initializing 
 [`BufferGeometry`](https://threejs.org/docs/#api/en/core/BufferGeometry).
 ```json
 {
    "positions": [
        [
          -0.8506508,
          0,
          0.5257311
        ],
        ...
    ],
    "cells": [
        [
            0,
            1,
            2,
        ],
        ...
    ],
 }
 ```
 For the `"cells"` array, we'll need to serialize `Triangle` into an array of 3 
 integer arrays:
 ```rust
 impl Serialize for Triangle {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        let vec_indices = vec![self.a, self.b, self.c];
        let mut seq = serializer.serialize_seq(Some(vec_indices.len()))?;
        for index in vec_indices {
            seq.serialize_element(&index)?;
        }
        seq.end()
    }
 }
 ```
 I had some trouble serializing the `cgmath::Vector3` to an array, so I made my 
 own type that wrapped `Vector3` that could be serialized to an array of 3 
 floats.
 ```rust
 #[derive(Debug)]
 pub struct ArraySerializedVector(pub Vector3<f32>);
 impl Serialize for ArraySerializedVector {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        let values = vec![self.0.x, self.0.y, self.0.z];
        let mut seq = serializer.serialize_seq(Some(values.len()))?;
        for value in values {
            seq.serialize_element(&value)?;
        }
        seq.end()
    }
 }
 ```
 And now `Polyhedron` needs to use this new type and implement `Serialize` for 
 the whole shape to get serialized:
 ```rust
 #[derive(Serialize, Debug)]
 pub struct Polyhedron {
    pub positions: Vec<ArraySerializedVector>,
    pub cells: Vec<Triangle>,
 }
 ```
 The actual serialization is done with:
 ```rust
 fn write_to_json_file(polyhedron: Polyhedron, path: &Path) {
    let mut json_file = File::create(path).expect("Can't create file");
    let json = serde_json::to_string(&polyhedron).expect("Problem serializing");
    json_file
        .write_all(json.as_bytes())
        .expect("Can't write to file");
 }
 ```
 On the JS side, the `.json` file can be read and simply fed into either Three.js 
 or [regl](https://github.com/regl-project/reg) to be rendered in WebGL ([more on 
 that later](#rendering-in-webgl-with-regl)).
 ![Regular Icosahedron](/img/blog/icosahedron_colored_1.png)
 ## Subdivided Icosahedron
 Now, we need to take our regular icosahedron and subdivide its faces N number of 
 times to generate an icosahedron with a detail level of N.
 I pretty much copied must of [the subdividing code from 
 Three.js](https://github.com/mrdoob/three.js/blob/34dc2478c684066257e4e39351731a93c6107ef5/src/geometries/PolyhedronGeometry.js#L90) 
 directly into Rust.
 I won't bore you with the details here, you can find the function 
 [here](https://github.com/thallada/icosahedron/blob/9643757df245e29f5ecfbb25f9a2c06b3a4e1217/src/lib.rs#L160-L205).
 ![Subdivided Icosahedron](/img/blog/icosahedron_colored_3.png)
 ### Truncated Icosahedron
 Now we get to the meat of this project. Transforming an icosahedron into a 
 hexsphere by 
 [truncating](https://en.wikipedia.org/wiki/Truncation_%28geometry%29) the points 
 of the icosahedron into hexagon and pentagon faces.
 You can imagine this operation as literally cutting off the points of the 
 subdivided icosahedron at exactly the midpoint between the point and it's six or 
 five neighboring points.
 ![Image of biggest dodecahedron inside 
 icosahedron](/img/blog/dodecahedron_in_icosahedron.png)
 ([image source](http://www.oz.nthu.edu.tw/~u9662122/DualityProperty.html))
 In this image you can see the regular icosahedron (0 subdivisions) in wireframe 
 with a yellow shape underneath which is the result of all 12 points truncated to 
 12 pentagon faces, in other words: the [regular 
 dodecahedron](https://en.wikipedia.org/wiki/Dodecahedron).
 You can see that the points of the new pentagon faces will be the exact center 
 of the original triangular faces. It should now make sense why truncating a 
 shape with 20 faces of 3 edges each results in a shape with 12 faces of 5 edges 
 each. Each pair multiplied still equals 60.
 #### Algorithm
 There are many different algorithms you could use to generate the truncated 
 shape, but this is roughly what I came up with:
 1. Store a map of every icosahedron vertex to faces composed from that vertex 
   (`vert_to_faces`).
 2. Calculate and cache the [centroid](https://en.wikipedia.org/wiki/Centroid) of 
   every triangle in the icosahedron (`triangle_centroids`).
 3. For every vertex in the original icosahedron:
 4. Find the center point between all of centroids of all of the faces for that 
   vertex (`center_point`). This is essentially the original icosahedron point 
   but lowered towards the center of the polygon since it will eventually be the 
   center of a new flat hexagon face.
   ![hexagon center point in red with original icosahedron faces fanning 
   out](/img/blog/hexagon_fan.png)
 5. For every triangle face composed from the original vertex:
   ![hexagon fan with selected triangle face in 
   blue](/img/blog/hexagon_fan_triangle_selected.png)
 6. Sort the vertices of the triangle face so there is a vertex `A` in the center 
   of the fan like in the image, and two other vertices `B` and `C` at the edges 
   of the hexagon.
 7. Find the centroid of the selected face. This will be one of the five or six 
   points of the new pentagon or hexagon (in brown in diagram below: 
   `triangleCentroid`).
 8. Find the mid point between `AB` and `AC` (points `midAB` and `midAC` in 
   diagram).
 9. With these mid points and the face centroid, we now have two new triangles 
   (in orange below) that form one-fifth or one-sixth of the final pentagon or 
   hexagon face. Add the points of the triangle to the `positions` array. Add 
   the two new triangles composed from those vertices as indexes into the 
   `positions` array to the `cells` array. We need to compose the pentagon or 
   hexagon out of triangles because in graphics everything is a triangle, and 
   this is the simplest way to tile either shape with triangles:
   ![hexagon fan ](/img/blog/hexagon_fan_construct.png)
 10. Go to step 5 until all faces of the icosahedron vertex have been visited.  
    Save indices to all new triangles in the `cells` array, which now form a 
    complete pentagon or hexagon face, to the `faces` array.
    ![hexagons tiling on icosahedron faces](/img/blog/hexagon_tiling.png)
 11. Go to step 3 until all vertices in the icosahedron have been visited. The 
    truncated icosahedron is now complete.
 ![colored hexsphere of detail level 3](/img/blog/hexsphere_colored_3.png)
 #### Code
 The `truncate` function calls out to a bunch of other functions, so [here's a 
 link to the function within the context of the whole 
 file](https://github.com/thallada/icosahedron/blob/9643757df245e29f5ecfbb25f9a2c06b3a4e1217/src/lib.rs#L227).
 ### Calculating Normals
 It took me a surprisingly long time to figure out how to compute 
 [normals](https://en.wikipedia.org/wiki/Normal_(geometry)) for the truncated 
 icosahedron. I tried just using an out-of-the-box solution like 
 [angle-normals](https://github.com/mikolalysenko/angle-normals/blob/master/angle-normals.js) 
 which could supposedly calculate the normal vectors for you, but they came out 
 all wrong.
 ![hexsphere with bad normals](/img/blog/bad_hexsphere_normals.png)
 So, I tried doing it myself. Most tutorials on computing normal vectors for a 
 mesh assume that it is tiled in a particular way. But, my algorithm spins around 
 icosahedron points in all different directions, and so the triangle points are 
 not uniformly in clockwise or counter-clockwise order.
 I could have sorted these points into the correct order, but I found it easier 
 to instead just detect when the normal was pointing the wrong way and just 
 invert it.
 ```rust
 pub fn compute_triangle_normals(&mut self) {
    let origin = Vector3::new(0.0, 0.0, 0.0);
    for i in 0..self.cells.len() {
        let vertex_a = &self.positions[self.cells[i].a].0;
        let vertex_b = &self.positions[self.cells[i].b].0;
        let vertex_c = &self.positions[self.cells[i].c].0;
        let e1 = vertex_a - vertex_b;
        let e2 = vertex_c - vertex_b;
        let mut no = e1.cross(e2);
        // detect and correct inverted normal
        let dist = vertex_b - origin;
        if no.dot(dist) < 0.0 {
            no *= -1.0;
        }
        let normal_a = self.normals[self.cells[i].a].0 + no;
        let normal_b = self.normals[self.cells[i].b].0 + no;
        let normal_c = self.normals[self.cells[i].c].0 + no;
        self.normals[self.cells[i].a] = ArraySerializedVector(normal_a);
        self.normals[self.cells[i].b] = ArraySerializedVector(normal_b);
        self.normals[self.cells[i].c] = ArraySerializedVector(normal_c);
    }
    for normal in self.normals.iter_mut() {
        *normal = ArraySerializedVector(normal.0.normalize());
    }
 }
 ```
 ### Assigning Random Face Colors
 Finally, all that's left to generate is the face colors. The only way I could 
 figure out how to individually color a shape's faces in WebGL was to pass a 
 color per vertex. The issue with this is that each vertex of the generated 
 shapes could be shared between many different faces.
 How can we solve this? At the cost of memory, we can just duplicate a vertex 
 every time it's used by a different triangle. That way no vertex is shared.
 This can be done after a shape has been generated with shared vertices.
 ```rust
 pub fn unique_vertices(&mut self, other: Polyhedron) {
    for triangle in other.cells {
        let vertex_a = other.positions[triangle.a].0;
        let vertex_b = other.positions[triangle.b].0;
        let vertex_c = other.positions[triangle.c].0;
        let normal_a = other.normals[triangle.a].0;
        let normal_b = other.normals[triangle.b].0;
        let normal_c = other.normals[triangle.c].0;
        self.positions.push(ArraySerializedVector(vertex_a));
        self.positions.push(ArraySerializedVector(vertex_b));
        self.positions.push(ArraySerializedVector(vertex_c));
        self.normals.push(ArraySerializedVector(normal_a));
        self.normals.push(ArraySerializedVector(normal_b));
        self.normals.push(ArraySerializedVector(normal_c));
        self.colors
            .push(ArraySerializedVector(Vector3::new(1.0, 1.0, 1.0)));
        self.colors
            .push(ArraySerializedVector(Vector3::new(1.0, 1.0, 1.0)));
        self.colors
            .push(ArraySerializedVector(Vector3::new(1.0, 1.0, 1.0)));
        let added_index = self.positions.len() - 1;
        self.cells
            .push(Triangle::new(added_index - 2, added_index - 1, added_index));
    }
    self.faces = other.faces;
 }
 ```
 With unique vertices, we can now generate a random color per face with the [rand 
 crate](https://crates.io/crates/rand).
 ```rust
 pub fn assign_random_face_colors(&mut self) {
    let mut rng = rand::thread_rng();
    for i in 0..self.faces.len() {
        let face_color = Vector3::new(rng.gen(), rng.gen(), rng.gen());
        for c in 0..self.faces[i].len() {
            let face_cell = &self.cells[self.faces[i][c]];
            self.colors[face_cell.a] = ArraySerializedVector(face_color);
            self.colors[face_cell.b] = ArraySerializedVector(face_color);
            self.colors[face_cell.c] = ArraySerializedVector(face_color);
        }
    }
 }
 ```
 ### Binary Serialization
 Now that we have to duplicate vertices for individual face colors, the size of 
 our JSON outputs are getting quite big:
 | File  | Size |
 |---|---|
 | icosahedron_r1_d6.json | 28 MB |
 | icosahedron_r1_d7.json | 113 MB |
 | hexsphere_r1_d5.json | 42 MB |
 | hexsphere_r1_d6.json | 169 MB |
 Since all of our data is just floating point numbers, we could reduce the size 
 of the output considerably by using a binary format instead.
 I used the [byteorder](https://docs.rs/byteorder/1.3.2/byteorder/) crate to 
 write out all of the `Vec`s in my `Polyhedron` struct to a binary file in 
 little-endian order.
 The binary format is laid out as:
 1. 1 32 bit unsigned integer specifying the number of vertices (`V`)
 2. 1 32 bit unsigned integer specifying the number of triangles (`T`)
 3. `V` * 3 number of 32 bit floats for every vertex's x, y, and z coordinate
 4. `V` * 3 number of 32 bit floats for the normals of every vertex
 5. `V` * 3 number of 32 bit floats for the color of every vertex
 6. `T` * 3 number of 32 bit unsigned integers for the 3 indices into the vertex 
   array that make every triangle
 The `write_to_binary_file` function which does all that is 
 [here](https://github.com/thallada/icosahedron/blob/9643757df245e29f5ecfbb25f9a2c06b3a4e1217/src/bin.rs#L13).
 That's a lot better:
 | File  | Size |
 |---|---|
 | icosahedron_r1_d6.bin | 9.8 MB |
 | icosahedron_r1_d7.bin | 11 MB |
 | hexsphere_r1_d5.bin | 14 MB |
 | hexsphere_r1_d6.bin | 58 MB |
 On the JavaScript side, the binary files can be read into `Float32Array`s like 
 this:
 ```javascript
 fetch(binaryFile)
  .then(response => response.arrayBuffer())
  .then(buffer => {
    let reader = new DataView(buffer);
    let numVertices = reader.getUint32(0, true);
    let numCells = reader.getUint32(4, true);
    let shape = {
      positions: new Float32Array(buffer, 8, numVertices * 3),
      normals: new Float32Array(buffer, numVertices * 12 + 8, numVertices * 3),
      colors: new Float32Array(buffer, numVertices * 24 + 8, numVertices * 3),
      cells: new Uint32Array(buffer, numVertices * 36 + 8, numCells * 3),
    })
 ```
 ### Rendering in WebGL with Regl
 I was initially rendering the shapes with Three.js but switched to 
 [regl](https://github.com/regl-project/regl) because it seemed like a more 
 direct abstraction over WebGL. It makes setting up a WebGL renderer incredibly 
 easy compared to all of the dozens cryptic function calls you'd have to 
 otherwise use.
 This is pretty much all of the rendering code using regl in my [3D hexsphere and 
 icosahedron viewer project](https://github.com/thallada/planet).
 ```javascript
 const drawShape = hexsphere => regl({
  vert: `
  precision mediump float;
  uniform mat4 projection, view;
  attribute vec3 position, normal, color;
  varying vec3 fragNormal, fragPosition, fragColor;
  void main() {
    fragNormal = normal;
    fragPosition = position;
    fragColor = color;
    gl_Position = projection * view * vec4(position, 1.0);
  }`,
  frag: `
  precision mediump float;
  struct Light {
    vec3 color;
    vec3 position;
  };
  uniform Light lights[1];
  varying vec3 fragNormal, fragPosition, fragColor;
  void main() {
    vec3 normal = normalize(fragNormal);
    vec3 light = vec3(0.1, 0.1, 0.1);
    for (int i = 0; i < 1; i++) {
      vec3 lightDir = normalize(lights[i].position - fragPosition);
      float diffuse = max(0.0, dot(lightDir, normal));
      light += diffuse * lights[i].color;
    }
    gl_FragColor = vec4(fragColor * light, 1.0);
  }`,
  attributes: {
    position: hexsphere.positions,
    normal: hexsphere.normals,
    color: hexsphere.colors,
  },
  elements: hexsphere.cells,
  uniforms: {
    "lights[0].color": [1, 1, 1],
    "lights[0].position": ({ tick }) => {
      const t = 0.008 * tick
      return [
        1000 * Math.cos(t),
        1000 * Math.sin(t),
        1000 * Math.sin(t)
      ]
    },
  },
 })
 ```
 I also imported [regl-camera](https://github.com/regl-project/regl-camera) which 
 handled all of the complex viewport code for me.
 It was fairly easy to get a simple renderer working quickly in regl, but I 
 couldn't find many examples of more complex projects using regl. Unfortunately, 
 the project looks a bit unmaintained these days as well. If I'm going to 
 continue with rendering in WebGL, I think I will try out 
 [Babylon.js](https://www.babylonjs.com/) instead.
 ### Running in WebAssembly
 Since rust can be compiled down to wasm and then run in the browser, I briefly 
 tried getting the project to run completely in the browser.
 The [wasm-pack](https://github.com/rustwasm/wasm-pack) tool made it pretty easy 
 to get started. My main struggle was figuring out an efficient way to get the 
 megabytes of generated shape data into the JavaScript context so it could be 
 rendered in WebGL.
 The best I could come up with was to export all of my structs into flat 
 `Vec<f32>`s and then create `Float32Array`s from the JS side that are views into 
 wasm's memory.
 To export:
 ```rust
 pub fn fill_exports(&mut self) {
    for position in &self.positions {
        self.export_positions.push(position.0.x);
        self.export_positions.push(position.0.y);
        self.export_positions.push(position.0.z);
    }
    for normal in &self.normals {
        self.export_normals.push(normal.0.x);
        self.export_normals.push(normal.0.y);
        self.export_normals.push(normal.0.z);
    }
    for color in &self.colors {
        self.export_colors.push(color.0.x);
        self.export_colors.push(color.0.y);
        self.export_colors.push(color.0.z);
    }
    for cell in &self.cells {
        self.export_cells.push(cell.a as u32);
        self.export_cells.push(cell.b as u32);
        self.export_cells.push(cell.c as u32);
    }
 }
 ```
 And then the wasm `lib.rs`:
 ```rust
 use byteorder::{LittleEndian, WriteBytesExt};
 use js_sys::{Array, Float32Array, Uint32Array};
 use wasm_bindgen::prelude::*;
 use web_sys::console;
 mod icosahedron;
 #[cfg(feature = "wee_alloc")]
 #[global_allocator]
 static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;
 #[wasm_bindgen(start)]
 pub fn main_js() -> Result<(), JsValue> {
    #[cfg(debug_assertions)]
    console_error_panic_hook::set_once();
    Ok(())
 }
 #[wasm_bindgen]
 pub struct Hexsphere {
    positions: Float32Array,
    normals: Float32Array,
    colors: Float32Array,
    cells: Uint32Array,
 }
 #[wasm_bindgen]
 pub fn shape_data() -> Result<Array, JsValue> {
    let radius = 1.0;
    let detail = 7;
    let mut hexsphere = icosahedron::Polyhedron::new_truncated_isocahedron(radius, detail);
    hexsphere.compute_triangle_normals();
    let mut unique_hexsphere = icosahedron::Polyhedron::new();
    unique_hexsphere.unique_vertices(hexsphere);
    unique_hexsphere.assign_random_face_colors();
    unique_hexsphere.fill_exports();
    let positions = unsafe { Float32Array::view(&unique_hexsphere.export_positions) };
    let normals = unsafe { Float32Array::view(&unique_hexsphere.export_normals) };
    let colors = unsafe { Float32Array::view(&unique_hexsphere.export_colors) };
    let cells = unsafe { Uint32Array::view(&unique_hexsphere.export_cells) };
    Ok(Array::of4(&positions, &normals, &colors, &cells))
 }
 ```
 With wasm-pack, I could import the wasm package, run the `shape_data()` 
 function, and then read the contents as any other normal JS array.
 ```javascript
 let rust = import("../pkg/index.js")
 rust.then(module => {
    const shapeData = module.shape_data()
    const shape = {
        positions: shapeData[0],
        normals: shapeData[1],
        colors: shapeData[2],
        cells: shapeData[3],
    }
    ...
 })
 ```
 I could side-step the issue of transferring data from Rust to JavaScript 
 entirely by programming literally everything in WebAssembly. But the bindings 
 from rust wasm to the WebGL API are still way too complicated compared to just 
 using regl. Plus, I'd have to implement my own camera from scratch.
 ### The Stats
 So how much faster is Rust than JavaScript in generating icosahedrons and 
 hexspheres?
 Here's how long it took with generating shapes in JS with Three.js in Firefox 
 versus in native Rust with a i5-2500K 3.3 GHz CPU.
 | Shape  | JS generate time | Rust generate time |
 |---|---|---|
 | Icosahedron detail 6 | 768 ms | 28.23 ms |
 | Icosahedron detail 7 | 4.25 s | 128.81 ms |
 | Hexsphere detail 6 | 11.37 s | 403.10 ms |
 | Hexsphere detail 7 | 25.49 s | 1.85 s |
 So much faster!
 ### Todo
 * Add a process that alters the shape post-generation. Part of the reason why I 
  decided to fan the hexagon faces with so many triangles is that it also allows 
  me to control the height of the faces better. This could eventually allow me 
  to create mountain ranges and river valleys on a hexsphere planet. Stretching 
  and pulling the edges of the polygon faces in random directions could add 
  variation and make for a more organic looking hexsphere.
 * Conversely, it would be nice to be able to run a process post-generation that 
  could reduce the number of triangles by tiling the hexagons more efficiently 
  when face elevation isn't needed.
 * Add parameters to the generation that allows generating sections of the 
  hexsphere / icosahedron. This will be essential for rendering very detailed 
  polyhedrons since at a certain detail level it becomes impossible to render 
  the entire shape at once.
  In WebGL, figure out what part of the shape is in the current viewport and 
  pass these parameters to the generation.
 * Render the shapes in a native Rust graphics library instead of WebGL. I'm 
  curious how much slower WebGL is making things.
 * Parallelize the generation. Right now the generation is very CPU bound and 
  each subdivide/truncate iteration is mostly independent from each other, so I 
  think I could get some decent speed-up by allowing the process to run on 
  multiple cores. Perhaps the [rayon](https://github.com/rayon-rs/rayon) crate 
  could make this pretty straightforward.
 * Find some way to avoid unique vertices. The size of the shape is *much* bigger 
  because of this. There might be a way to keep shared vertices while also 
  having a separate color per face by using texture mapping.
 * In the renderer, implement face selection (point and click face and show an 
  outline around selected face).
 * In the renderer, implement fly-to-face zooming: given a face, fly the camera 
  around the sphere in an orbit and then zoom in on the face.
--- a/_posts/2022-10-05-modmapper-putting-every-skyrim-mod-on-a-map-with-rust.md
+++ b/_posts/2022-10-05-modmapper-putting-every-skyrim-mod-on-a-map-with-rust.md
@ -1,686 +0,0 @@
 ---
 title: "Modmapper: Putting every Skyrim mod on a map with Rust"
 layout: post
 image: /img/blog/modmapper.jpg
 ---
 [Modmapper](https://modmapper.com) is a website that I made that puts every mod 
 for the game [Elder Scrolls V: 
 Skyrim](https://en.wikipedia.org/wiki/The_Elder_Scrolls_V:_Skyrim) uploaded to 
 [Nexus Mods](https://www.nexusmods.com/) on an interactive map.
 <a href="https://modmapper.com" target="_blank">
    ![Screenshot of modmapper.com](/img/blog/modmapper.jpg)
 </a>
 You can view the map at [https://modmapper.com](https://modmapper.com).
 Released in 2011, Skyrim is over a decade old now. But, its vast modding 
 community has kept it alive and relevant to this day. [Skyrim is still in the 
 top 50 games being played on Steam in 2022](https://steamcharts.com/top/p.2) and 
 I think it's no coincidence that [it's also one of the most modded games 
 ever](https://www.nexusmods.com/games?).
 <!--excerpt-->
 The enormous and enduring modding community around the Elder Scrolls games is 
 why I have a special fondness for the series. I was 13 when I first got 
 interested in programming through [making mods for Elder Scrolls IV: 
 Oblivion](https://www.nexusmods.com/users/512579?tab=user+files&BH=2). I quickly 
 realized I got way more satisfaction out of modding the game than actually 
 playing it. I was addicted to being able to create whatever my mind imagined in 
 my favorite game.
 I was working on mod for Skyrim earlier in the year[^bazaarrealm] and was 
 looking for the best places to put new buildings in the game world. I really 
 wanted areas of the game world off the beaten (heavily-modded) path. After over 
 a decade of modifications, there could be conflicts with hundreds of mods in any 
 area I chose which could cause issues like multiple buildings overlapping or 
 terrain changes causing floating rocks and trees.
 <p>
    <div class="row">
        <figure>
            <img alt="Example of a conflict between two mods that both chose the 
            same spot to put a lamp post and sign post so they are clipping" 
            src="/img/blog/modmapper-clipping-example2.jpg" />
            <figurecaption>
                <em>
                    Example of a conflict between two mods that both chose the same 
                    spot to put a lamp post and sign post so they are clipping. 
                    Screenshot by <a 
                    href="https://www.nexusmods.com/users/63732336">
                    AndreySG</a>.
                </em>
            </figurecaption>
        </figure>
        <figure>
            <img alt="Example of a conflict between two mods that both chose the 
            same spot to put a building and rock so they are clipping" 
            src="/img/blog/modmapper-clipping-example1.jpg" />
            <figurecaption>
                <em>
                    Conflict between a building and rock. Screenshot by <a 
                    href="https://www.reddit.com/user/LewdManoSaurus">
                    LewdManoSaurus</a>.
                </em>
            </figurecaption>
        </figure>
        <figure>
            <img alt="Example of a conflict between two mods that both chose the 
            same spot to put a building and tree so they are clipping" 
            src="/img/blog/modmapper-clipping-example3.jpg" />
            <figurecaption>
                <em>
                    Conflict between a building and a tree. Screenshot by <a 
                    href="https://www.nexusmods.com/skyrimspecialedition/users/51448566">
                    Janquel</a>.
                </em>
            </figurecaption>
        </figure>
        <figure>
            <img alt="Example of a conflict between two mods that both chose the 
            same spot to put a woodcutting mill" 
            src="/img/blog/modmapper-clipping-example4.jpg" />
            <figurecaption>
                <em>
                    Conflict between two woodcutting mills. Screenshot by <a 
                    href="https://www.nexusmods.com/skyrimspecialedition/users/51448566">
                    Janquel</a>.
                </em>
            </figurecaption>
        </figure>
    </div>
 </p>
 Mod authors usually use a tool like 
 [TES5Edit](https://www.nexusmods.com/skyrim/mods/25859) to analyze a group of 
 mod plugins to find conflicts and create patches to resolve them on a 
 case-by-case basis. But, I was unsatisfied with that. I wanted to be assured 
 that there would be no conflicts, or at least know the set of all possible mods 
 out there that could conflict so I could manually patch those few mods. There 
 was no good solution for finding conflicts across all mods though. Mod authors 
 would need to download every Skyrim mod ever and no one has time to download all 
 85,000+ Skyrim mods, and no one has the computer memory to load all of those in 
 TES5Edit at the same time.
 Through that frustration, Modmapper was born with the mission to create a 
 database of all Skyrim mod exterior cell edits. With that database I can power 
 the website which visualizes how popular cells are in aggregate as well as allow 
 the user to drill down to individual cells, mods, or plugins to find potential 
 conflicts without ever having to download files themselves.
 When I [released the website about 7 months 
 ago](https://www.reddit.com/r/skyrimmods/comments/sr8k4d/modmapper_over_14_million_cell_edits_from_every/) 
 it made a big splash in the Skyrim modding community. No one had ever visualized 
 mods on a map like this before, and it gave everyone a new perspective on the 
 vast library of Skyrim mods. It was even [featured on the front page of PC 
 Gamer's 
 website](https://www.pcgamer.com/skyrim-modmapper-is-a-weirdly-beautiful-way-to-manage-your-mods/). 
 Thirteen-year-old me, who regularly read the monthly PC Gamer magazine, would 
 have been astounded.
 <a 
 href="https://www.pcgamer.com/skyrim-modmapper-is-a-weirdly-beautiful-way-to-manage-your-mods/" 
 target="_blank">
    ![Screenshot of PC Gamer article titled "Skyrim Modmapper is a weirdly 
    beautiful way to manage your mods" by Robert Zak published April 20, 
    2022](/img/blog/modmapper-pcgamer.jpg)
 </a>
 The comments posted to the initial mod I posted on Nexus Mods[^takedown] for the 
 project were very amusing. It seemed to be blowing their minds:
 > "Quite possibly this could be the best mod for 
 Skyrim. This hands-down makes everyone's life easier to be able to see which of 
 their mods might be conflicting." -- [Nexus Mods comment by 
 lorddonk](/img/blog/modmapper-comment15.png)
 > "The 8th wonder of Skyrim. That's a Titan's work requiring a monk's 
 > perserverance. Finally, a place to go check (in)compatibilities !!! Voted. 
 > Endorsed." -- [Nexus Mods comment by 
 > jfjb2005](/img/blog/modmapper-comment3.png)
 > "They shall sing songs of your greatness! Wow, just wow." -- [Nexus Mods 
 > comment by 
 LumenMystic](/img/blog/modmapper-comment7.png)
 > "Holy Batman Tits! Be honest..... You're a Govt Agent and made this mod during 
 > your "Terrorist Watch Shift" using a CIA super computer.." -- [Nexus Mods 
 comment by toddrizzle](/img/blog/modmapper-comment1.png)
 > "What drugs are you on and can I have some?" -- [Nexus Mods comment by 
 > thappysnek](/img/blog/modmapper-comment11.png)
 > "This is madness! Author are some kind of overhuman?! GREAT work!"-- [Nexus 
 > Mods comment by TeodorWild](/img/blog/modmapper-comment10.png)
 > "You are an absolute legend. Bards will sing tales of your exploits" -- [Nexus 
 > Mods comment by burntwater](/img/blog/modmapper-comment2.png)
 > "I wanted to say something, but I'll just kneel before thee and worship. This 
 > would have taken me a lifetime. Amazing." -- [Nexus Mods comment by 
 > BlueGunk](/img/blog/modmapper-comment8.png)
 > "Finally found the real dragonborn" -- [Nexus Mods comment by 
 > yag1z](/img/blog/modmapper-comment6.png)
 > "he is the messiah!" -- [Nexus Mods comment by 
 > Cursedobjects](/img/blog/modmapper-comment12.png)
 > "A god amongst men." -- [Nexus Mods comment by 
 > TheMotherRobbit](/img/blog/modmapper-comment13.png)
 Apparently knowing how to program is now a god-like ability! This is the type of 
 feedback most programmers aspire to get from their users. I knew the tool was 
 neat and fun to build, but I didn't realize it was *that* sorely needed by the 
 community.
 Today, Modmapper has a sustained user-base of around 7.5k unique visitors a 
 month[^analytics] and I still see it mentioned in reddit threads or discord 
 servers whenever someone is asking about the places a mod edits or what mods 
 might be conflicting in a particular cell.
 The rest of this blog post will delve into how I built the website and how I 
 gathered all of the data necessary to display the visualization.
 ### Downloading ALL THE MODS!
 ![Meme with the title "DOWNLOAD ALL THE MODS!"](/img/blog/allthemods.jpg)
 In order for the project to work I needed to collect all the Skyrim mod plugin 
 files.
 While there are a number of places people upload Skyrim mods, [Nexus 
 Mods](https://nexusmods.com) is conveniently the most popular and has the vast 
 majority of mods. So, I would only need to deal with this one source. Luckily, 
 [they have a nice API 
 handy](https://app.swaggerhub.com/apis-docs/NexusMods/nexus-mods_public_api_params_in_form_data/1.0). 
 [modmapper](https://github.com/thallada/modmapper) is the project I created to 
 do this. It is a Rust binary that:
 * Uses [reqwest](https://crates.io/crates/reqwest) to make requests to [Nexus 
  Mods](https://nexusmods.com) for pages of last updated mods.
 * Uses [scraper](https://crates.io/crates/scraper) to scrape the HTML for 
  individual mod metadata (since the Nexus API doesn't provide an endpoint to 
  list mods).
 * Makes requests to the Nexus Mods API to get file and download information for 
  each mod, using [serde](https://serde.rs/) to parse the 
  [JSON](https://en.wikipedia.org/wiki/JSON) responses.
 * Requests the content preview data for each file and walks through the list of 
  files in the archive looking for a Skyrim plugin file (`.esp`, `.esm`, or 
  `.esl`).
 * If it finds a plugin, it decides to download the mod. It hits the download API 
  to get a download link and downloads the mod file archive.
 * Then it extracts the archive using one of: 
  [compress_tools](https://crates.io/crates/compress-tools), 
  [unrar](https://crates.io/crates/unrar), or [7zip](https://www.7-zip.org/) via 
  [`std::process::Commmand`](https://doc.rust-lang.org/std/process/struct.Command.html) 
  (depending on what type of archive it is).
 * With the ESP files (Elder Scrolls Plugin files) extracted, I then use my 
  [skyrim-cell-dump](https://github.com/thallada/skyrim-cell-dump) library (more 
  on that later!) to extract all of the cell edits into structured data.
 * Uses [seahash](https://crates.io/crates/seahash) to create a fast unique hash 
  for plugin files.
 * It then saves all of this data to a [postgres](https://www.postgresql.org/) 
  database using the [sqlx crate](https://crates.io/crates/sqlx).
 * Uses extensive logging with the [tracing 
  crate](https://crates.io/crates/tracing) so I can monitor the output and have 
  a history of a run to debug later if I discover an issue.
 It is designed to be run as a nightly [cron](https://en.wikipedia.org/wiki/Cron) 
 job which downloads mods that have updated on Nexus Mods since the last run.
 To keep costs for this project low, I decided to make the website entirely 
 static. So, instead of creating an API server that would have to be constantly 
 running to serve requests from the website by making queries directly to the 
 database, I would dump all of the data that the website needed from the database 
 to JSON files, then upload those files to [Amazon 
 S3](https://aws.amazon.com/s3/) and serve them through the [Cloudflare 
 CDN](https://www.cloudflare.com/cdn/) which has servers all over the world.
 So, for example, every mod in the database has a JSON file uploaded to 
 `https://mods.modmapper.com/skyrimspecialedition/<nexus_mod_id>.json` and the 
 website frontend will fetch that file when a user clicks a link to that mod in 
 the UI.
 The cost for S3 is pretty reasonable to me ($~3.5/month), and Cloudflare has a 
 [generous free tier](https://www.cloudflare.com/plans/#price-matrix) that allows 
 me to host everything through it for free.
 The server that I actually run `modmapper` on to download all of the mods is a 
 server I already have at home that I also use for other purposes. The output of 
 each run is uploaded to S3, and I also make a full backup of the database and 
 plugin files to [Dropbox](https://www.dropbox.com).
 A lot of people thought it was insane that I downloaded every mod[^adult-mods],
 but in reality it wasn't too bad once I got all the issues resolved in
 `modmapper`. I just let it run in the background all day and it would chug
 through the list of mods one-by-one. Most of the time ended up being spent
 waiting while the Nexus Mod's API hourly rate limit was reached on my
 account.[^rate-limit] 
 As a result of this project I believe I now have the most complete set of all 
 Skyrim plugins to date (extracted plugins only without other textures, models, 
 etc.)[^plugin-collection]. Compressed, it totals around 99 GB, uncompressed: 191 
 GB.
 [After I downloaded Skyrim Classic mods in addition to Skyrim Special 
 Edition](#finishing-the-collection-by-adding-all-skyrim-classic-mods), here are 
 some counts from the database:
 |:---|:---|
 | **Mods** | 113,028 |
 | **Files** | 330,487 |
 | **Plugins** | 534,831 |
 | **Plugin Cell Edits** | 33,464,556 |
 ### Parsing Skyrim plugin files
 The Skyrim game engine has a concept of 
 [worldspaces](https://en.uesp.net/wiki/Skyrim:Worldspaces) which are exterior 
 areas where the player can travel to. The biggest of these being, of course, 
 Skyrim itself (which, in the lore, is a province of the continent of 
 [Tamriel](https://en.uesp.net/wiki/Lore:Tamriel) on the planet 
 [Nirn](https://en.uesp.net/wiki/Lore:Nirn)). Worldspaces are recorded in a 
 plugin file as [WRLD 
 records](https://en.uesp.net/wiki/Skyrim_Mod:Mod_File_Format/WRLD).
 Worldspaces are then chunked up into a square grid of cells. The Skyrim 
 worldspace consists of a little over 11,000 square cells. Mods that make a 
 changes to the game world have a record in the plugin (a [CELL 
 record](https://en.uesp.net/wiki/Skyrim_Mod:Mod_File_Format/CELL)) with the 
 cell's X and Y coordinates and a list changes in that cell.
 There is some prior art ([esplugin](https://github.com/Ortham/esplugin), 
 [TES5Edit](https://github.com/TES5Edit/TES5Edit), 
 [zedit](https://github.com/z-edit/zedit)) of open-source programs that could 
 parse Skyrim plugins and extract this data. However, all of these were too broad 
 for my purpose or relied on the assumption of being run in the context of a load 
 order where the master files of a plugin would also be available. I wanted a 
 program that could take a single plugin in isolation and skip through all of the 
 non-relevant parts of it and dump just the CELL and WRLD record data plus some 
 metadata about the plugin from the header as fast as possible.
 After discovering [the wonderful documentation on the UESP wiki about the Skyrim 
 mod file format](https://en.uesp.net/wiki/Skyrim_Mod:Mod_File_Format), I 
 realized this would be something that would be possible to make myself. 
 [skyrim-cell-dump](https://github.com/thallada/skyrim-cell-dump) is a Rust 
 library/CLI program that accepts a Skyrim mod file and spits out the header 
 metadata of the plugin, the worlds edited/created, and all of the cells it 
 edits/creates.
 Under the hood, it uses the [nom crate](https://crates.io/crates/nom) to read 
 through the plugin until it finds the relevant records, then uses 
 [flate2](https://crates.io/crates/flate2) to decompress any compressed record 
 data, and finally outputs the extracted data formatted to JSON with 
 [serde](https://crates.io/crates/serde).
 Overall, I was pretty happy with this toolkit of tools and was able to quickly 
 get the data I needed from plugins. My only gripe was that I never quite figured 
 out how to properly do error handling with nom. If there was ever an error, I 
 didn't get much data in the error about what failed besides what function it 
 failed in. I often had to resort to peppering in a dozen `dbg!()` statements to 
 figure out what went wrong.
 I built it as both a library and binary crate so that I could import it in other 
 libraries and get the extracted data directly as Rust structs without needing to 
 go through JSON. I'll go more into why this was useful later.
 ### Building the website
 Since I wanted to keep server costs low and wanted the site to be as fast as 
 possible for users, I decided pretty early on that the site would be purely 
 static HTML and JavaScript with no backend server. I decided to use the [Next.js 
 web framework](https://nextjs.org/) with 
 [TypeScript](https://www.typescriptlang.org/) since it was what I was familiar 
 with using in my day job. While it does have [server-side rendering 
 support](https://nextjs.org/docs/basic-features/pages#server-side-rendering) 
 which would require running a backend [Node.js](https://nodejs.org/en/) server, 
 it also supports a limited feature-set that can be [exported as static 
 HTML](https://nextjs.org/docs/advanced-features/static-html-export).
 I host the site on [Cloudflare pages](https://pages.cloudflare.com/) which is 
 available on their free tier and made deploying from Github commits a 
 breeze[^cloudflare]. The web code is in my [modmapper-web 
 repo](https://github.com/thallada/modmapper-web).
 The most prominent feature of the website is the interactive satellite map of 
 Skyrim. Two essential resources made this map possible: [the map tile images 
 from the UESP skyrim map](https://srmap.uesp.net/) and 
 [Mapbox](https://www.mapbox.com/).
 [Mapbox provides a JS library for its WebGL 
 map](https://docs.mapbox.com/mapbox-gl-js/api/) which allows specifying a 
 [raster tile 
 source](https://docs.mapbox.com/mapbox-gl-js/example/map-tiles/)[^3d-terrain].
 The [UESP team painstakingly loaded every cell in the Skyrim worldspace in the 
 Creation Kit and took a 
 screenshot](https://en.uesp.net/wiki/UESPWiki:Skyrim_Map_Design). Once I figured 
 out which image tiles mapped to which in-game cell it was relatively easy to put 
 a map together by plugging them into the Mapbox map as a raster tile source.
 The heatmap overlaid on the map is created using a [Mapbox 
 layer](https://docs.mapbox.com/help/glossary/layer/) that fills a cell with a 
 color on a gradient from green to red depending on how many edits that cell has 
 across the whole database of mods.
 ![Screenshot closeup of modmapper.com displaying a grid of colored cells from 
 green to red overlaid atop a satellite map of 
 Skyrim](/img/blog/modmapper-heatmap-closeup.jpg)
 The sidebar on the site is created using [React](https://reactjs.org/) and 
 [Redux](https://redux.js.org/) and uses the 
 [next/router](https://nextjs.org/docs/api-reference/next/router) to keep track 
 of which page the user is on with URL parameters.
 <p>
    <div class="row">
        <img alt="Screenshot of modmapper.com sidebar with a cell selected" 
        src="/img/blog/modmapper-cell-sidebar.jpg" class="half-left" />
        <img alt="Screenshot of modmapper.com sidebar with a mod selected"
        src="/img/blog/modmapper-mod-sidebar.jpg" class="half-right" />
    </div>
 </p>
 The mod search is implemented using 
 [MiniSearch](https://lucaong.github.io/minisearch/) that asynchronously loads 
 the giant search indices for each game containing every mod name and id.
 ![Screenshot of modmapper.com with "trees" entered into the search bar with a 
 number of LE and SE mod results listed underneath in a 
 dropdown](/img/blog/modmapper-search.jpg)
 One of the newest features of the site allows users to drill down to a 
 particular plugin within a file of a mod and "Add" it to their list. All of the 
 added plugins will be listed in the sidebar and the cells they edit displayed in 
 purple outlines and conflicts between them displayed in red outlines.
 ![Screenshot of modmapper.com with 4 Added Plugins and the map covered in purple 
 and red boxes](/img/blog/modmapper-added-plugins.jpg)
 ### Loading plugins client-side with WebAssembly
 A feature that many users requested after the initial release was being able to 
 load a list of the mods currently installed on their game and see which ones of 
 that set conflict with each other[^second-announcement]. Implementing this 
 feature was one of the most interesting parts of the project. Choosing to use 
 Rust made made it possible, since everything I was running server-side to 
 extract the plugin data could also be done client-side in the browser with the 
 same Rust code compiled to [WebAssembly](https://webassembly.org/).
 I used [wasm-pack](https://github.com/rustwasm/wasm-pack) to create 
 [skyrim-cell-dump-wasm](https://github.com/thallada/skyrim-cell-dump-wasm/) 
 which exported the `parse_plugin` function from my 
 [skyrim-cell-dump](https://github.com/thallada/skyrim-cell-dump) Rust library 
 compiled to WebAssembly. It also exports a `hash_plugin` function that creates a 
 unique hash for a plugin file's slice of bytes using 
 [seahash](https://crates.io/crates/seahash) so the site can link plugins a user 
 has downloaded on their hard-drive to plugins that have been downloaded by 
 modmapper and saved in the database.
 Dragging-and-dropping the Skyrim Data folder on to the webpage or selecting the 
 folder in the "Open Skyrim Data directory" dialog kicks off a process that 
 starts parsing all of the plugin files in that directory in parallel using [Web 
 Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers).
 I developed my own 
 [`WorkerPool`](https://github.com/thallada/modmapper-web/blob/4af628559030c3f24618b29b46d4a40af2f200a6/lib/WorkerPool.ts) 
 that manages creating a pool of available workers and assigns them to plugins to 
 process. The pool size is the number of cores on the user's device so that the 
 site can process as many plugins in parallel as possible. After a plugin 
 finishes processing a plugin and sends the output to the redux store, it gets 
 added back to the pool and is then assigned a new plugin to process if there are 
 any[^wasm-troubles].
 Once all plugins have been loaded, the map updates by displaying all of the 
 cells edited in a purple box and any cells that are edited by more than one 
 plugin in a red box.
 ![Screenshot of modmapper.com with 74 Loaded Plugins and the map filled with 
 purple and red boxes](/img/blog/modmapper-loaded-plugins.jpg)
 Users can also drag-and-drop or paste their `plugins.txt` file, which is the 
 file that the game uses to define the load order of plugins and which plugins 
 are enabled or disabled. Adding the `plugins.txt` sorts the list of loaded 
 plugins in the sidebar in load order and enables or disables plugins as defined 
 in the `plugins.txt`.
 ![Screenshot of modmapper.com with the Paste plugins.txt dialog 
 open](/img/blog/modmapper-pluginstxt-dialog.jpg)
 Selecting a cell in the map will display all of the loaded cells that edit that 
 cell in the sidebar.
 ![Screenshot of modmapper.com with a conflicted cell selected on the map and 4 
 Loaded Plugins displayed](/img/blog/modmapper-conflicted-cell.jpg)
 The ability to load plugins straight from a user's hard-drive allows users to 
 map mods that haven't even been uploaded to Nexus Mods.
 ### Vortex integration
 The initial mod I released on the Skyrim Special Edition page of Nexus Mods was 
 [taken 
 down](https://www.reddit.com/r/skyrimmods/comments/svnz4a/modmapper_got_removed/) 
 by the site admins since it didn't contain an actual mod and they didn't agree 
 that it qualified as a "Utility".
 Determined to have an actual mod page for Modmapper on Nexus Mods, I decided to 
 make a [Vortex](https://www.nexusmods.com/about/vortex/) integration for 
 modmapper. Vortex is a mod manager made by the developers of Nexus Mods and they 
 allow creating extensions to the tool and have their own [mod section for Vortex 
 extensions](https://www.nexusmods.com/site).
 With the help of [Pickysaurus](https://www.nexusmods.com/skyrim/users/31179975), 
 one of the community managers for Nexus Mods, I created a [Vortex integration 
 for Modmapper](https://www.nexusmods.com/site/mods/371). It adds a context menu 
 option on mods to view the mod in Modmapper with all of the cells it edits 
 selected in purple. It also adds a button next to every plugin file to view just 
 that plugin in Modmapper (assuming it has been processed by Modmapper).
 <p>
    <div class="row">
        <img alt="Screenshot of Vortex mod list with a mod context menu open which 
        shows a 'See on Modmapper' option" 
        src="/img/blog/modmapper-vortex-mod-menu.jpg" class="half-left" />
        <img alt="Screenshot of Vortex plugin list with 'See on Modmapper' buttons 
        on the right of each plugin row"
        src="/img/blog/modmapper-vortex-plugin-button.jpg" class="half-right" />
    </div>
 </p>
 To enable the latter part, I had to include `skyrim-cell-dump-wasm` in the 
 extension so that I could hash the plugin contents with `seahash` to get the 
 same hash that Modmapper would have generated. It only does this hashing when 
 you click the "See on Modmapper" button to save from excessive CPU usage when 
 viewing the plugin list.
 After releasing the Vortex plugin, Pickysaurus [published a news article about 
 modmapper to the Skyrim Special Edition 
 site](https://www.nexusmods.com/skyrimspecialedition/news/14678) which also got 
 a lot of nice comments ❤️.
 ### Finishing the collection by adding all Skyrim Classic mods
 Skyrim is very silly in that it has [many 
 editions](https://ag.hyperxgaming.com/article/12043/every-skyrim-edition-released-over-the-last-decade). 
 But there was only one that split the modding universe into two: [Skyrim Special 
 Edition (SE)](https://en.uesp.net/wiki/Skyrim:Special_Edition).
 It was released in October 2016 with a revamped game engine that brought some 
 sorely needed graphical upgrades. However, it also contained changes to how mods 
 worked, requiring all mod authors to convert their mods to SE. This created big 
 chasm in the library of mods, and Nexus Mods had to make a separate section for 
 SE-only mods.
 When I started downloading mods in 2021, I started only with Skyrim SE mods, 
 which, at the time of writing, totals at over [55,000 mods on Nexus 
 Mods](https://www.nexusmods.com/skyrimspecialedition/mods/).
 After releasing with just SE mods, many users requested that all of the classic 
 pre-SE Skyrim mods be added as well. This month, I finally finished downloading 
 all Skyrim Classic mods, which, at the time of writing, totals at over [68,000 
 mods on Nexus Mods](https://www.nexusmods.com/skyrim/mods/). That brings the 
 total downloaded and processed mods for Modmapper at over 113,000 
 mods[^adult-mods]!
 ### The future
 A lot of users had great feedback and suggestions on what to add to the site. I 
 could only implement so many of them, though. The rest I've been keeping track 
 of on [this Trello board](https://trello.com/b/VdpTQ7ar/modmapper).
 Some of the headline items on it are:
 * Add [Solstheim map](https://dbmap.uesp.net/)
  Since map tiles images are available for that worldspace and because I have 
  already recorded edits to the worldspace in my database, it shouldn't be too 
  terribly difficult.
 * Add [Mod Organizer 2](https://www.modorganizer.org/) plugin
  Lots of people requested this since it's a very popular mod manager compared 
  to Vortex. MO2 supports python extensions so I created 
  [skyrim-cell-dump-py](https://github.com/thallada/skyrim-cell-dump-py) to 
  export the Rust plugin processing code to a Python library. I got a bit stuck 
  on actually creating the plugin though, so it might be a while until I get to 
  that.
 * Find a way to display interior cell edits on the map
  The map is currently missing edits to interior cells. Since almost all 
  interior cells in Skyrim have a link to the exterior world through a door 
  teleporter, it should be possible to map an interior cell edit to an exterior 
  cell on the map based on which cell the door leads out to.
  That will require digging much more into the plugin files for more data, and 
  city worldspaces will complicate things further. Then there's the question of 
  interiors with multiple doors to different exterior cells, or interior cells 
  nested recursively deep within many other interior cells.
 * Create a standalone [Electron](https://www.electronjs.org) app that can run 
  outside the browser
  I think this would solve a lot of the issues I ran into while developing the 
  website. Since Electron has a Node.js process running on the user's computer 
  outside the sandboxed browser process, it gives me much more flexibility. It 
  could do things like automatically load a user's plugin files. Or just load 
  plugins at all wihtout having to deal with the annoying dialog that lies to 
  the user saying they're about to upload their entire Data folder hundreds of 
  gigabytes full of files to a server (I really wish the 
  [HTMLInputElement.webkitdirectory](https://developer.mozilla.org/en-US/docs/Web/API/HTMLInputElement/webkitdirectory) 
  API would use the same underlying code as the [HTML Drag and Drop 
  API](https://developer.mozilla.org/en-US/docs/Web/API/HTML_Drag_and_Drop_API) 
  which is a lot better).
 * Improving the search 
  The mod search feature struggles the most with the static generated nature of 
  the site. I found it very hard to pack all of the necessary info for the 
  search index for all 100k+ mods (index for both SE and LE is around 6 MB). 
  Asynchronously loading the indices with MiniSearch keeps it from freezing up 
  the browser, but it does take a very long time to fully load. I can't help 
  think that there's a better way to shard the indices somehow and only fetch 
  what I need based on what the user is typing into the search.
 To be clear, a lot of the Todos on the board are pipe-dreams. I may never get to 
 them. This project is sustained purely by my motivation and self-interests and 
 if something is too much of a pain to get working I'll just drop it.
 There will also be future Elder Scrolls games, and [future Bethesda games based 
 on roughly the same game engine](https://bethesda.net/en/game/starfield). It 
 would be neat to create similar database for those games as the modding 
 community develops in realtime.
 Overall, I'm glad I made something of use to the modding community. I hope to 
 keep the site running for as long as people are modding Skyrim (until the 
 heat-death of the universe, probably).
 <br>
 --- 
 #### Footnotes
 [^bazaarrealm]:
    Unfortunately, I basically lost interest on the mod after working on 
    Modmapper. I might still write a blog post about it eventually since I did a 
    lot of interesting hacking on the Skyrim game engine to try to add some 
    asynchronous multiplayer aspects. [Project is here if anyone is curious in 
    the meantime](https://github.com/thallada/BazaarRealmPlugin).
 [^takedown]:
    I sadly only have screenshots for some of the comments on that mod since it 
    was eventually taken down by the Nexus Mod admins. See explanation about 
    that in the [Vortex integration section](#vortex-integration).
 [^analytics]:
    As recorded by Cloudflare's server side analytics, which may record a fair 
    amount of bot traffic. I suspect this is the most accurate number I can get 
    since most of my users probably use an ad blocker that blocks client-side 
    analytics.
 [^adult-mods]:
    Every mod on Nexus Mods except for adult mods since the site restricts
    viewing adult mods to only logged-in users and I wasn't able to get my
    scraping bot to log in as a user.
 [^rate-limit]:
    Apparently my mass-downloading did not go unnoticed by the Nexus Mod admins. 
    I think it's technically against their terms of service to automatically 
    download mods, but I somehow got on their good side and was spared the 
    ban-hammer. I don't recommend anyone else run modmapper themselves on the 
    entire site unless you talk to the admins beforehand and get the okay from 
    them.
 [^plugin-collection]:
    If you would like access to this dataset of plugins to do some data-mining 
    please reach out to me at [tyler@hallada.net](mailto:tyler@hallada.net) 
    (Note: only contains plugins files, no models, textures, audio, etc.). I 
    don't plan on releasing it publicly since that would surely go against many 
    mod authors' wishes/licenses.
 [^cloudflare]:
    I'm not sure I want to recommend anyone else use Cloudflare after [the whole 
    Kiwi Farms 
    debacle](https://www.theverge.com/2022/9/6/23339889/cloudflare-kiwi-farms-content-moderation-ddos). 
    I now regret having invested so much of the infrastructure in them. However, 
    I'm only using their free-tier, so at least I am a net-negative for their 
    business? I would recommend others look into 
    [Netlify](https://www.netlify.com/) or [fastly](https://www.fastly.com/) for 
    similar offerings to Cloudflare pages/CDN.
 [^3d-terrain]:
    I also tried to add a [raster Terrain-DEM 
    source](https://docs.mapbox.com/data/tilesets/reference/mapbox-terrain-dem-v1/) 
    for rendering the terrain in 3D. I got fairly far [generating my own DEM RGB 
    tiles](https://github.com/syncpoint/terrain-rgb) from an upscaled [greyscale 
    heightmap](https://i.imgur.com/9RErBDo.png) [constructed from the LAND 
    records in Skyrim.esm](https://www.nexusmods.com/skyrim/mods/80692) (view it 
    [here](https://www.dropbox.com/s/56lffk021riil6h/heightmap-4x_foolhardy_Remacri_rgb.tif?dl=0)). 
    But, it came out all wrong: [giant cliffs in the middle of the 
    map](/img/blog/modmapper-terrain-cliff.jpg) and [tiny spiky lumps with big 
    jumps in elevation at cell boundaries](/img/blog/modmapper-bad-terrain.jpg). 
    Seemed like too much work to get right than it was worth it.
 [^second-announcement]:
    [This was the announcement I posted to /r/skyrimmods for this feature](
 https://www.reddit.com/r/skyrimmods/comments/ti3gjh/modmapper_update_load_plugins_in_your_load_order/)
 [^wasm-troubles]:
    At first, I noticed a strange issue with re-using the same worker on 
    different plugins multiple times. After a while (~30 reuses per worker), the 
    processing would slow to a crawl and eventually strange things started 
    happening (I was listening to music in my browser and it started to pop and 
    crack). It seemed like the speed of processing increased exponentially to 
    the number of times the worker was reused. So, to avoid this, I had to make 
    the worker pool terminate and recreate workers after every plugin processed. 
    This ended up not being as slow as it sounds and worked fine. However, I 
    recently discovered that [wee_alloc, the most suggested allocator to use 
    with rust in wasm, has a memory leak and is mostly unmaintained 
    now](https://www.reddit.com/r/rust/comments/x1cle0/dont_use_wee_alloc_in_production_code_targeting/). 
    I switched to the default allocator and I didn't run into the exponentially 
    slow re-use problem. For some reason, the first run on a fresh tab is always 
    much faster than the second run, but subsequent runs are still fairly stable 
    in processing time.
--- a/assets/avenue_poplars.jpg
+++ b/assets/avenue_poplars.jpg
--- a/assets/blurry_clouds.png
+++ b/assets/blurry_clouds.png
--- a/assets/blurry_clouds2.png
+++ b/assets/blurry_clouds2.png
--- a/assets/boston_aerial.jpg
+++ b/assets/boston_aerial.jpg
--- a/assets/boston_skyline.jpg
+++ b/assets/boston_skyline.jpg
--- a/assets/copenhagen_park.jpg
+++ b/assets/copenhagen_park.jpg
--- a/assets/cyrodiil_ingame.jpg
+++ b/assets/cyrodiil_ingame.jpg
--- a/assets/cyrodiil_satellite_hex_terrain.png
+++ b/assets/cyrodiil_satellite_hex_terrain.png
--- a/assets/cyrodiil_terrain1.png
+++ b/assets/cyrodiil_terrain1.png
--- a/assets/dead_forest_sitting.png
+++ b/assets/dead_forest_sitting.png
--- a/assets/dead_forest_standing.png
+++ b/assets/dead_forest_standing.png
--- a/assets/desert_satellite.jpg
+++ b/assets/desert_satellite.jpg
--- a/assets/fallout4_map.png
+++ b/assets/fallout4_map.png
--- a/assets/forest_hill.jpg
+++ b/assets/forest_hill.jpg
--- a/assets/forrest_autumn.jpg
+++ b/assets/forrest_autumn.jpg
--- a/assets/halo_ring_mountains.jpg
+++ b/assets/halo_ring_mountains.jpg
--- a/assets/halo_shenandoah.png
+++ b/assets/halo_shenandoah.png
--- a/assets/head_lioness.jpg
+++ b/assets/head_lioness.jpg
--- a/assets/jungle_nyc.png
+++ b/assets/jungle_nyc.png
--- a/assets/kaelan_hex_terrain.jpg
+++ b/assets/kaelan_hex_terrain.jpg
--- a/assets/kaelan_terrain1.png
+++ b/assets/kaelan_terrain1.png
--- a/assets/kaelan_terrain2.jpg
+++ b/assets/kaelan_terrain2.jpg
--- a/assets/kaelan_terrain3.jpg
+++ b/assets/kaelan_terrain3.jpg
--- a/assets/kowloon.jpg
+++ b/assets/kowloon.jpg
--- a/assets/kowloon_ngs.png
+++ b/assets/kowloon_ngs.png
--- a/assets/lion_portrait.png
+++ b/assets/lion_portrait.png
--- a/assets/me.png
+++ b/assets/me.png
--- a/assets/mexico_city.jpg
+++ b/assets/mexico_city.jpg
--- a/assets/midnight_screenshot_inverted.png
+++ b/assets/midnight_screenshot_inverted.png
--- a/assets/midnight_screenshot_redshift.png
+++ b/assets/midnight_screenshot_redshift.png
--- a/assets/minecraft_map.jpg
+++ b/assets/minecraft_map.jpg
--- a/assets/mountain_brook.jpg
+++ b/assets/mountain_brook.jpg
--- a/assets/mountain_brook_hill.png
+++ b/assets/mountain_brook_hill.png
--- a/assets/mountain_view.jpg
+++ b/assets/mountain_view.jpg
--- a/assets/mountain_view_pixels.png
+++ b/assets/mountain_view_pixels.png
--- a/assets/ngs_map.jpg
+++ b/assets/ngs_map.jpg
--- a/assets/nyc.jpg
+++ b/assets/nyc.jpg
--- a/assets/pixels.png
+++ b/assets/pixels.png
--- a/assets/pockets.jpg
+++ b/assets/pockets.jpg
--- a/assets/pockets_portrait.png
+++ b/assets/pockets_portrait.png
--- a/assets/poplars.png
+++ b/assets/poplars.png
--- a/assets/rainforest.jpg
+++ b/assets/rainforest.jpg
--- a/assets/random_mexico_city.png
+++ b/assets/random_mexico_city.png
--- a/assets/random_treatment_plant.png
+++ b/assets/random_treatment_plant.png
--- a/assets/river_satellite.png
+++ b/assets/river_satellite.png
--- a/assets/satellite_fallout4_map.png
+++ b/assets/satellite_fallout4_map.png
--- a/assets/satellite_hex_terrain.png
+++ b/assets/satellite_hex_terrain.png
--- a/assets/satellite_minecraft_map.png
+++ b/assets/satellite_minecraft_map.png
--- a/assets/satellite_terrain1_process.png
+++ b/assets/satellite_terrain1_process.png
--- a/assets/satellite_terrain2.png
+++ b/assets/satellite_terrain2.png
--- a/assets/satellite_terrain3.png
+++ b/assets/satellite_terrain3.png
--- a/assets/shenandoah_mountains.jpg
+++ b/assets/shenandoah_mountains.jpg
--- a/assets/side_portrait.jpg
+++ b/assets/side_portrait.jpg
--- a/assets/sitting_forest.jpg
+++ b/assets/sitting_forest.jpg
--- a/assets/stained_glass.jpg
+++ b/assets/stained_glass.jpg
--- a/assets/stained_glass_portrait.png
+++ b/assets/stained_glass_portrait.png
--- a/assets/standing_forest.jpg
+++ b/assets/standing_forest.jpg
--- a/assets/starry_boston.png
+++ b/assets/starry_boston.png
--- a/assets/starry_night.jpg
+++ b/assets/starry_night.jpg
--- a/assets/svalbard_satellite.jpg
+++ b/assets/svalbard_satellite.jpg
--- a/assets/treatment_plant.jpg
+++ b/assets/treatment_plant.jpg
--- a/assets/uk_satellite.jpg
+++ b/assets/uk_satellite.jpg
--- a/assets/volcano_satellite.jpg
+++ b/assets/volcano_satellite.jpg
--- a/blog/index.html
+++ b/blog/index.html
@ -4,42 +4,40 @@ title: Tyler Hallada - Blog
 ---
 {% for post in paginator.posts %}
-  {% if post.hidden == null or post.hidden == false %}
+  <div class="card">
-    <div class="card">
+    <div class="row clearfix post-header">
-      <div class="row clearfix">
+      <div class="column three-fourths">
-        <div class="column full post-header">
+        <a href="{{ post.url }}"><h2 class="post-title">{{ post.title }}</h2></a>
          <h2 class="post-title"><a href="{{ post.url }}">{{ post.title }}</a></h2>
          <span class="timestamp">{{ post.date | date_to_string }}</span>
        </div>
      </div>
-      <div class="row clearfix">
+      <div class="column fourth">
-        <div class="column full post">
+        <span class="timestamp">{{ post.date | date_to_string }}</span> 
          {{ post.excerpt }}
        </div>
      </div>
      <div class="row clearfix more-row">
        <div class="column full">
          <a href="{{ post.url }}" class="read-more">Read More &raquo;</a>
        </div>
      </div>
    </div>
-  {% endif %}
+    <div class="row clearfix">
      <div class="column full post">
        {{ post.excerpt }}
      </div>
    </div>
    <div class="row clearfix more-row">
      <div class="column full">
        <a href="{{ post.url }}" class="read-more">Read More &raquo;</a>
      </div>
    </div>
  </div>
 {% endfor %}
 <div class="row clearfix pagination">
  <div class="column full">
    {% if paginator.previous_page %}
      {% if paginator.page == 2 %}
-        <a href="/blog/" class="previous">&laquo; Previous</a>
+        <a href="/blog" class="previous">&laquo; Previous</a>
      {% else %}
-        <a href="/blog/page{{ paginator.previous_page }}/" class="previous">&laquo; Previous</a>
+        <a href="/blog/page{{ paginator.previous_page }}" class="previous">&laquo; Previous</a>
      {% endif %}
    {% endif %}
    <span class="page_number ">Page {{ paginator.page }} of {{ paginator.total_pages }}</span>
    {% if paginator.next_page %}
-      <a href="/blog/page{{ paginator.next_page }}/" class="next">Next &raquo;</a>
+      <a href="/blog/page{{ paginator.next_page }}" class="next">Next &raquo;</a>
    {% endif %}
  </div>
 </div>
 {% include mail-form.html %}
--- a/css/main.css
+++ b/css/main.css
@ -13,21 +13,12 @@
 }
 html {
-  background-color: whitesmoke;
+  width: 100%;
-  margin: 0;
+  height: 100%;
-  padding: 0;
+  background: whitesmoke;
 }
 body {
  margin: 0;
  padding: 0;
  width: 100%;
  height: 100%;
  min-height: 100vh;
  background-color: whitesmoke;
 }
 .root {
  font-family: 'Open Sans', Arial, sans-serif;
  font-style: normal;
  font-weight: 400;
@ -48,39 +39,9 @@ a {
 }
 p {
-  margin: 0 0 12px;
+  margin: 0 0 10px;
  font-size: 1rem;
  line-height: 1.45rem;
 }
 img + em {
  display: block;
  margin-top: 4px;
 }
 li {
  margin-bottom: 12px;
 }
 table {
  min-width: 85%;
  margin: 1em auto 1em;
  border: 1px solid #cbcbcb;
 }
 td, th {
  padding: 0.5em 0.5em;
 }
 td {
  border: 1px solid #cbcbcb;
 }
 tr:nth-child(2n-1) td {
  background-color: #f2f2f2;
 }
 a:visited {
  color:#855C85;
 }
@ -128,14 +89,10 @@ blockquote p {
 .container {
  margin: 0 auto;
  max-width: 48rem;
-  width: 100%;
+  width: 90%;
 }
@media (min-width: 40rem) {
  .container {
    width: 90%;
  }
  .column {
    float: left;
    padding-left: 1rem;
@ -148,8 +105,6 @@ blockquote p {
  .column.third { width: 33.3%; }
  .column.fourth { width: 25%; }
  .column.three-fourths { width: 75%; }
  .column.fifth { width: 20%; }
  .column.four-fifths { width: 80%; }
  .column.flow-opposite { float: right; }  
 }
@ -203,15 +158,17 @@ img { width: auto; max-width: 100%; height: auto; }
 .hide-mobile {
  display: none;
 }
-.hide-desktop-inline-block { display: inline-block }
+.hide-desktop-inline-block {
-.hide-desktop-block { display: block }
+  display: inline-block;
-.hide-desktop-inline { display: inline }
+}
 .hide-desktop-block {
  display: block;
 }
@media (min-width: 40rem) {
  .hide-desktop { display: none }
  .hide-mobile-inline-block { display: inline-block }
  .hide-mobile-block { display: block }
  .hide-mobile-inline { display: inline }
 }
 /*****************************************************************************/
@ -226,12 +183,10 @@ h1.title {
  font-size: 2rem;
  font-weight: 200;
  margin: 0;
  margin-left: 0.5rem;
 }
 div.header {
-  /* this is padding instead of margin to prevent <html> from poking out behind top of page */
+  margin: 20px 0 20px;
  padding: 20px 0 20px;
  text-align: center;
 }
@ -246,18 +201,18 @@ div.header a {
 }
 span.timestamp {
  display: block;
  font-size: 0.85rem;
-  margin-top: 8px;
+  margin-top: 7px;
 }
@media (min-width: 40rem) {
  span.timestamp {
    float: right;
  }
 }
 .post-header {
-  padding-bottom: 24px;
+  padding-bottom: 10px;
  display: flex;
  align-items: start;
  justify-content: space-between;
  column-gap: 12px;
  flex-wrap: wrap;
 }
 a.read-more {
@ -287,7 +242,7 @@ div.more-row {
 div.pagination {
  text-align: center;
-  margin-bottom: 20px;
+  margin-bottom: 10px;
 }
 div.rss {
@ -362,11 +317,6 @@ a.rss img {
  background-color: #333;
 }
 .post img {
  display: block;
  margin: 0 auto;
 }
 .post img.half-left {
  float: none;
  width: 100%;
@ -390,58 +340,6 @@ a.rss img {
  }
 }
 .post .row {
  display: flex;
  align: center;
  justify-content: center;
  gap: 8px 8px;
  flex-wrap: wrap;
 }
 .post .row figure {
  flex-basis: calc(50% - 8px);
 }
@media (max-width: 40rem) {
  .post .row {
    flex-direction: column;
  }
 }
 .post figure {
  margin: 0;
 }
 .post figure figurecaption {
  display: block;
  margin-top: 4px;
 }
 /*****************************************************************************/
 /*
 /* Subscribe form
 /*
 /*****************************************************************************/
 .subscribe-form h3 {
  margin-top: 10px;
  margin-left: 10px;
 }
 .subscribe-form input {
  margin: 10px;
 }
 .subscribe-form label {
  margin-left: 10px;
 }
 .subscribe-form span.form-rss {
  display: block;
  margin-top: 20px;
  margin-left: 10px;
 }
 /*****************************************************************************/
 /*
 /* Homepage
@ -464,6 +362,7 @@ a.rss img {
  background: white;
  margin: 0 0.5rem 1rem;
  border-radius: 3px;
  user-select: none;
 }
 h1.big-name {
@ -594,42 +493,3 @@ div.options-panel form {
 div.options-panel form label {
  display: block;
 }
 /*****************************************************************************/
 /*
 /* Comments (isso)
 /*
 /*****************************************************************************/
 .isso-postbox .textarea-wrapper {
  margin-top: 10px;
  margin-bottom: 10px;
 }
 div.isso-postbox div.form-wrapper section.auth-section p.input-wrapper {
  margin-right: 10px;
 }
 /*****************************************************************************/
 /*
 /* Light & Dark Theme Toggle
 /*
 /*****************************************************************************/
 div.theme-toggle {
  position: static;
  bottom: 0;
  width: 100%;
  text-align: center;
  z-index: 2;
  padding: 0.25rem 0.75rem;
 }
@media (min-width: 40rem) {
  div.theme-toggle {
    position: absolute;
    top: 0;
    right: 0;
    width: inherit;
  }
 }
--- a/css/main_dark.css
+++ b/css/main_dark.css
@ -1,40 +0,0 @@
 /* Dark theme */
 html {
  filter: invert(90%);
  background-color: rgba(10, 10, 10, 0.9);
 }
 img:not(.icon), video, div.video-container {
  filter: invert(100%);
 }
 img:not(.icon) {
  opacity: .90;
  transition: opacity .5s ease-in-out;
 }
 img:hover:not(.icon) {
  opacity: 1;
 }
 a:not(.card), a:hover:not(.card) {
  filter: invert(100%);
 }
 a:not(.card) img:not(.icon) {
  filter: none;
 }
 .card {
  box-shadow: 0 1px 2px #fff !important;
 }
 .post pre {
  background-color: #fff !important;
 }
 .post a code {
  background-color: #222 !important;
  border: 1px solid #333 !important;
 }
--- a/Show More
+++ b/Show More