The filter invert style was causing weird issues. Instead, I computed what the colors should be inverted 90% and set them absolutely instead.
