February 5, 2018

Image Alignment and Color Compositing

This post is about aligning images with Python.

We will be aligning images from Sergei Mikhailovich Prokudin-Gorskii(1863-1944). He took photographs at three exposures using a glass plate filters in order to separate the photograph into three color channels(red, green, and blue). The photographs are online at the library of congress.

First Approach: Brute Force #

This gif is an example of what what were are attempting to do in photoshop.
Alignment

Pseudocode

def align(channel_to_align, image_channel):
    for i in range(-15, 15):
        for j in range(-15, 15):
            # Move the image and score it
     return aligned_image

Results #

Smaller Images #

This example works well on smaller images.

1024px x 398px
Original

Unaligned Image

Aligned Image

Larger Images #

There is very little difference to the unaligned image when using the same algorithm. The reason being, as images become larger the less of an impact our search has on the image as whole. Instead of aligning it in a -15, 15 pixel area we need to scale this with the size of the image in order to get similar results. The issue we run into is that speed becomes a major factor when working with large images.

When scaling down the image the strategy still works well.

Improving the Algorithm #

Image Pyramid #

Image pyramid help us scale down the image so we can analyze an image at multiple scales. Image Pyramid wiki

In order to create one in python we need two things.

Blurring
Image Reduction

The blur I chose to use is a Gaussian blur.

Creating a gaussian kernel. #

Gaussian Function Wiki

Using the equation for a 1 dimensional gaussian function you can create a gaussian kernel with your desired length and sigma.

For this I used a length=5 and sigma=1
[ 0.05399097 0.24197072 0.39894228 0.24197072 0.05399097]

Next in order to reduce the image you can choose every row and column to reduce the image by half after the blur.

Gaussian Blur #

Blurred

Unblurred

Using the image pyramid #

When using the image pyramid, I decide on the number of levels to generate based on this equation.

 levels = int(np.floor(np.log2(np.sqrt(number of rows + columns+ 1)/125)) + 1))

Then I go to the highest level of pyramid, then I use the brute force solution. This gives a relative position of where to search when going down the image pyramid which speeds up the image alignment.