Call to Action for Non-Overlapping Image Stitching

Searching for an automation tool or usable algorithm.

Posted by Vance Feldman on December 3, 2014

Have a software suggestion?

Or are you a CS student looking for a thesis topic?

I’ve been searching (for several years) for a way to programmatically stitch non-overlapping images of the ForeverScape. So far I’ve turned up nothing viable (that would reduce manual labor time). Image stitching is essential to assembling images for printing, the website and eventual mobile app. I often find myself with a stack of 60 pages to scan and hand-crop and save-out, a task that can take 8-16 hours that otherwise could be used drawing. Next, high quality stitches can take another 8-12 hours for a 30 page segment. That’s a couple days of mindless work for only a little output. Doing this work by hand is painstaking and imperfect, the quality goes down as I rush the job.

Post Sample Image It is most noticeable in places with low values or solid hues, and not as much in areas with higher levels of detail or dark colors.

I often find bad stitches where either too much paper edge is visible, or where the image is cropped slightly. For an example, check out the beta site where you can really zoom in to see the detail in the paper. Now you’ve seen what a poor job I’ve done… Now consider the fact that there are years and years of drawings that demand re-scanning and cropping. I need non-overlapping automated boundary rectification and stitching like Van Gogh needed his older brother to pay for his paintbrushes and rent.

The Stitching Challenge:

1) Almost every existing commercial solution requires overlapping images. The algorithms for feature descriptors simply look for overlaps (positives). Feature detection and alignment stops at the edge. What is really needed is a way to project features into adjescent areas. If you can project “predicted” features outward, you could then look for matches in the next image. I’ve used some OpenCV libraries to detect edges, and I’ve heard of “fill in the missing part” algorithms for inpainting, but I’ve not seen the two combined in image-synth applications where there’s poor overlapping data points.

2) Warping: due to disturbances in the paper’s surface, some warping may occur and not all objects perfectly align along the edge due to how the paper warped (back again) when placed under the scanner cover. You can see this in the example above, where objects align better on the left than on the right. Some amount of de-warping (a distortion mesh) might help in this case.

I recently found a promising paper implying the possibility entitled Alignment and mosaicing of non-overlapping images. However, I do not see any over-the-counter commercial products or open source projects that can accomplish this with ease and accuracy. This is where I’m calling on you!

Does anyone know of tools? Please comment or tweet me @ForeverScape!

The Cropping Challenge:

1) It takes a lot of time to perfectly crop, paper edges align with edge of scanner (which is used as horizontal alignment calibration).

2) Arbitrarily rotated images when manually cropped are imperfect. You have to manually crop every edge instead of relying on the corner.

3) Edges are gritty: paper’s edges have miniscule (but visible) defects that manifest as gray lines when scanned.

Potential Cropping Solution: Multicrop. While intended for cropping multiple images from a single one, it may prove useful for cropping single images. It will take some time getting up and running with this, it may require some additional scripting to do this in batch.