Mosaicking
It is often desirable to combine the output of multi-camera systems into a single image, a mosaic. Such image aligns the different cameras into a common coordinate system and provides the "big picture" of the captured scene. Accurate and efficient estimate of this mosaic is necessary, as this is often the first step in a computer vision or image processing pipeline.
Our Solution
Our mosaicking algorithm estimates mosaics with unparalleled accuracy and efficiency, with only a minimal overlap between cameras needed. It achieves this by performing joint optimization of the mosaic parameters, handling lens or other camera distortions, and judiciously choosing the constraints in the objective function for maximum efficiency. Furthermore, it minimizes photometric differences between cameras to ensure a seamless render of the mosaic.
Joint Optimization
Optimizing all the parameters at once allows for quick convergence. Moreover, it enables reaching the global optimum of the objective, provided a good parameter initialization.
Camera Distortion Model
Our camera distortion model, based on the piecewise affine deformation of an image, is able to handle arbitrary non-linear distortions in images. Moreover, the change in the deformation with respect to its parameters can be expressed analytically, allowing an easy integration of the deformation model in the joint optimization and quick convergence.
Optimization Constraints
The joint optimization of the parameters can use feature correspondences or intensity variance of the overlap region (brightness constancy) as constraints. Intensity variance constraints are sampled sparsely, yet ensuring maximum effect on the optimization. An efficient context-sensitive validation of feature correspondences removes nearly all false matches.
Seamless Render
Brightness differences across cameras are compensated by modeling these differences with a simple, yet effective, parametric model and solving for a jointly optimal brightness correction that minimizes them. Any remaining photometric differences virtually disappear with a blending of the overlap regions.
Results
Our mosaicking solution is able to efficiently generate mosaics for the entire CLIF2006 and WPAFB2009 datasets. The estimated parameters for these datasets are available for download and can be used for mosaic rendering with the provided demonstration program.
Mosaic Parameters
Demonstration Program
Download (1.2 MiB)
Linux x86-64 (Red Hat 7 or later, Ubuntu 14.04 or later)
Video Demonstration
Licensing
Our mosaicking solution is available for commercial licensing. Please contact us at info@jgradient.com for a customized offer or any additional information.