3.4. Homographies

Homography

A homography (or projective transformation) is a matrix $H\in {\mathbb{R}}^{3\times 3}$ that maps points between two planes in homogeneous coordinates, such that $p^{\prime }=Hp$, where
$H=\left[\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& 1\end{array}\right].$
Since $H$ is defined up to scale (i.e., $H\equiv sH$), $h_{33}$ is conventionally set to $1$ to remove scale ambiguity. Therefore, any $H$ has 8 degrees of freedom.

Homographies capture all affine transformations with the addition of perspective distortion, which makes objects appear smaller when farther away and larger when closer and is not affine.

In practice, the exact transformation between two images (or between a scene plane and the image) is unknown. We estimate $H$ using point correspondences between the two planes, taking at least 4 correspondences to satisfy the 8 unknowns. To estimate $H$, methods such as least squares and SVD can be used by solving $Ah=b$, where $h\in {\mathbb{R}}^9$ is the vectorized homography matrix. Note that $H$ is the right singular vector of $A$.

Common applications of $H$ include:

• Rectification: Warp an image so that a planar surface appears fronto-parallel (remove perspective distortion).
• Inverse Rectification: Apply $H^{-1}$ to map rectified coordinates back to the original image (useful for overlaying or texture mapping).
• Image Mosaicing: Stitch multiple overlapping images into a panorama by warping each image into a common coordinate frame using estimated homographies.

Intuition

A homography is the most general transformation between two images of the same plane — and, surprisingly, also between two images taken from the same camera center (pure rotation, no translation). In both cases there is no parallax, and a $3\times 3$ projective map fully accounts for the geometric relationship.

The key geometric fact is that homographies preserve straight lines but not parallelism or angles. A straight edge in one image stays straight in the warped image, but a pair of rails that were parallel in the world converge to a vanishing point. This is exactly the phenomenon of perspective — and inverting it (rectification) is the primary use of homographies outside of stitching.

Properties

(Degrees of freedom) A homography has 8 DoF: 9 matrix entries minus 1 for the scale ambiguity.

Equivalently, $H$ is defined up to scalar multiplication, and fixing $h_{33}=1$ (or $\Vert H{\Vert }_F=1$) removes the ambiguity. Four point correspondences give 8 linear constraints and determine $H$ uniquely.

(Applicable cases) A homography exactly relates two images iff (a) the scene is planar, or (b) the camera rotates purely about its optical center.

In any other case (a 3D scene viewed from distinct optical centers), a single homography cannot align the entire image — the fundamental matrix is needed. For mosaicing, condition (b) is typically arranged by having the photographer rotate in place.

(Hierarchy of transformations) Identity ⊂ Translation ⊂ Rigid ⊂ Similarity ⊂ Affine ⊂ Homography.

Each step adds degrees of freedom. The leap from affine (6 DoF) to homography (8 DoF) adds perspective foreshortening — the two extra parameters control how quickly lines converge toward vanishing points.

(Estimation from correspondences) Given $\ge 4$ point correspondences, $H$ is recovered by the Direct Linear Transform (DLT).

Each correspondence $(x,x^{\prime })=((u,v,1),(u^{\prime },v^{\prime },1))$ gives two linear constraints $(x^{\prime }\times Hx)=0$. Stacking $n\ge 4$ correspondences into $Ah=0$ where $h=\text{vec}(H)$ and taking the right singular vector of $A$ for its smallest singular value yields $H$. Robust estimation wraps DLT in RANSAC.

(Forward vs. inverse warping) Applying $H$ to sample pixels is best done in the inverse direction.

For each output pixel $x^{\prime }$, compute $x=H^{-1}{x}^{\prime }$ and sample the source image at $x$ (with bilinear interpolation). Forward-mapping each source pixel leaves holes in the output because the mapping is not surjective onto the pixel lattice.

Mosaicing Workflow

Image mosaicing (also panorama stitching) is the canonical multi-image application of homographies. The standard pipeline:

1. Acquire overlapping images. The photographer rotates about the optical center (a tripod minimizes translation).
2. Detect and match features. Use SIFT or ORB keypoints, then match descriptors across adjacent image pairs.
3. Estimate pairwise homographies using RANSAC with the DLT as the minimal-sample solver (4 correspondences per sample).
4. Choose a reference frame. Typically the central image of the sequence.
5. Compose homographies along paths to the reference: $H_{i\to \text{ref}}=H_{j\to \text{ref}}\cdot H_{i\to j}$.
6. Warp and blend. Inverse-warp each image into the reference frame, then blend overlapping regions (feathering, multi-band blending, or Poisson compositing) to hide seams.

A naive averaging blend produces visible seams wherever exposure or alignment differ; multi-band blending decomposes each image into Laplacian pyramid bands and blends each band with a different feather width, producing seamless results.

See Also

• 3.1. Affine Transformations — the next-most-general class (no perspective)
• 3.3. Image Interpolation — the pixel-sampling step inside warping
• 5.3. Epipolar Geometry — the model that replaces homography for non-planar scenes with camera translation
• 4.4.1. RANSAC — the robust estimator wrapped around homography fitting
• SfM — where homographies serve as a degeneracy detector