CS180/280A · Proj3A · Image Warping & Mosaicing (Part A)

A · IMAGE WARPING and MOSAICING

A1 · Shoot and Digitize Pictures

I captured multiple photographs with overlapping views. These images will be used in the later parts for homography estimation, rectification, and mosaicing.

Set 1 · indoor example

Set 2 · outdoor example

A2 · Recover Homographies

Manual correspondences → DLT & normalized DLT → reprojection errors.

Correspondence Visualization

A2 correspondences — **Clicked correspondences**

DLT Linear System

Each correspondence (x,y) → (x′,y′) contributes two rows in the matrix A:


        [-x  -y  -1   0   0   0   x·x′   y·x′   x′]
        [ 0   0   0  -x  -y  -1   x·y′   y·y′   y′]

These stacked rows form a 2N×9 system A·h=0, solved by SVD to obtain the homography vector h.

Recovered Homography H


          0.450826  -0.032532  1687.801036
          -0.196410   0.802739   252.621928
          -0.000130  -0.000011     1.000000

Reprojection Errors

Applying H to im1_pts and comparing against ground truth im2_pts, I measured per-point reprojection errors:


          x1,y1         x2,y2          pred_x2,pred_y2   error_px
      393.41,916.92   1957.90,965.23    1955.11,970.88    6.30
      478.88,1024.69  2024.79,1058.13   2018.84,1059.01   6.01
      300.51,1151.04  1887.29,1177.05   1883.50,1178.70   4.14
      ... (etc) ...

RMSE: 7.3651 px

A3 · Warp the Images & Rectification

Inverse warping with NN/Bilinear, rectifying a planar quadrilateral to a canonical rectangle.

Example 1 · iPad Cover Rectification

Input poster — **Original** · Input image

Poster rectified NN — **Nearest Neighbor** · Rectified

Poster rectified Bilinear — **Bilinear** · Rectified

Example 2 · Poster Rectification

Input book cover — **Original** · Input image

Book rectified NN — **Nearest Neighbor** · Rectified

Book rectified Bilinear — **Bilinear** · Rectified

Observation. Nearest Neighbor preserves edges but causes jagged artifacts. Bilinear produces smoother results, though with slight blurring. Overall, bilinear is preferred for natural-looking rectification.

A4 · Blend Images into a Mosaic

Warp each image to a global canvas and feather-blend using soft alpha from EDT.

Feather blended mosaic — five pic example

Homography direction: A4 expects H: img_i → reference. If your saved H is ref → img_i, pass it as inv:path.npy (loader will invert).

B · FEATURE MATCHING for AUTOSTITCHING

Following Brown et al. (MSOP, 2005). B.1: Harris + ANMS · B.2: 40→8 descriptors · B.3: Lowe ratio matching · B.4: 4-point RANSAC + mosaics.

B1 · Harris Corner Detection + ANMS

Detect Harris corners (single-scale), then apply Adaptive Non-Maximal Suppression (ANMS) to obtain a spatially uniform subset.

Harris corners with ANMS — **Harris + ANMS (N≈500)**

Corners are drawn directly onto the image pixels (OpenCV). ANMS uses the “stronger-by-ratio” radius (ratio≈1.11) and keeps top-N radii.

B2 · Feature Descriptor Extraction (40×40 → 8×8)

For each ANMS keypoint, sample a 40×40 gray patch, lightly blur, downsample to 8×8, then bias/gain normalize (z-score). Descriptors are stored as 64-D row vectors (N×64).

Keypoints used for descriptors — **Keypoint locations used for descriptors**

descriptors.npyshape = (N, 64) · float32 coords.npyshape = (N, 2) = (y, x)

B3 · Feature Matching (Lowe Ratio Test)

Compute pairwise distances in descriptor space. For each feature in image A, accept match to image B if d₁/d₂ < ρ (ρ≈0.7), optionally with mutual check.

**Side-by-side matches** (random colors, capped by max_draw)

matches.npyshape = (M, 2), rows are [idx_in_img1, idx_in_img2] ratio0.6–0.8 (0.7 default) cross-checkoptional, improves inlier rate

B4 · 4-point RANSAC for Robust Homography + Mosaics

Estimate H (img2 → img1) via 4-point RANSAC with normalized DLT and reprojection error thresholding. Re-fit H on inliers, then warp & feather-blend to get panoramas.