Compute Vision : Exercise 4 – Panorama Stitching
Computer Vision
Exercise 4 – Panorama Stitching
13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
The task
13/01/2015 2
Pipeline
1. Interest point detector: Harris, Scale-‐space analysis, Edge detection, MSER-‐s …
2. Image features: Patches (squared distance, correlation
coefficients), SIFT, SURF, moments, geometric features, affine invariance, projective invariance …
3. Matching: brute force + cross-‐validation, marriage problem … 4. Homography estimation: RANSAC (OpenCV), other RANSAC-‐
variants, robust optimization … 5. Rendering
3 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Harris detector
4
1. Computation of coefficients
2. Smoothing – integral image, separable Gaussian kernel 3. Computation of the “Cornerness”.
4. Non-‐maxima suppression 5. Some post-‐processing
13/01/2015 Compute Vision : Exercise 4 – Panorama Stitching
Matching
Here:
squared difference of image patches, brute force comparison, cross-‐
validation
Other possibilities:
Features – correlation coefficients, SIFT, explicit color correction (iterative)
Matching – marriage problem …
5 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Homography estimation
Here:
1. RANSAC (OpenCV implementation)
2. Estimation of two homographies (needed for rendering)
6 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Rendering
Standard (OpenCV) – left to right and vice versa
Symmetric:
7 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Problems – homogenous areas
8
Interest points are found on textured regions only
13/01/2015 Compute Vision : Exercise 4 – Panorama Stitching
Problems – small overlap
Not enough matches – very coarse alignment
9 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Problem – different colors
Original:
Interest points – Ok.
10 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Problem – different colors
Matches –
obviously too few
11 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Problem – different colors
Panorama – the homographies are wrong
12 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Assignments
• You have the code and the images, play with it – change the parameters, introduce image distortions, identify problems …
• You are allowed to use the code, you can replace its parts (or even completely) by OpenCV functions or by your own implementations
• There should be at least one step (function) in the whole pipeline implemented by yourself !!! Note: own solutions for the 2nd exercise (e.g. Harris detector, filtering etc.) will be not considered as solutions for this one – something new should be done for this exercise as well
• Address problems considered above e.g. colors, homogenous regions etc.
– think what steps could be improved (and how) in order to reach better performance
• Address the pipeline steps – think how to do them generally better
13 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Assignments
All the stuff (these slides + the code) is available under
http://www.inf.tu-‐dresden.de/index.php?node_id=2091&ln=de
Deadline – 06.02.2014 per e-‐mail an Dmytro.Shlezinger@...
Add-‐ons (extra points):
• Stereo – estimate the fundamental matrix instead of the homography
• Many images instead of only two (in a row)
• Many images aligned in a grid
• Many not aligned images
• …
14 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Some suggestions, hints, other possible works …
Let some “approximate” homography be known. What it can be useful for?
− You can adjust image coloring in order to achieve better
dissimilarity measure for patches (since you know the “true”
matches)
− You can (drastically) reduce the search space for estimation of the matches from interest points, i.e. do not compare all to all. If so, you can use much more interest points without to essentially increase the time complexity
15 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Some suggestions, hints, other possible works …
16 13/01/2015
Compute Vision : Exercise 4 – Panorama Stitching
Some suggestions, hints, other possible works …
Another way – use resolution pyramid 1. Scale down the images
2. Estimate the homography for downscaled images
3. Use this homography to estimate the one for the originals
4. Use this estimation to adjust colorings (speed up the matching etc.) for the original images
Some other possible works:
− Think about an iterative approach using the “symmetric” rendering
− Do stereo-‐matching or multi-‐view 3D-‐reconstruction
− …
13/01/2015 17 Compute Vision : Exercise 4 – Panorama Stitching
