What is the Range of Surface Reconstructions from a
Gradient Field?
Writer: Amit Agrawal, Ramesh Raskar, and Rama Chellappa
Presenter: Hosna Sattar
Uni Saarland
Milestones and Advances in Image Analysis, 2013
(ECCV 2006)
Motivation
motivation M-estimator diffusion
Alpha-Surface Poisson solver
General framework
Seamless Cloning Selection Editing
Texture Flattening
• Compute gradient of image
• Manipulate the gradient field in order to achieve the desired goal
Gradient field and its application
Second derivatives: and . They are identical! Right: Integration of gradient field ( , ) which is identical to original image.
Integrating the Modified Gradient Field
(x) f
(x) f
(x) (x) - f
f (x))
(x), f curl (f
f(x)) curl (
xy yx
xy yx
T
x y
0
In order to integrate the gradient field it should be curl-free:
f
xyf
yxf
• In fact, the modified gradient field might even be non-integrable!
Integrating the Modified Gradient Field
Left: space of all solution right: add add correction gradient field to make it integrable. x , y
x
y
• A common approach to achieve the surface from the non-integrable gradient field is to minimized the last square error function:
• The goal is to obtain surface Z. p(x,y) and q(x,y) are given non-integrable gradient field.
Problem statement
)dxdy - q)
(Z - p)
((Z
J(Z) x 2 y 2
• Which can also write as:
• The Euler-Lagrange equation gives the Poisson equation:
q
div p
2 Z
y x q
p Z
Z
y x
Problem statement
content
motivation M-estimator diffusion
Alpha-Surface Poisson solver
General framework
Effect of outliers in 2D integration (a) True surface (b) Reconstruction using Poisson solver. (c) If the location of outliers were known, rest of the gradients can be integrated to obtain a much better estimate.
• Least square solution doesn't perform well in presence of outliers:
Problem of Poisson equation
General framework
E(Z, p, q, Z , p , q , . . .) dxdy
J
a b c d c dy y x
y x x
A general solution can be obtained by minimizing the following n-th order error functional:
) dxdy Z
Z E(Z, p, q, J
if k n;
k
y x q q
y x p p
y x
Z Z
k, k -
d k, c
b a
y
d c
k- y
d x c
k- y
b x a
k y
xa b c d c d
,
1
1
,
,
integer
positive
some for
1
1 1
General framework
(p,q) f
y ) x ,Z
(Z f
Z y E
(p,q), f
y ) x ,Z
(Z f
Z x E
: Z y
, E Z x
E )
Z y , E Z x
div( E Z
E
4 2
3 1
for form
following consider
we if
: gives equation
Lagrange -
Euler the
(1)
(2)
(p,q)) (p,q),f
div(f Z
) E ,Z y
(Z x f
y ) x ,Z (Z f
div ( ,
2) 3 4
1
: in to(1) (2)
ing substituit
by
free curl
is field gradient
modified the
while
In all solutions we assume Neumann boundary conditions given by:
0 .
Z n
General framework
• To achieve Poisson equation from the general solution its just need to assume:
Poisson solver
div(p, q) Z
2
(p,q)) (p,q),f
div(f Z
) E ,Z y
(Z x f
y ) x ,Z (Z f
div ( ,
2) 3 4
1
0
Z
E )
,Z y (Z x
f
1)
,Z y (Z x
f
2f (p,q),
3 f (p,q), 4
Z x
Z y p q
content
motivation M-estimator diffusion
Alpha-Surface Poisson solver
General framework
• Techniques for robust estimation:
1. α- Surface: Anisotropic scaling using binary weights
2. Anisotropic scaling using continuous weight
3. Affine transformation of gradient using diffusion tensor
Continuum of solution
• Define initial spanning tree which is all gradient correspond to edge and are
inliers
α- Surface
q Z
y
p Z
x
y
x
x
y
If α=0 we get our initial spanning tree and if α=1 we will get our
poisson solver.
The α- Surface is a weighted approach where the weight are 1 for gradients in S and otherwise zero.
α- Surface formulation
dxdy - q)
(Z b - p)
(Z b J(Z)
y x b
y y x
x x
2 2
y y
x
S, 0 o.w., b (x, y) 1 if Z S, 0 o.w., Z
if 1 ) , (
Corresponding Euler_ Lagrange is:
) ,
( )
,
( b Z b Z div b p b q
div x x y y x y
• M- estimator: the effect of outliers is reduced by replacing the squared error residual by another function of residual:
Anisotropic scaling using continuous weight
dxdy - q)
(Z y k
w y - p)
Z x k
w x
J(Z) ( 1 )( 2 ( 1 ) 2
• A method for image restoration from noisy image.
• The Euler-Lagrange gives:
Affine transformation of gradient using diffusion tensor
dxdy
q p Z
D Z Z
E
yx
2
) ) (
(
) (
) .
(
q
D p div Z
D
div
q d p d
q d p div d
Z d Z
d
Z d Z
div d
y x d
y x d
y x d
y x D d
y x
y x
22 21
12 11
22 21
12 11
22 12 21
11
: below as
combined lineary
and scaled
are gradients
The
) , (
) , (
) , (
) , (
Affine transformation of gradient using diffusion tensor
D is 2×2 symmetric , positive-definite matrix at each pixel.
results
Photometric Stereo on Vase: (Top row) Noisy input images and true surface (Next two rows) Reconstructed surfaces using various algorithms. (Right Column) One-D height plots for a can line across the middle of Vase. Better results are obtained using α-surface, Diffusion and M-estimator as compared to Poisson solver, FC and Regularization
results
Photometric Stereo on Mozart: Top row shows noisy input images and the true surface. Next two rows show the reconstructed surfaces using various algorithms. (Right Column) One-D height plots for a scan line across the Mozart face. Notice that all the features of the face are preserved in the solution given by α-surface, Diffusion and M-estimator as compared to other algorithms.
results
Photometric Stereo on Flowerpot: Left column shows 4 real images of a flower pot. Right columns show the reconstructed surfaces using various algorithms. The reconstructions using Poisson solver and Frankot-Chellappa algorithm are noisy and all features (such as top of flower pot) are not recovered. Diffusion, α-surface and M-estimator methods discount noise while recovering all the salient features.
