Image-based object reconstruction is the process of estimating the
shape and surface reflectance properties on an object from its images.
Applications include graphics (accurate rendering
for virtual and augmented reality) and shape measurement (reverse engineering,
visual inspection, digital object archival).
What makes the problem a difficult one is the fact that reflectance
and shape are coupled (along with illumination) in the image data. If
we are given the shape and illumination, we can say something about
the reflectance, and vice versa. But what if we are given neither?
Most conventional object reconstruction techniques (e.g., stereo and
photometric stereo) make the problem tractable by assuming something
about the surface reflectance -- usually by choosing the simplest
reflectance model (Lambertian reflectance).
In contrast, we are interested in recovering shape (and then
reflectance) without making the usual assumptions.
Specifically, we are interested in object reconstruction for objects
with arbitrary (or general) surface reflectance (BRDF).
The idea behind Helmholtz Stereopsis is to exploit the symmetry of
surface reflectance (commonly referred to as Helmholtz
reciprocity). Consider taking two images as shown below. First we
capture an image of the object illuminated by a point light source,
and then we capture another with the light source and camera positions
swapped exactly. Because of Helmholtz reciprocity the resulting two
images have an important property: for corresponding pixels, the ratio
of incident irradiance (onto the object) to emmitted radiance (from the
object) is the same. Put more simply, we can derive a relationship
between the intensities of corresponding pixels that does not
depend on the BRDF of the surface.
FIGURE 1: Capturing a reciprocal pair of images
The relationship between corresponding pixel intensities in a
reciprocal pair of images depends on the depth of the projected
surface point as well as the surface normal at that point.
What this means is that by collecting multiple reciprocal image pairs,
we can recover both the surface depth and the field of surface
normals. This unique property of directly estimating surface normals
is extremely valuable, since accurate normals are essential
for image-based reflectance measurement and for rendering.
FIGURE 2: An examples of recovered surface normals
FIGURE 3: The corresponding surface.
"Reciprocal Image Features for Uncalibrated Helmholtz Stereopsis."
Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2006.
Todd Zickler, Jeffrey Ho, David J. Kriegman, Jean Ponce,
and Peter N. Belhumeur, "Binocular Helmholtz Stereopsis."
Proc. IEEE International Conference on Computer Vision, October 2003. pp. 1411-1417.
Todd Zickler, Peter N. Belhumeur, and David J. Kriegman, "Toward a
Stratification of Helmholtz Stereopsis."
Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2003.
Vol. I, pp 548-555.
[IEEE Computer Soc.]
Todd Zickler, Peter N. Belhumeur, and David J. Kriegman, "Helmholtz
Stereopsis: Exploiting Reciprocity for Surface Reconstruction."
Proc. 7th European Conference on Computer Vision, May 2002.Vol. III, pp 869-884.
Sebastian Magda, David J. Kriegman, Todd Zickler, and Peter N. Belhumeur,
"Beyond Lambert: Reconstructing Surfaces with Arbitrary BRDFs,"
Proc. 8th IEEE International Conference on Computer Vision, June 2001.
Vol. II, pp 391-398.