Todd Zickler
John L. Loeb Associate Professor of the Natural Sciences
School of Engineering and Applied Science
Harvard University

email: zickler {at} eecs.harvard.edu


home research publications courses contact CV

HELMHOLTZ STEREOPSIS

Image-based object reconstruction is the process of estimating the shape and surface reflectance properties on an object from its images. Applications include graphics (accurate rendering for virtual and augmented reality) and shape measurement (reverse engineering, visual inspection, digital object archival).

What makes the problem a difficult one is the fact that reflectance and shape are coupled (along with illumination) in the image data. If we are given the shape and illumination, we can say something about the reflectance, and vice versa. But what if we are given neither? Most conventional object reconstruction techniques (e.g., stereo and photometric stereo) make the problem tractable by assuming something about the surface reflectance -- usually by choosing the simplest reflectance model (Lambertian reflectance).

In contrast, we are interested in recovering shape (and then reflectance) without making the usual assumptions.  Specifically, we are interested in object reconstruction for objects with arbitrary (or general) surface reflectance (BRDF).

The idea behind Helmholtz Stereopsis is to exploit the symmetry of surface reflectance (commonly referred to as Helmholtz reciprocity). Consider taking two images as shown below. First we capture an image of the object illuminated by a point light source, and then we capture another with the light source and camera positions swapped exactly. Because of Helmholtz reciprocity the resulting two images have an important property: for corresponding pixels, the ratio of incident irradiance (onto the object) to emmitted radiance (from the object) is the same. Put more simply, we can derive a relationship between the intensities of corresponding pixels that does not depend on the BRDF of the surface.


Reciprocal Pair of
Images

FIGURE 1: Capturing a reciprocal pair of images

The relationship between corresponding pixel intensities in a reciprocal pair of images depends on the depth of the projected surface point as well as the surface normal at that point. What this means is that by collecting multiple reciprocal image pairs, we can recover both the surface depth and the field of surface normals. This unique property of directly estimating surface normals is extremely valuable, since accurate normals are essential for image-based reflectance measurement and for rendering.


Depth and
normals

FIGURE 2: An examples of recovered surface normals


Depth and
normals

FIGURE 3: The corresponding surface. [WRL] [WRL.GZ]


References:

  • Todd Zickler, "Reciprocal Image Features for Uncalibrated Helmholtz Stereopsis." Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2006. [PDF]
  • Todd Zickler, Jeffrey Ho, David J. Kriegman, Jean Ponce, and Peter N. Belhumeur, "Binocular Helmholtz Stereopsis." Proc. IEEE International Conference on Computer Vision, October 2003. pp. 1411-1417. [PDF] [IEEE Xplore]
  • Todd Zickler, Peter N. Belhumeur, and David J. Kriegman, "Toward a Stratification of Helmholtz Stereopsis." Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2003. Vol. I, pp 548-555. [PDF] [IEEE Computer Soc.]
  • Todd Zickler, Peter N. Belhumeur, and David J. Kriegman, "Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction." Proc. 7th European Conference on Computer Vision, May 2002.Vol. III, pp 869-884. [PDF] [SpringerLink]
  • Sebastian Magda, David J. Kriegman, Todd Zickler, and Peter N. Belhumeur, "Beyond Lambert: Reconstructing Surfaces with Arbitrary BRDFs," Proc. 8th IEEE International Conference on Computer Vision, June 2001. Vol. II, pp 391-398. [IEEE Xplore]
Updated: June 28, 2006