Todd Zickler
Assistant Professor of Electrical Engineering
Division of Engineering and Applied Science
Harvard University

Principal Investigator: Todd Zickler
Primary sponsor: National Science Foundation CAREER award IIS-0546408

An appearance model of an object (or scene) is a representation that allows it to be virtually rotated, re-lit and seamlessly composited with other imagery. For opaque and non-refracting surfaces, such a model typically comprises the three-dimensional shape of the object along with a reflectance function defined on that shape.

Appearance capture is the process of creating an appearance model. Traditionally, accurate appearance capture requires rather specialized equipment: laser scanners, projectors for structured-lighting, and/or complex imaging systems that capture hundreds or thousands of images per object.

What if appearance capture required only web-cams and LED light-sources? What if it could be readily accomplished by the average PC user? This might fundamentally change the way in which we exchange and display visual information, and finally move us beyond fixed, two-dimensional imagery.

The goal of this project is to lay the foundation for ubiquitous image-based appearance capture systems. Our approach is based on a simple observation: although the world contains a wide variety of materials, there are common reflectance properties (isotropy, reciprocity, spatial coherence, etc.) that are exhibited by broad classes of these materials. Buy developing tools that exploit these properties, we hope to build image-based appearance capture systems that are simultaneously accurate and practical.

RELATED PUBLICATIONS

		Color Subspaces as Photometric Invariants. Todd Zickler, Satya Mallick, David Kriegman, and Peter Belhumeur. International Journal of Computer Vision `Projections' in RGB color space yield images that preserve diffuse shading information and are independent of specular reflections. Computing these projections as a pre-process can improve the performance of Lambertian-based tools for stereo, shape-from-shading, motion estimation, and more.
		Toward a Theory of Shape from Specular Flow. Yair Adato, Yuriy Vasilyev, Ohad Ben-Shahar, and Todd Zickler. ICCV 2007 Relative motion between a curved, specular (mirror-like) surface and its environment induces a motion field on the image plane--termed specular flow--that provides direct access to surface shape information.
		The von Kries Hypothesis and a Basis for Color Constancy. Hamilton Chong, Steven J. Gortler, and Todd Zickler. ICCV 2007 Necessary and sufficient conditions for a set of sensors, illuminant spectra, and material reflectances to support generalized diagonal color constancy; and an algorithm for computing an optimal color basis for von Kries adaptation.
		Model-based Stereo with Occlusions. Fabiano Romeiro, and Todd Zickler. IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG) 2007 Fitting three-dimensional morphable models to stereo pairs of images in the presence of foreign-body occlusions.
		Isotropy, Reciprocity and the Generalized Bas-relief Ambiguity. Ping Tan, Satya P. Mallick, Long Quan, David J. Kriegman, and Todd Zickler. CVPR 2007 Isotropy and reciprocity, which are common reflectance phenomena, induce intensity-based constraints on surface shape. These constraints are sufficient for resolving the generalized bas-relief ambiguity.
		Specularity Removal in Images and Videos: A PDE Approach. Satya Mallick, Todd Zickler, David Kriegman, and Peter Belhumeur ECCV 2006 Interpreting SUV color space as a partial separation of the diffuse and specular components of an image, we complete this separation using multi-scale morphological filters.
		Reciprocal Image Features for Uncalibrated Helmholtz Stereopsis. Todd Zickler CVPR 2006 Feature detection and matching in Helmholtz stereo pairs provides both geometric and radiometric calibration. Combined with previous methods for dense reconstruction, this provides an automated reconstruction system for surfaces with arbitrary, complex BRDFs.
		Reflectance Sharing. Todd Zickler, Sebastian Enrique, Ravi Ramamoorthi, and Peter Belhumeur Eurographics Symposium on Rendering 2005 Exploiting spatial coherence to recover a reflectance function from a small number of images of a known shape. The PAMI 2006 journal version includes new analysis of spatial reflectance in terms of Fourier theory.

Some of this material is based upon work supported by the National Science Foundation under CAREER Grant No. ISS-0546408. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.