Lines, Shading, and the Perception of 3D Shape


August 1, 2012


Ted Adelson




Humans can easily see 3D shape from single 2D images, exploiting multiple kinds of information. There are several subfields (in both human vision and computer vision) devoted to the study of particular cues to 3D shape, such as shading, texture, and contours. However, the resulting algorithms remain specialized and fragile (in contrast with the flexibility and robustness of human vision). Recent work in graphics and psychophysics has demonstrated the importance of local orientation structure in conveying 3D shape. This information is fairly stable and reliable across rendering condition. We have developed an exemplar-based system (which we call Shape Collage) that learns to associate image patches with corresponding 3D shape patches.
We train it with synthetic images of “blobby” objects rendered in various ways, including solid texture, Phong shading, and line drawings. Given a new image, it finds the best candidate scene patches and assembles them into a coherent interpretation of the object shape.
Our system is the first that can retrieve the shape of naturalistic objects from line drawings. The same system, without modification, works for shape-from-texture and can also do shape-from-shading without requiring Lambertian surfaces. Thus disparate types of image information can be processed by a single mechanism to extract 3D shape.
(Collaborative work with Forrester Cole, Phillip Isola, Fredo Durand, and William Freeman.)


Ted Adelson

Edward Adelson is the John and Dorothy Wilson Professor in the Dept. of Brain and Cognitive Science, MIT, and is a member of MIT’s Computer Science and AI Lab CSAIL). He has published numerous papers on human vision, computer vision, computational neuroscience, image coding, and computer graphics. He has received the Optical Society of American’s Lomb Medal, the Rank Prize in Opto-electronics, the IEEE Computer Society’s Longuet-Higgins Prize in Computer Vision. He is a member of the National Academy of Sciences.