Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Project Vermont: Improving camera performance through curved image sensors

May 30, 2017 | By Microsoft blog editor

By Richard Stoakley, Microsoft Research

It’s hard to remember when our world wasn’t full of cameras. They’re in our phones and tablets, posted on street corners and in our buildings and cars. We rely on them not just to capture moments and memories, but for security, navigation, 3D spatial mapping and processing medical data.

While significant advances have been made to camera sensors and the algorithms that process images, optical systems have not seen the same radical improvements. Our small team of Microsoft researchers, including Brian Guenter and Neel Joshi, set out to solve one of the most well-known optical issues in conventional photography: field curvature.

Project Vermont field curvatureField curvature exists because lenses don’t want to focus on a plane. As illustrated to the right, the lens delivers sharpest focus along a curved path, rather than on a flat surface. The result: objects appear out of focus across the image. By curving the focal surface, we eliminate the need for the lens to address field curvature and can use the resulting lens design freedom to improve other aspects of the system.

How can we curve the focal surface? There are multiple approaches to this and some work well in specific circumstances (for example, curved arrays of flat sensors can be useful in astronomical telescopes), but most known techniques fail in conventional, human-scale cameras. An obvious approach would be to bend the sensor itself, but since silicon sensors are monocrystalline, they are prone to shattering rather than bending.

Enter Project Vermont

Several companies have attempted to bend silicon, but attaining useful amounts of curvature has remained elusive. To address these limitations, Brian, Neel and I joined together on Project Vermont — a project designed to push the limits of silicon sensor curvature and deliver substantial camera performance improvements.

We designed numerous optical models and built several early prototypes to validate our hypotheses regarding these improvements. Those early successes helped us identify key opportunities to attain higher degrees of curvature. We then collaborated with scientists at HRL Laboratories to capitalize on these opportunities. In our new paper, Highly curved image sensors: a practical approach for improved optical performance , published today in Optics Express, the team outlines the various techniques we developed to attain useful amounts of curvature and shows empirical prototype results.

Project Vermont camera lens curvature

The Future

What does Project Vermont mean for the future of cameras? Curving the image surface can dramatically improve performance along many axes — resolution, light-gathering, and illumination uniformity — while also reducing system size, cost, and complexity. While the amount of improvement is dependent on many camera system variables, including focal length, sensor size, and lens complexity, Project Vermont showed substantial progress over previous approaches.

In our most recent research prototype, we significantly outperformed a premium 35mm digital single- lens reflex camera (DLSR) in both sharpness and uniformity of illumination, using a much smaller cellphone-sized sensor. At present, our prototype camera system is sharper (in terms of line pairs per picture height) than any known commercially available consumer camera.

Applying this technique to cellphones will require additional research and engineering. However, for more conventional camera systems, we see a near-term opportunity for significant improvement and are working with third parties to advance the state of the art. We hope this work inspires further investigation into new optical systems and an exciting new world of cameras.

For more information, please visit Project Vermont.

Related:

May 30, 2017, The Optical Society: Breakthrough Curved Sensor Could Dramatically Improve Image Quality Captured with Digital Cameras

Editor’s Note: Richard is a principal program manager with Microsoft Research and the project manager for Project Vermont.

Up Next

Artificial intelligence, Computer vision, Graphics and multimedia

Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode

Microsoft HoloLens is the world’s first self-contained holographic computer. Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it’s also a potent computer vision research device. Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision […]

Marc Pollefeys

Partner Director of Science

Artificial intelligence, Computer vision, Graphics and multimedia

Second version of HoloLens HPU will incorporate AI coprocessor for implementing DNNs

By Marc Pollefeys, Director of Science, HoloLens It is not an exaggeration to say that deep learning has taken the world of computer vision, and many other recognition tasks, by storm. Many of the most difficult recognition problems have seen gains over the past few years that are astonishing. Although we have seen large improvements […]

Microsoft blog editor

Holograms: The future of near-eye display?

Graphics and multimedia, Hardware, devices and quantum computing

Holograms: The future of near-eye display?

By Andrew Maimone, Researcher; Andreas Georgiou, Researcher; Joel Kollin, Principal Research Hardware Development Engineer Last week at the SCIEN Workshop on Augmented and Mixed Reality, a group of industry and academic researchers met to discuss the future of virtual and mixed reality (VR/MR). One goal was clear: we’d all like to put on devices that look and feel […]

Microsoft blog editor