Project Vermont: Improving camera performance through curved image sensors
By Richard Stoakley, Microsoft Research
It’s hard to remember when our world wasn’t full of cameras. They’re in our phones and tablets, posted on street corners and in our buildings and cars. We rely on them not just to capture moments and memories, but for security, navigation, 3D spatial mapping and processing medical data.
While significant advances have been made to camera sensors and the algorithms that process images, optical systems have not seen the same radical improvements. Our small team of Microsoft researchers, including Brian Guenter and Neel Joshi, set out to solve one of the most well-known optical issues in conventional photography: field curvature.
Field curvature exists because lenses don’t want to focus on a plane. As illustrated to the right, the lens delivers sharpest focus along a curved path, rather than on a flat surface. The result: objects appear out of focus across the image. By curving the focal surface, we eliminate the need for the lens to address field curvature and can use the resulting lens design freedom to improve other aspects of the system.
How can we curve the focal surface? There are multiple approaches to this and some work well in specific circumstances (for example, curved arrays of flat sensors can be useful in astronomical telescopes), but most known techniques fail in conventional, human-scale cameras. An obvious approach would be to bend the sensor itself, but since silicon sensors are monocrystalline, they are prone to shattering rather than bending.
Enter Project Vermont
Several companies have attempted to bend silicon, but attaining useful amounts of curvature has remained elusive. To address these limitations, Brian, Neel and I joined together on Project Vermont — a project designed to push the limits of silicon sensor curvature and deliver substantial camera performance improvements.
We designed numerous optical models and built several early prototypes to validate our hypotheses regarding these improvements. Those early successes helped us identify key opportunities to attain higher degrees of curvature. We then collaborated with scientists at HRL Laboratories to capitalize on these opportunities. In our new paper, Highly curved image sensors: a practical approach for improved optical performance , published today in Optics Express, the team outlines the various techniques we developed to attain useful amounts of curvature and shows empirical prototype results.
What does Project Vermont mean for the future of cameras? Curving the image surface can dramatically improve performance along many axes — resolution, light-gathering, and illumination uniformity — while also reducing system size, cost, and complexity. While the amount of improvement is dependent on many camera system variables, including focal length, sensor size, and lens complexity, Project Vermont showed substantial progress over previous approaches.
In our most recent research prototype, we significantly outperformed a premium 35mm digital single- lens reflex camera (DLSR) in both sharpness and uniformity of illumination, using a much smaller cellphone-sized sensor. At present, our prototype camera system is sharper (in terms of line pairs per picture height) than any known commercially available consumer camera.
Applying this technique to cellphones will require additional research and engineering. However, for more conventional camera systems, we see a near-term opportunity for significant improvement and are working with third parties to advance the state of the art. We hope this work inspires further investigation into new optical systems and an exciting new world of cameras.
For more information, please visit Project Vermont.
Editor’s Note: Richard is a principal program manager with Microsoft Research and the project manager for Project Vermont.