Interactive Media Group

Established: October 6, 1993

Exploring 2D and 3D graphics algorithms and representations for rendering, geometric modeling, and animation.

rmc.grouppageInteractive Media Group

There has been remarkable growth in sensor resolution on consumer devices. We expect that as storage and bandwidth continue to scale, video streams will disruptively displace photos as default capture medium. Moreover, high-resolution displays are enabling exciting immersive VR and AR scenarios. We are exploring new representations, algorithms, and devices to allow high-fidelity audio and visual rendering of rich dynamic content in these interactive experiences.

Please visit our personal pages below to see our latest projects, publications, and downloads.
















Freeform Vector Graphics

September 2011

An interactive authoring system that builds on thin-plate splines to enable a richer class of vector graphics. The user sketches a variety of curves and points to intuitively control color interpolation within the resulting image.

Size: 3 MB

    Click the icon to access this download

  • Website


Link description

Revealing the Invisible


June 3, 2013


Fredo Durand


Massachusetts Institute of Technology

Link description



May 13, 2005


Benjamin Watson and David Luebke


University of Virginia & Northwestern University

Link description

Making Faces


September 26, 1998


Brian Guenter


Neural Network Languages

Established: January 1, 2013

The goals of this project are to develop a neural network language that: is easy to use and understand can be compiled to very efficient code allows derivatives of any order makes it easy for the end user to keep track of complex tensor expressions In particular, the derivatives of matrix expressions in deep neural networks result in complex tensor expressions. These can be difficult for the average programmer to understand and manipulate and also…

Foveated 3D Display

Established: September 20, 2012

We exploit the falloff of visual acuity away from the gaze direction in the human visual system for dynamic 3D rendering. Through user studies, we have honed our system parameters and demonstrated the effectiveness of the system. We have also shown the system to bring significant performance increases, or equivalent reductions in hardware and power requirements, in typical 3D rendering applications on existing hardware. Finally, the method is easily integrated into existing 3D applications.

Cliplets: Juxtaposing Still and Dynamic Imagery

Established: March 6, 2012

What Are Cliplets?      Microsoft Research Cliplets is an interactive app that uses semi-automated methods to give users the power to create "Cliplets" - a type of imagery that sits between stills and video from handheld videos. The app provides a creative lens one can use to focus on important aspects of a moment by mixing static and dynamic elements from a video clip. Please see the BLINK and BLINK Cliplets page for more information about the…

Lens Fitting for Correction of Lens Aberrations

Established: January 1, 2012

All lenses have optical aberrations which reduce image sharpness. These aberrations can be reduced by deconvolving an image using the lens point spread function (PSF). However, fully measuring a PSF is laborious. Alternatively, one can simulate the PSF if the lens model is known. However, due to manufacturing tolerances lenses differ subtly from their models, so often a simulated PSF is a poor match to measured data. We present an algorithm that uses a PSF…

Symbolic Differentiation an HLSL language feature

Established: January 1, 2011

Derivatives arise frequently in graphics applications. To meet this need we have added symbolic differentiation as a built-in language feature in the HLSL shading language, available in the June 2010 DirectX SDK release (a much-improved version is in the Windows 8 SDK preview). The symbolic derivative is computed at compile time so it is available in all types of shaders (geometry, pixel, vertex, etc.). The algorithm for computing the symbolic derivative is simple and has…

Linear Programming for Optimal Power Control of Data Center Computers

Established: January 1, 2011

In a modern data center the cost of power, for computers and air conditioning, can be more than the cost of the computer hardware. Modern computers have a variety of power states with different power vs. response time tradeoffs: off, sleep, hibernate, etc. With thousands of computers in a typical data center it is challenging to determine what power state each computer should be in at any moment in order to minimize power while maximizing…

D* Symbolic Differentiation Code and Geometric Modeling and Dynamics Examples

Established: January 1, 2009

The download includes the D* executable and full source code for the following: basic D* programming examples, Lagrangian dynamics, and a basic interactive geometric modeling tool. This is unsupported code but if you find bugs or have feature requests send an email to There is a book that describes the dynamics algorithms, as well as simple procedural modeling techniques. You will probably want the book if you are downloading the code, since it has more…

Symbolic Differentiation

Established: January 1, 2007

D* generates symbolic derivatives which can be thousands of times faster than those generated by Mathematica or automatic differentiation. Functions with densely interconnected expression graphs, which arise in applications such as dynamics, spacetime optimization (also known as the optimal control problem), and PRT, can be difficult to efficiently differentiate using existing symbolic or automatic differentiation techniques. The D* algorithm computes efficient symbolic derivatives for these functions by symbolically executing the expression graph at compile time…

Procedural Modeling

Established: January 1, 2005

Generative CSG models, while having the desirable characteristics of compactness and resolution independence, have never been used for real time rendering because no algorithms existed which could both maintain their compact representation and render them efficiently at run time. The key difficulty in doing this was finding a compact, exact representation of the implicit curve of intersection that arises from CSG operations. The primary contribution of this paper is a new algorithm for finding a…

Face Animation

Established: January 1, 2000

Video: Making Faces. An actress reads from a script designed to provide maximal phonetic coverage (we were planning to do automatic lip sync as a follow-on research project but never got around to it). Hence the funky monologue. Real-time, Photo-realistic, Physically Based Fine-scale Human Skin Structure Rendering A. Haro, B. Guenter, and I. Essa, Proceedings 12th Eurographics Workshop on Rendering, London, England, June 2001 Modeling and Rendering for Realistic Facial Animation Stephen R. Marschner, Brian Guenter, and…


Scanning Inside the Box at SIGGRAPH

SIGGRAPH 2013, the 40th International Conference and Exhibition on Computer Graphics and Interactive Techniques, occurs in Anaheim, California, from July 21 to 25. Sponsored by the Association for Computing Machinery, the conference is expected to attract 20,000 professionals eager to learn about advances across many technologies. The event has long been considered a breeding ground for significant new areas of research. When principal researcher Andy Wilson attends SIGGRAPH 2013, he will represent two contributions. The…

July 2013

Microsoft Research Blog

Terapixel Project: Lots of Data, Expertise

By Rob Knies, Managing Editor, Microsoft Research How can you achieve the impossible? Easy—as long as you have the right people and the right tools. The Terapixel project from Microsoft Research Redmond is proof positive. The effort—to create the largest, seamless spherical image ever made of the night sky—has tantalized astronomers for decades, but the sheer volume of data and the challenges in data manipulation have proved frustrating. Until now, that is. A small but…

July 2010

Microsoft Research Blog

Using Repeated Image Content to Render Photos More Efficiently

By Rob Knies, Managing Editor, Microsoft Research In these days of nearly ubiquitous access to digital cameras, and the consequent explosion of images available via the Internet, it has become more important than ever to find ways to render those images efficiently. Unfortunately, the complex nature of digital imagery consumes prodigious amounts of bandwidth and memory. How can resources be utilized efficiently while maintaining a high level of quality in coping with the deluge of…

August 2008

Microsoft Research Blog