Foveated 3D Graphics
We exploit the falloff of acuity in the visual periphery to accelerate graphics computation by a factor of 5-6 on a desktop HD display (19201080). Our method tracks the user’s gaze point and renders three image layers around it at progressively higher angular size but lower sampling rate. The three layers are then magnified to display resolution and smoothly composited. We develop a general and efficient antialiasing algorithm easily retrofitted into existing graphics code to minimize “twinkling” artifacts in the lower-resolution layers. A standard psychophysical model for acuity falloff assumes that minimum detectable angular size increases linearly as a function of eccentricity. Given the slope characterizing this falloff, we automatically compute layer sizes and sampling rates. The result looks like a full-resolution image but reduces the number of pixels shaded by a factor of 10-15. We performed a user study to validate these results. It identifies two levels of foveation quality: a more conservative one in which users reported foveated rendering quality as equivalent to or better than non-foveated when directly shown both, and a more aggressive one in which users were unable to correctly label as increasing or decreasing a short quality progression relative to a high-quality foveated reference. Based on this user study, we obtain a slope value for the model of 1.32-1.65 arc minutes per degree of eccentricity. This allows us to predict two future advantages of foveated rendering: (1) bigger savings with larger, sharper displays than exist currently (e.g. 100 times speedup at a field of view of 70° and resolution matching foveal acuity), and (2) a roughly linear (rather than quadratic or worse) increase in rendering cost with increasing display field of view, for planar displays at a constant sharpness.
Foveated 3D Graphics
September 17, 2014
This download represents the source code supporting the 2012 SIGGRAPH paper, “Foveated 3D Graphics”, available at http://research.microsoft.com. Dependencies on any specific gaze tracker have been removed, and replaced by function stubs which track the mouse cursor. The stubs may be filled in with calls to any high speed low latency gaze tracker. In particular, the key anti-aliasing techniques as described in the paper are demonstrated. The source code implementing the foveated rendering techniques is isolated into a small library. This library is provided in two formats, supporting both the D3DX API from the June 2010 DirectX SDK, as well as the XVECTOR API of the latest DirectX Windows Kits. A generic application rendering objects on a terrain is provided to drive the foveated rendering technique, but the technique is general to most real time 3D rendering. The application is provided as a Visual Studio 2010 solution which compiles against the June 2010 DirectX SDK.
A short summary motivating our foveated 3D rendering technique, the challenges faced and our solutions, and our validation and tuning through user studies. Finally we extrapolate the benefits of our technique onto hardware trends into the future.