ClearType display for Surface

When you look up at the night sky how many stars do you see? Well, it depends on where you are. If you are out in the country there is not much light pollution, and the night sky lights up with stars. If you are in the city, there is a lot of light pollution and you won’t see as many. The stars are still there, it is the same sky, but the contrast is low, so as a consequence the tiny dots in the sky are hidden. Even the color in the night sky will look more washed out in the city. No matter how many “pixels” the sky has, you are not going to see any more colorful stars because the contrast is lower. This analogy illustrates that contrast, color, and detail are all strongly related, they affect each other. The same ideas and effect are also true on computer displays. See the figure below, and note how the image on the ClearType Display has more contrast in a normally lit room.

Image of ClearType display in a dark room and a normally lit room — Figure 1: The ClearType display reflects less light, and as a result has more contrast in lit environments. More contrast means easier for the viewer to see high detail. These are actual photographs taken from Surface RT against a popular tablet in the exact same lighting and display conditions.

High Contrast Display Uncovers Hidden Detail in Lit Environments

Just like how the city lights hide the stars in the night sky, light pollution including room-light reflections like glare, internal haze from the glass and touch screen, and multiple internal reflections from the glass and plastic components can all hide and reduce detail of the display and even washout its colors. The reflections are competing with the display information. The ClearType Display technology reduces this light pollution and as a result you will see more detail over any other device with lesser contrast. In a lit room just hold up a Surface, leave the display off, and hold up another tablet next to it and notice which one has the darker screen. This is your black point; your screen cannot go darker than in this situation. How do we accomplish this reduced reflections (glare and diffused) in Surface?

Opto-Mechanical Fusion of Touch and Display

Most touch screens have to throw a bunch of stuff in front of the screen in order to enable touch. This stuff primarily includes touch sensor layers (plastic, glass, or combinations of, and transparent conductors) and the top glass cover to protect it all. This is all good for features, but it does something bad to the display quality, it adds optical interfaces. At these interfaces haze, glare, and double and triple reflections all reduce image quality and perceived detail.

In the ClearType displays for Surface the touch layers, the cover glass, and the LCD itself are all completely fused together, this minimizes the optical interface to only one—the front glass. In addition, the transparent conductors for the touch sensor (which have a high index of refraction that introduces glare), are index matched out using various layers of coatings. You may have noticed on some touch screens there is almost a mirror-like finish on the screen; this absolutely reduces image quality, resulting from transparent conductors without optical matching layers. So the combination of one optical boundary, good management of the transparent conductors with index matching adjacent layers and picking materials with minimal haze results in a low reflectance, high contrast display. As figure 2 below shows, Surface’s display reflects less of the room light. This is why it is one of the highest contrast tablet displays on the market with lowest glare reflections at 5.8%.

Image of a traditional touch display, a popular tablet and ClearType display — Figure 2: Reduced optical boundaries, minimizing the effect of the transparent touch conductors, and low haze materials mean less glare and high contrast images.

Modulation Transfer Function, Cameras, and the Human Eye

So, as we can see, screen resolution is one component of perceived detail and image quality. Focusing on resolution while ignoring the other components will not yield the most optimal experience. The true measure of resolvability of a screen is called the Modulation Transfer Function (MTF). The MTF is a combination of both contrast and resolution. There are over a dozen subsystems that affect this MTF number. Just like the stars in the city night sky, without good contrast, perceived detail and image quality decreases.

When digital cameras started competing on megapixels, some cameras had/have better image quality than cameras with more pixels. How can that be? We all intuitively understand that if a camera lens is poor the resultant image will be poor (low MTF number) no matter how many pixels sits behind the lens. The combination of a good lens and a good image sensor results in a good, high quality MTF image. The same is true with displays.

Displays are like cameras but in reverse. The optics is partly in your eye and partly on the display. If the display reflects a lot of room light off the screen it will reduce the quality of the image (just like in a camera). And also like the camera lens, if your own eye has trouble focusing on something near (Presbyopia, something that will eventually affect us all), then very high resolution will not do much for you either.

Image quality is a highly multifaceted measurement which is far more complex than “resolution” or “d.p.i.”. The system is comprised of many parts which ultimately affect perceived detail and image quality, and the scientific way to measure this performance is with the MTF of each subcomponent. Some of these components include the content itself, encoding/decoding, how the pixel is rendered, the performance of the LCD, the electronics, the optics of the display itself, and even the performance of your own eyes.

Contrast Sensitivity of the Human Eye

Other than the ability to focus, the contrast sensitivity of the human eye has a huge influence on whether we are able to see the detail on the computer screen. The graph below shows that the eyes’ sensitivity to resolution, represented as dots per inch (d.p.i.), is not a constant relative to the contrast of that content. The Y axis is contrast sensitivity of the human eye plotted against spatial frequency (one can equate the x axis to a specific d.p.i. at a defined distance: cycles/deg). As d.p.i. at a specific distance increases, the eyes’ sensitivity drops drastically. This is somewhat intuitive in that it is harder to make out things as they get closer, and as they get closer together, they blur.

Figure 4 below is a visual representation of this contrast vs. d.p.i. effect shown in Figure 3. Here we are plotting a pattern with increasing frequency relative to the X-axis. On the Y axis we are varying the contrast of that pattern. Take a look at the graph and notice there is subtle upside down “U” shape, similar to the one in figure 3. Above this upside down “U“ shape your eye cannot make out that specific combination of contrast and resolution. Going above this upside down “U” is not necessary as it is not perceptible. BTW, as a side note, this upside down “U” shape changes the further it is held from your eye, this should tell you that this shape only exists in your mind’s eye. So as d.p.i. increases, the eye becomes more sensitive up to a certain d.p.i., but then the eyes sensitivity sharply drops off. This just means that when the resolution is very high it is really difficult to notice the high detail (if it is there). The just noticeable difference decreases as a function of resolution and contrast. These two graphs show that resolution and contrast are strongly linked, and inseparable. One cannot talk about one without the other, and these two metrics are what make up the MTF number.

Image of a visual representation of this contrast vs. d.p.i. effect — Figure 4: Visual representation of the contrast sensitivity of the human eye. Notice the curve. Above the curve is a combination of contrast and resolution that your eye will not detect.

Let’s go back to resolution. Westerink and Roufs (1989) measured the effect of resolution and image quality and developed a very interesting model (graph taken from Barten, 1999). Figure 5 shows that as resolution/d.p.i. increases the just noticeable difference decreases non-linearly. Intuitively this makes sense, the higher the resolution/d.p.i. of an image the more difficult it is to see the difference between the image at that resolution and at a slightly lower resolution. And this observation follows the contrast sensitivity graphs in figure 3 and 4. I have plotted two points on figure 5 for illustration. The y-axis on the right is in human perceptible units called “just noticeable difference” (JND). A unit of 1 JND is the minimum unit of noticeable difference for human. The left most red line is a 148 d.p.i. screen held at 15”, and the right most red line is a 260 d.p.i. screen also held at 15”. Notice that the x axis is logarithmic, so while the step in x axis is very large (large d.p.i. jump), the noticeable difference only corresponds to 3 units of human perception. The difference is noticeable but small relative to the jump in resolution. It is obvious that there is diminishing return on d.p.i.. Even displays under and around 150 d.p.i. (held at 15”) are near the plateau of human perception given specific conditions. Also note that Figure 5 is generated under ideal viewing conditions. Add glare, and low contrast to the equation and the differences all of sudden become negligible. All this says, in bright room a low-resolution high-contrast display will surely out do a higher-resolution lower-contrast display. Contrast and resolution are both important.

Image of a d.p.i. screen — Figure 5: Subjective image quality and "Just Noticeable Difference" plotted against cycles per degree (think of that as d.p.i. of a device held at a specific distance). The left most red line is a 148-d.p.i. screen held at 15 inches, and the right most red line is a 260-d.p.i. screen also held at 15 inches. Notice that the x axis is logarithmic so while the step in x axis is very large (large d.p.i. jump), the noticeable difference actually only corresponds to 3 units of human perception. One JND is the minimum unit of human perception. (Barten, 1999).

There is much more to be said about this topic, but for the sake of this paper, this is a good place to stop. Figure 3 and 4 explain why we cannot see some of the stars in the city night sky. The stars are small and the contrast is low. This combination hides detail. Just like on a computer screen, very high resolution content with lower contrast or high front reflections will hide detail and make two screens of different resolutions appear to look similar. These two figures link contrast and resolution, they are inseparable. One cannot talk about detail without speaking to both contrast and resolution, and this is called the MTF. Figure 5, shows that there is a diminishing return to increasing d.p.i., and that there is indeed a plateau of human perception. All these things need to be considered when designing a screen for a mobile computer.

Additional Benefits: Mechanical Strength and bringing you closer to the screen

In addition, the opto-mechanical fusion used for the Surface ClearType display not only reduces glare and improves image quality (higher MTF number), but also strengthens the whole assembly by mechanically coupling the cover glass, the LCD, the impact absorbing optical bond between the two, and the magnesium frame. This is why we can use Surface as a skateboard J or drop it without easily shattering the glass. This optical fusion also brings your finger closer to the image you are trying to touch (think of how a straw looks under water). The optical distance from your finger to Surface RT is 0.9 mm, that is some of the thinnest currently available for tablets. Mechanical strength and bringing you closer to the image are two additional and important benefits of fusing the optical stack of a display.

Designed for the Web, Designed for the Movies

Movies or web pages are rendered and created for a certain resolution and dots per inch (d.p.i.), and it is best to render that content as it was originally designed. Scaling the content up or down always reduces quality. Surface and Windows were designed around presenting content in its best and most original form. Currently, most of the web is designed for 1024x768 at 96 d.p.i.. And most digital movies are stored at 720P (1280x720 or 1388x768) with1080pslowly becoming more popular. On a higher d.p.i. and resolution screen, this content has to be scaled up; else it will often look too small. If not handled properly, images and text will often look more pixelated than if they were rendered in their original d.p.i. (see the two figures below). Some of you may have noticed that on a very high d.p.i. screen, some of those websites just don’t look like they should, this is the reason why.

Image of blurred content — Figure 6: Scaling introduces visual artifacts that may blur the original content.

Image of scaling image — Figure 7: Scaling artifacts on video going from 1:1 to 1:2.

When you dock a second application Surface with Windows RT, the small docked application occupies 342 pixels wide; this leaves 1024 pixels for the main application. If the main application is browsing web browser, the resultant resolution is 1024x768 ( 148 d.p.i.) at a 4:3 aspect ratio, the combination makes most of the web look good without being too small or needing to be rescaled. Of course, over time we expect this resolution to change, but the web is a very big place, so it will take some time.

Also, while watching a movie, everyone wants to see every bit of the screen the director intended. For that, the 16:9 screen on Surface ensures that most movies created in wide screen format will fill as much of the screen as possible for optimum viewing.

Kind of cool; user interface and hardware came together here. In one mode, 16:9 surface and windows was designed for movies and entertainment. And in a second mode, you have a screen and UI optimized for today’s web content.

Designed for you

The importance of reducing glare and improving contrast has a number of benefits beyond just a very nice looking high-quality detailed screen. There is a strong correlation between contrast and reading speed, blink rate, and eye strain (Legge and Foley Model see figure 8). The higher the contrast of the text, the faster you read, the less you blink, and the less strain on the eyes. So one can even say the Surface display helps you read faster.

Graphic of reading rate chart — Figure 8: Legge and Foley model that correlate contrast and reading speed. The higher the contrast of the text the faster you read, the less you blink, and the less eye strain.

Very high d.p.i. is not bad in the least, but there are diminishing returns and nothing comes for free, so it has to be balanced against the tradeoffs and perceived benefits (over the MTF of the system). Very high resolution decreases battery life, increases weight, increases heat, and increases charge time. Currently, a 270 d.p.i. display can consume as much as 30-40% more power than one at 210 d.p.i.. This means the battery has to be all that much thicker, heavier, hotter, and take all that more time to charge. Also if the content is not rendered at that d.p.i., like most of the Web, then it will either be small on the screen or it will be scaled up, which introduces artifacts that tend to reduce image quality. Balancing the gains of perceived detail, with what content format, and optimizing for the MTF number rather than just a single specification is the key to delivering a balanced design. Doing so ensures the device, software, and services deliver the best possible experience for you.

References

References for figures 3 and 5 are from “Contrast Sensitivity of the Human Eye and its effects on Image Quality” by P. G. J. Barten, SPIE Optical Engineering Press, 1999.