Microsoft Research Blog

Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Uncanny Valley and the Sense of Touch

April 18, 2018 | By Microsoft blog editor

The notion of an “uncanny valley” is well known in robotics[1]. This is when increasing the realism of a robot’s human-like appearance can lead to feelings of unease—or even revulsion—as a robot becomes more and more (but never quite fully) human-like[2].

While this uncanny valley is related to sight, researchers wondered if an uncanny valley experience also exists for the sense of touch—also known as haptics.

In a paper recently published in Science Robotics[3], a group of Microsoft researchers put forward a new theory of haptics interaction that identifies an uncanny valley for the sense of touch. Their research suggests that when the haptics are not aligned with the rest of the sensory stimulation, it can hinder the experience. The researchers believe that this research will create a new school of thought among designers of touch-enabled interactions and devices.

Researchers Mar Gonzalez Franco, Eyal Ofek, Ken Hinckley and Christopher Berger, a visiting researcher from California Institute of Technology, chose the platform of virtual reality (VR) as a testbed. VR is a great tool for revealing the strange and otherworldly foundation of our everyday sensory perceptions. We can use VR to produce a sensation of touch that feels like it originates in the completely empty space between one’s hands, while holding two controllers.

Each controller provides the user with synchronous vibrotactile stimuli of different amplitudes. In the virtual environment, we show the participant a wooden dowel that they grip with each hand. The dowel is only imaginary, of course; the hands are not physically linked and the dowel does not exist. We then show a small white marble that seems to knock on the dowel at different locations as we vary the vibratory sensations.

Spatialized Touch Illusion

Figure 1: Top: A diagram showing the stimulation points for the spatialized touch generation. Middle: The participant holding the controllers. Bottom: First-person perspective view of their virtual hands and the stick. The amorphous cloud shown here reduces the uncanny valley effect.

What’s interesting is that not only is the dowel imaginary, but also that the apparent spatial location of the touch sensation of the marble will persist, even if the marble is invisible. This is because the brain follows the unity rule by which two synchronous events that are connected in time should also be connected in space, thus they have the same cause and origin. In this case, the two vibrations felt in the hands are perceived to be the result of a single touch that comes from the same location along the virtual dowel, no matter if the marble is visible or not.

In virtual environments, producing higher-fidelity haptic sensations of this sort come with the assumption that such improvements will always yield a more realistic and immersive experience. However, this research found that adding the spatialized touch on its own, without the visual counterpart of the marble, can greatly reduce the perceived realism of and immersion into the virtual environment. That is despite the fact that participants were able to locate the touch sensation equally well when the marble was invisible or visible.

The video shows the newly found uncanny valley, where an increased quality of haptics generates a drop in the subjective experience. However, this uncanny valley can be shifted or eliminated by subtly manipulating the experimental conditions. The researchers found that by rebalancing the ray of the sensorial experience, particularly by increasing the visual quality to provide a plausible source for the haptic experience, a fuller sense of immersion returns and increases the overall subjective experience.

Likewise, the sensation can be manipulated based on whether we are simply holding the virtual dowel—thus experiencing the knock of the marble as a passive participant—or actively moving the dowel up and down, thus becoming the agent of the sensation through our own actions, which again increases the sense of realism and immersion.

Understanding these effects more deeply helps us build better and more convincing virtual environments. The result is a key insight that lends a deeper understanding of how human perception really works and offers a boost to future endeavors that can truly increase the sensory realism of all of our would-be virtual dreams. The researchers believe that this demonstration of the newly found uncanny valley of haptics, and additional insights on how to overcome the haptic touch revulsion, are critical not only in VR controllers and robotic teleoperation, but also across all types of touch-enabled devices.

[1] Mori, M. Translated by MacDorman, K. F. and Kageki, Norri. The uncanny valley. IEEE Robotics and Automation. 2012.
[2] Chattopadhyay, D., & MacDorman, K. F. Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley. Journal of Vision, 16. 2016.
[3] Berger, C.C., Gonzalez-Franco, M., Ofek, E. & Hinckley, K., The uncanny valley of haptics. Science Robotics. 2018.

Up Next

Human-computer interaction

Touching the Virtual: How Microsoft Research is Making Virtual Reality Tangible

“Ask yourself how you really want to interact with virtual objects? The simple answer is that we want to handle them as if they are real, just reach out a hand to grasp them, pick them up, feel what they’re made of, and do all that in a natural way that requires no learning.” said […]

Microsoft blog editor

Computer vision, Graphics and multimedia, Human-computer interaction

Typing in the Virtual Office Environment: Examining the Effects of Keyboard and Hand Representation in VR

Much of the hype around Virtual Reality, or VR, has focused on immersive gaming and entertainment, but the technology also has profound implications for enhancing the way we work. Imagine the perfect office environment: multiple large displays, proper lighting, free from distractions and isolated from outside disturbances. VR can be used to generate the ideal virtual office—even from the confines of an airplane seat or a tiny touchdown space.

Eyal Ofek

Senior Researcher

CHI Part 1: Interacting through touch, virtually and physically

Human-computer interaction

CHI Part 1: Interacting through touch, virtually and physically

A preview of the ACM CHI Conference on Human Factors in Computing Systems By Justin Cranshaw, Researcher, Microsoft Research Human-computer interaction (HCI) research at Microsoft provides a human-centered perspective on the future of computing, reshaping the roles computers play in our lives and propelling society forward through the tools, systems, and methods we investigate and […]

Microsoft blog editor