Portrait of Desney Tan

Desney Tan

Senior Director, Medical Devices Group

About

I am Senior Director of the Microsoft Medical Devices Group in Redmond, Washington. I also hold an affiliate faculty appointment in the Department of Computer Science and Engineering at the University of Washington.

My research interests include Human-Computer Interaction, Mobile Computing, and Healthcare. I spend large chunks of my time building devices that digitize the world and human body in interesting ways and then applying signal processing and machine learning to make sense of it all. My work has been shipped in various versions of Windows and Office, the Microsoft Handwriting Recognition Engine and Tablet Input Panel, Xbox Kinect, and most recently the Microsoft Band.

I received my Bachelor of Science in Computer Engineering from the University of Notre Dame in 1996, after which I spent a couple of years building bridges and blowing things up in the Singapore Armed Forces. I later returned to Carnegie Mellon University, where I worked with Randy Pausch and earned my PhD in Computer Science in 2004.

I was honored as one of MIT Technology Review’s 2007 Young Innovators Under 35 for my work on brain-computer interfaces and a 2012 Kavli Fellow by the US National Academy of Sciences. I was named one of SciFi Channel’s Young Visionaries at TED 2009, as well as  Forbes’ Revolutionaries: Radical Thinkers and their World-Changing Ideas for my work on Whole Body Computing. I have served as Technical Program Chair for CHI 2008 as well as General Chair for CHI 2011. I sit on multiple journal editorial boards and serve as Editor-in-Chief for Foundations and Trends in HCI as well as the Springer HCI Series.

I am amazingly fortunate to have a family that supports (or at least tolerates) my crazy work habits, and it is great that my wife Angela and kids Ayden and Zachary are always there when I come up for air.

View my CV.

Projects

Enhancing Input On and Above the Interactive Surface with Muscle Sensing

Current interactive surfaces provide little or no in-formation about which fingers are touching the surface, the amount of pressure exerted, or gestures that occur when not in contact with the surface. These limitations constrain the interaction vocabulary available to interactive surface systems. In our work, we extend the surface interaction space by using muscle sensing to provide complementary information about finger movement and posture. In this paper, we describe a novel system that combines muscle…

SoundWave: Using the Doppler Effect to Sense Gestures

Established: November 28, 2016

Gesture is becoming an increasingly popular means of interacting with computers. However, it is still relatively costly to deploy robust gesture recognition sensors in existing mobile platforms. We present SoundWave, a technique that leverages the speaker and microphone already embedded in most commodity devices to sense in-air gestures around the device. To do this, we generate an inaudible tone, which gets frequency-shifted when it reflects off moving objects like the hand. We measure this shift…

Skinput: Appropriating the Body as an Input Surface

Established: November 28, 2016

We present Skinput, a technology that appropriates the human body for acoustic transmission, allowing the skin to be used as an input surface. In particular, we resolve the location of finger taps on the arm and hand by analyzing mechanical vibrations that propagate through the body. We collect these signals using a novel array of sensors worn as an armband. This approach provides an always available, naturally portable, and on-body finger input system. We assess…

Humantenna: Sensing Gestures Using the Body as an Antenna

Computer vision and inertial measurement have made it possible for people to interact with computers using whole-body gestures. Although there has been rapid growth in the uses and applications of these systems, their ubiquity has been limited by the high cost of heavily instrumenting either the environment or the user. In this paper, we use the human body as an antenna for sensing whole-body gestures. Such an approach requires no instrumentation to the environment, and…

Muscle-Computer Interfaces (muCIs)

Established: November 28, 2016

Many human-computer interaction technologies are currently mediated by physical transducers such as mice, keyboards, pens, dials, and touch-sensitive surfaces. While these transducers have enabled powerful interaction paradigms and leverage our human expertise in interacting with physical objects, they tether computation to a physical artifact that has to be within reach of the user. As computing and displays begin to integrate more seamlessly into our environment and are used in situations where the user is not…

Always-Available Mobile Interfaces

We have continually evolved computing to not only be more efficient, but also more accessible, more of the time (and place), and to more people. We have progressed from batch computing with punch cards, to interactive command line systems, to mouse-based graphical user interfaces, and more recently to mobile computing. Each of these paradigm shifts has drastically changed the way we use technology for work and life, often in unpredictable and profound ways. With the…

AirWave: Non-Contact Haptic Feedback Using Air Vortex Rings

Input modalities such as speech and gesture allow users to interact with computers without holding or touching a physical device, thus enabling at-a-distance interaction. It remains an open problem, however, to incorporate haptic feedback into such interaction. In this work, we explore the use of air vortex rings for this purpose. Unlike standard jets of air, which are turbulent and dissipate quickly, vortex rings can be focused to travel several meters and impart perceptible feedback.…

Patient-Friendly Medical Information Displays

Patients’ basic understanding of clinical events has been shown to dramatically improve patient care. Unfortunately, patients are frequently under-informed and unclear about our own hospital/clinical courses. The recent emergence of Electronic Medical Records (EMRs) and Personal Health Records (PHRs) makes vast amounts of data available to patients, but does little to help patients understand that data. Our work focuses on designing and building simplified information displays that will help patients understand our medical treatment and…

InkSeine

Established: December 8, 2008

This page is an archive of the InkSeine project's web page. The project is no longer active. The prototype is no longer maintained and does not work on Windows 10. If you have a tablet running Windows 7 or Windows Vista, InkSeine should still work on those platforms. It also has dependencies on older versions of Office, such as for interop with Microsoft Outlook. For those of you who were fans of InkSeine, we are…

Publications

2016

Health-OS: A Position Paper
Darko Kirovski, Nuria Oliver, Mike Sinclair, Desney Tan, in Poster at the 1st International Workshop on Systems and Networking Support for Healthcare and Assisted Living Environments (HealthNet07), January 23, 2016, View abstract, Download PDF

2015

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

1999

1996

1995

Projects

Link description

Hot Topics

Date

July 14, 2014

Speakers

Desney Tan, Doug Burger, Krysta M. Svore, and Mary Gray

Affiliation

Microsoft Research

Other

[P…] >30 inventions patent-pending

[P.14] Supporting User Multitasking with Clipping Lists
Tara Matthews, George Robertson, Mary Czerwinski, Desney Tan
United States Patent #8108785, granted 31 Jan 2012

[P.13] Activity Centric Granular Application Functionality
Steven Macbeth, Roland Fernandez, Brian Meyers, Desney S Tan, George Robertson, Nuria Oliver, Oscar Murillo, Elin Pedersen
United States Patent #7970637, granted 28 Jun 2011

[P.12] Activity Centric Domain Scoping
Steven Macbeth, Roland Fernandez, Brian Meyers, Desney S Tan, George Robertson, Nuria Oliver, Oscar Murillo
United States Patent #7836002, granted 16 Nov 2010

[P.11] Creating and Managing Activity-Centric Workflow
Steven Macbeth, Roland Fernandez, Brian Meyers, Desney S Tan, George Robertson, Nuria Oliver, Oscar Murillo, Elin Pedersen, Mary Czerwinski
United States Patent #7761393, granted 20 July 2010

[P.10] Constraint-based Correction of Handwriting Recognition Errors
Michael Shilman, Desney S Tan, Patrice Simard, Mary Czerwinski
United States Patent #7720316, granted 18 May 2010

[P.9] In-Situ Search for Active Note Taking
Kenneth Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman, Desney S Tan
United States Patent #7693842, granted 6 April 2010

[P.8] Accessing 2D Graphic Content Using Axonometric Layer Views
George Robertson, Daniel Robbins, Desney S Tan, Kenneth Hinckley, Maneesh Agrawala, Mary Czerwinski, Patrick Baudisch, Gonzalo Ramos
United States Patent #7663620, granted 16 February 2010

[P.7] Resource Availability for User Activities Across Devices
Steven Macbeth, Roland Fernandez, Brian Meyers, Desney S Tan, George Robertson, Nuria Oliver, Oscar Murillo, Mary Czerwinski
United States Patent #7620610, granted 17 November 2009

[P.6] Using Electroencephalograph Signals for Task Classification and Activity Recognition
Desney S Tan, Johnny Lee
United States Patent #7580742, granted 25 August 2009

[P.5] Displaying 2D Graphic Content using Depth Wells
George Robertson, Daniel Robbins, Desney S Tan, Ken Hinckley, Maneesh Agrawala, Mary Czerwinski, Patrick Baudisch, Gonzalo Ramos
United States Patent #7523405, granted 21 April 2009

[P.4] Managing Arbitrary Window Regions for More Effective Use of Screen Space
Brian Meyers, Desney S Tan, George Robertson, Mary Czerwinski
United States Patent #7487454, granted 3 February 2009

[P.3] Real-time Preview for Panoramic Images
Patrick Baudisch, Chris Pal, Eric Rudolph, Drew Steedly, Richard Szeliski, Desney S Tan, Matthew Uyttendaele
United States Patent #7424218, granted 9 September 2008

[P.2] Spy-Resistant Keyboard
Desney S Tan, Pedram Keyani, Mary Czerwinski
United States Patent #7296233 granted 13 November 2007 & #7395506 granted 1 July 2008

[P.1] 3D Navigation Techniques
George Robertson, Mary Czerwinski, Desney S Tan
United States Patent #6987512, granted 17 January 2006 & #7148892 12 December 2006

Press

[Pr.47] Local Scientists Develop Contact Lenses to Help Diabetics
International Examiner, 15 February 2012

[Pr.46] Information-Rich Eyeballs? Talking to Microsoft’s Desney Tan about the Functional Contact Lens
AsweetLife, 12 January 2012

[Pr.45] Contact Lens Monitors Blood Sugar
Discovery News, 7 January 2012

[Pr.44] Microsoft Developing Electronic Contact Lens to Monitor Blood Sugar
Gizmag, 5 January 2012

[Pr.43] The Good, The Bad and The Stupid of 2011
Seattle Business Magazine, December 2011

[Pr.42] 自由亚洲电台人物专访:对话微软研究院高级研究员
Radio Free Asia, 2 November 2011

[Pr.41] Desney Tan and Scott Saponas: Your Body is a Wonder, Man
PopTech Blog, 22 October 2011

[Pr.40] Better Typing While Walking, and other Cool Stuff from UW
GeekWire, 20 October 2011

[Pr.39] Remote Control, With a Wave of a Hand
The New York Times, 10 September 2011

[Pr.38] How to Make a Human Antenna
Discovery News, 12 May 2011

[Pr.37] Turn your entire home into a game controller
New Scientist, 10 May 2011

[Pr.36] Talking to the Wall
Technology Review, 3 May 2011

[Pr.35] Sensors turn skin into gadget control pad
BBC News, 26 March 2010

[Pr.34] Skinput Turns Your Arm into a Touch-Screen
Wired, 3 March 2010

[Pr.33] Skinput Turns Any Bodily Surface Into a Touch Interface
Popular Science, 3 March 2010

[Pr.32] ‘Skinput’ Turns Your Body Into Touchscreen Interface
TechNews, 3 March 2010

[Pr.31] Body acoustics can turn your arm into a touchscreen
New Scientist, 1 March 2010

[Pr.30] Microsoft tapping current product line for personal health IT
ZDNet, 10 February 2010

[Pr.29] Microsoft E-health Research Taps Xbox, Mobile Phones
PCWorld, 9 February 2010

[Pr.28] 人机交互 那些触手可及的梦想 (HCI and the Touchable Dreams)
Chinese Popular Computer Weekly, 19 November 2009

[Pr.27] 人机交互:一个被忽视的重要领域 (HCI: An Important Field that is Overlooked)
Science Times, 19 November 2009

[Pr.26] 用舌头玩转俄罗斯方块 (Using the Tongue to Control Computers and Play Games)
sina, 10 November 2009

[Pr.25] Muscle-Bound Computer Interface
Technology Review, 28 October 2009

[Pr.24] The Revolutionaries: Radical Thinkers and their World-Changing Ideas – Desney Tan
Forbes, 22 September 2009

[Pr.23] Technology Shows What’s on a Customer’s Mind
Financial Times, 16 September 2009

[Pr.22] Inside These Lenses, a Digital Dimension
The New York Times, 25 April 2009

[Pr.21] Young Visionaries
SciFi Channel Visions of Tomorrow, 5 February 2009

[Pr.20] The Interface: Gestures will Force the Mouse into Retirement
Financial Times, 16 September 2008

[Pr.19] Microsoft Research: How We Watch the Computer and How it Watches Us
Bill Crounse’s HealthBlog, 18 August 2008

[Pr.18] Collabio Game Explores Social-Network Data Mining… And Social Psychology
Microsoft Research News and Highlights, August 2008
TechCrunch, August 2008

[Pr.17] My Awesome IT Job: Researcher and Manager, Microsoft
ZDNet, 25 July 2008

[Pr.16] Upgrading Humans
Discovery Channel, 16 June 2008

[Pr.15] High-tech Armband Puts your Fingers in Control
New Scientist, 24 April 2008

[Pr.14] Giving Back to the Machines
digit, 22 January 2008

[Pr.13] Microsoft’s Investigation into the Subconscious
MarketWatch from DowJones, 18 January 2008

[Pr.12] The Year in Hardware
MIT Technology Review, 26 December 2007

[Pr.11] I Spy with my AI, Something Beginning with T(hought)!
Information Technology Magazine, 1 October 2007 [pdf]

[Pr.10] Work on Paralysis Shows Just How Hard it is to Read Minds
Wall Street Journal, 26 September 2007

[Pr.9] Teaching Computers to Read Minds
MIT Technology Review, 15 August 2007

[Pr.8] Human-Aided Computing: Microsoft Researchers are Trying to Harness Untapped Brain Power
MIT Technology Review, 22 June 2007

[Pr.7] Computer Will Try to Adjust to Your Moods: This is your Brain Plugged In
Puget Sound Business Journal, 19 May 2006

[Pr.6] Software Notebook: Helping Users Peer into the Minds of Users
Seattle Post-Intelligencer, 13 March 2006

[Pr.5] Microsoft Research Nurtures Top Tech Talent with Student Fellowships
Microsoft PressPass, 6 March 2006

[Pr.4] Microsoft Research Interns Make Their Mark in Computer Usability at CHI 2003 Conference
Microsoft PressPass, 27 March 2003

[Pr.3] Large Displays Enhance Optical Flow and Close the Gender Gap in 3D Virtual Navigation
Microsoft PressPass, 3 April 2003
EurekAlert, 15 April 2003
New Scientist, 17 April 2003
New York Times, 21 April 2003 (requires subscription)
Subsequently covered by: Carnegie Mellon Newsclips, Yahoo.com, UK Yahoo.com, USA Today, Slashdot, bizInk, Morningstar, Stockwatch, WomenGamers.com, Creativepro.com, Bakersfield Channel, Milwaukee Channel, WRAL, Channel 4000, Boston Channel, SciGuy Science News, ScienceTicker, Kensei News, Mainichi Interactive, Ikson Portal, Hindustan Times, Centro Paulista, Dagbladet, BN DeStem, Vjesnik Online, Nkzagreb Online

[Pr.2] Foreign Students Fear INS is Targeting Them
Pittsburgh Post-Gazette, 15 February 2003
Subsequently covered by: De Virtutibus blog

[Pr.1] Students Scoff at Paying Internet Piper
Pittsburgh Post-Gazette, 1 June 2003