{"id":171348,"date":"2014-04-26T08:23:42","date_gmt":"2014-04-26T08:23:42","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/project\/type-hover-swipe-in-96-bytes-a-motion-sensing-mechanical-keyboard\/"},"modified":"2017-06-20T11:02:39","modified_gmt":"2017-06-20T18:02:39","slug":"type-hover-swipe-in-96-bytes-a-motion-sensing-mechanical-keyboard","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/type-hover-swipe-in-96-bytes-a-motion-sensing-mechanical-keyboard\/","title":{"rendered":"Type\u2013Hover\u2013Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard"},"content":{"rendered":"<p class=\"asset-content\"><!-- .asset-content -->We present a new type of augmented mechanical keyboard, sensing rich and expressive motion gestures performed both on and directly above the device. A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures. We propose the use of motion signatures a technique that utilizes pairs of motion history images and a random forest classifier to robustly recognize a large set of motion gestures. Our technique achieves a mean per-frame classification accuracy of 75:6% in leave\u2013one\u2013subject\u2013out and 89:9% in half-test\/half-training cross-validation. We detail hardware and gesture recognition algorithm, provide accuracy results, and demonstrate a large set of gestures designed to be performed with the device. We conclude with qualitative feedback from users, discussion of limitations and areas for future work.<\/p>\n<p><iframe loading=\"lazy\" title=\"Type-Hover-Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/Y3dUeGNIX4M?feature=oembed&rel=0\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<h1>Image Gallery<\/h1>\n<ul id='gallery-1' class='gallery galleryid-171348 gallery-columns-4 gallery-size-grid-third stripped ms-row fixed-small'><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000001.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000001.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000001-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000002.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000002.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000002-480x280.jpg\" alt=\"a close up of text on a black background\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000003.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000003.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000003-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000004.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000004.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000004-480x248.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><br style=\"clear: both\" \/><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000005.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000005.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000005-480x280.jpg\" alt=\"a close up of a computer keyboard\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000006.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000006.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000006-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000007.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000007.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000007-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000008.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000008.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000008-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><br style=\"clear: both\" \/><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000009.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000009.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000009-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000010.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000010.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000010-480x280.jpg\" alt=\"a close up of a keyboard\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000011.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000011.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000011-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><li class='s-col-6-24 xs-margin-bottom-sp1 s-margin-bottom-sp2'><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000012.jpg\" data-mfp-src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000012.jpg\" data-caption=\"\" class=\"gallery-item\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2014\/04\/typehoverswipe_214560-000012-480x280.jpg\" alt=\"\" class=\"db full-width\" \/><\/a><\/li><br style=\"clear: both\" \/>\n\t\t<\/ul>\n\n","protected":false},"excerpt":{"rendered":"<p>We present a new type of augmented mechanical keyboard, sensing rich and expressive motion gestures performed both on and directly above the device. A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13554],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-171348","msr-project","type-msr-project","status-publish","hentry","msr-research-area-human-computer-interaction","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"2014-04-26","related-publications":[166524],"related-downloads":[],"related-videos":[190856],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171348","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171348\/revisions"}],"predecessor-version":[{"id":392105,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171348\/revisions\/392105"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=171348"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=171348"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=171348"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=171348"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=171348"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}