{"id":566304,"date":"2019-02-05T08:09:35","date_gmt":"2019-02-05T16:09:35","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&p=566304"},"modified":"2019-02-05T16:05:30","modified_gmt":"2019-02-06T00:05:30","slug":"cross-device-taxonomy-survey-opportunities-and-challenges-of-interactions-spanning-across-multiple-devices","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/cross-device-taxonomy-survey-opportunities-and-challenges-of-interactions-spanning-across-multiple-devices\/","title":{"rendered":"Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices"},"content":{"rendered":"

Academic Collaboration<\/h4>\n

This survey is a collaboration between Microsoft Research, University College London Interaction Centre, Aarhus University, and Lancaster University. The paper will appear at ACM CHI 2019 and a preprint is available here<\/a>.<\/p>\n

Abstract<\/h2>\n
\n

Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the cross-device computing domain. For both new and experienced researchers in the field we provide: an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research.<\/p>\n

Taxonomy<\/h2>\n
\"Taxonomy

Taxonomy of cross-device design space dimensions: temporal, configuration, relationship, scale, dynamics and space.<\/p><\/div>\n

Dimension 1: Temporal.<\/strong> Cross-device work can be classified as either synchronous (when interactions happen at\u00a0the same time) or asynchronous (with a sequential flow of\u00a0interactions across devices). The large majority of the work\u00a0in our survey falls into the former category.<\/p>\n

Dimension 2: Configuration.<\/strong> This dimension classifies\u00a0the actual setup of the cross-device system as well as its\u00a0use of input and output modalities. The main categories\u00a0within synchronous use are mirrored and\u00a0 distributed user interfaces. Most active research is done within the distributed\u00a0UI category, investigating the spatial and logical distribution\u00a0of interfaces. The asynchronous work is divided across two\u00a0categories:\u00a0 interfaces that allow migration across devices,\u00a0and cross-platform research to make applications run consistently across diverse operating systems. Related taxonomies\u00a0align with this dimension, in particular Elmqvist\u2019s taxonomy\u00a0of distributed UIs and Rashid\u2019s focus on multi-device\u00a0attention switching.<\/p>\n

Dimension 3: Relationship.<\/strong> Research addresses different people-to-device relationships. While one person interacting with a single device (1..1) is usually not part of cross-device work, one person interacting with two or more devices\u00a0(1..m) covers work on cross-device workstations. Collaborative settings fall mostly in two categories: group activities where each person primarily uses a single device (1..1 x\u00a01..1), and collaborative settings with n-people and m-devices\u00a0(n..m). Examples of the last two categories relate closely to\u00a0research and studies done in the CSCW community.<\/p>\n

Dimension 4: Scale.<\/strong> Interactions can vary across the dimension of scale: from near, to personal, social, and public\u00a0rooms or buildings. Edward Hall\u2019s proxemics is a commonly\u00a0used anthropological lens for the scale of interactions,\u00a0which was later operationalised for cross-device work as\u00a0proxemic interactions. Scale dimension relates\u00a0closely to Terrenghi\u2019s taxonomy of display ecosystems across\u00a0scale and the progression of Weiser\u2019s Tab\/Pad\/Yard\u00a0computing.<\/p>\n

Dimension 5: Dynamics.<\/strong> Dynamics can vary between setups, and the categories of ad-hoc\/mobile, semi-fixed, and fixed\u00a0spaces closely relate to the phases of cross-device research\u00a0we introduced earlier. Fixed spaces often include larger-scale\u00a0wall displays and tabletops, while semi-fixed spaces allow a\u00a0certain degree of reconfigurability, and ad-hoc\/mobile spaces\u00a0focus on portable devices, allowing dynamic changes and\u00a0re-configuration.<\/p>\n

Dimension 6: Space.<\/strong> The last dimension differentiates between co-located and remote interactions (and corresponds\u00a0to Johnson\u2019s CSCW matrix). The large majority of cross-device work covers co-located scenarios, but few examples\u00a0address the challenges of providing cross-device interactions\u00a0across remotely distributed locations.<\/p>\n

Interaction Techniques<\/h2>\n
\"Overview

Overview of interaction techniques for cross-device computing.<\/p><\/div>\n

Ontology<\/h2>\n
\"Ontology

Ontology of cross-device research terminology.<\/p><\/div>\n

We untangle the diverse terminology of cross-device work and map terms out into a single ontology. The main goal of this synthesis is to provide better guidance about scope and specialisations of research within our field. The unified ontology is formed of two parts of key terms used to refer to cross-device literature.<\/p>\n

Top part: key term categories of cross-device sub disciplines. The nested categories begin at the bottom with dual-display and multi-monitor work (a), extend to work with multiple mobile devices, tablets or slates (b), further to the category of cross\/multi-display (c) and cross\/multi\/trans-surfaces (d), and finally to cross\/multi-device and distributed covering the broadest scope (e). The nested structure in the figure indicates focus areas associated with each term as well as the often included device form factors. It is important to note that a large subset of the terms are used interchangeably.<\/p>\n

Bottom part: a list of terms indicating focus areas of research projects (f): interactions and collaboration; interfaces; applications or systems; platform or middleware; environments or ecologies; and computing. We annotated these terms with the most common research focus of papers using each term.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Academic Collaboration This survey is a collaboration between Microsoft Research, University College London Interaction Centre, Aarhus University, and Lancaster University. The paper will appear at ACM CHI 2019 and a preprint is available here. Abstract Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_publishername":"ACM","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"ACM CHI 2019","msr_doi":"","msr_arxiv_id":"","msr_s2_paper_id":"","msr_mag_id":"","msr_pubmed_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_original_fields_of_study":"","msr_release_tracker_id":"","msr_s2_match_type":"","msr_citation_count_updated":"","msr_published_date":"2019-5-1","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_s2_pdf_url":"","msr_year":0,"msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_match_confidence":0,"msr_microsoftintellectualproperty":true,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13554],"msr-publication-type":[193716],"msr-publisher":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-566304","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-human-computer-interaction","msr-locale-en_us"],"msr_publishername":"ACM","msr_edition":"","msr_affiliation":"","msr_published_date":"2019-5-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/02\/brudy_holz_raedle_wu_houben_klokmose_marquardt-Cross-Device_Taxonomy-Survey_Opportunities_and_Challenges_of_Interactions_Spanning_Across_Multiple_Devices.pdf","id":"566307","title":"brudy_holz_raedle_wu_houben_klokmose_marquardt-cross-device_taxonomy-survey_opportunities_and_challenges_of_interactions_spanning_across_multiple_devices","label_id":"243109","label":0},{"type":"doi","viewUrl":"false","id":"false","title":"10.1145\/3290605.3300792","label_id":"243106","label":0}],"msr_related_uploader":"","msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[{"id":566307,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2019\/02\/brudy_holz_raedle_wu_houben_klokmose_marquardt-Cross-Device_Taxonomy-Survey_Opportunities_and_Challenges_of_Interactions_Spanning_Across_Multiple_Devices.pdf"}],"msr-author-ordering":[{"type":"text","value":"Frederik Brudy","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Christian Holz","user_id":31415,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Christian Holz"},{"type":"text","value":"Roman R\u00e4dle","user_id":0,"rest_url":false},{"type":"text","value":"Chi-Jui Wu","user_id":0,"rest_url":false},{"type":"text","value":"Steven Houben","user_id":0,"rest_url":false},{"type":"text","value":"Clemens Klokmose","user_id":0,"rest_url":false},{"type":"text","value":"Nicolai Marquardt","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[371909,396845],"msr_project":[],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"inproceedings","related_content":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/566304","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":14,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/566304\/revisions"}],"predecessor-version":[{"id":566478,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/566304\/revisions\/566478"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=566304"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=566304"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=566304"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=566304"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=566304"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=566304"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=566304"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=566304"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=566304"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=566304"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=566304"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=566304"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=566304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}