Sensing Techniques for Multi-Device Interfaces

As computing becomes more and more ubiquitous, there has been a trend for computing resources to become fragmented into small, specialized pieces: from mainframes, to personal computers, to an explosion of tablet computers, handheld devices, cell phones, pagers, wrist watches, and ear buds. Yet wireless networking acts as a medium that potentially can tie all of these pieces back together, enabling users to dynamically combine multiple devices to provide new capabilities that otherwise would not be possible. Jeff Pierce has a wonderful phrase for this vision of dynamically forming multi-device interfaces: opportunistic annexing [12]. In short, wireless networking will lead technology in entirely new directions that are difficult to foresee: in 100 years, I expect the phrase wireless networking will sound a lot like the phrase horseless carriage does to us now. This proposal contributes to workshop goal (2), enabling software and architectures, as well as

This proposal contributes to workshop goal (2), enabling software and architectures, as well as goal (3), the design of multi-device interfaces, in two significant ways. First we discuss the general strategy of recognizing synchronous gestures as an approach that can allow users to intuitively form connections (associations) between distributed mobile devices. Forming meaningful sets of devices is a key challenge for opportunistic annexing. Second, we discuss how sensors might be used to sense when and how the user is employing multiple devices, and to sense transitions between active use (when the user is attending to the device), versus disuse, when the user may be ignoring or only partially attending to the device. Sensing such transitions may be essential if we are to design multi-device systems that do not distract user’s from their current tasks.