 |
Christian Schafleitner ,
Claudia Oster,
Michael Hurnaus,
Verena Lugmayr,
Jürgen Oberngruber
University of Applied Sciences
Hagenberg, Digital Media
www.intoi.net |
Design and development of rich user interfaces using .NET. This year’s winners of the user interface design competition at the Microsoft Imagine Cup present their project INTOI, a digital whiteboard solution. The article describes how this novel user interface was designed and implemented with .NET technologies.
Introduction
We are five master students studying Digital Media at the University of Applied Sciences Hagenberg, Austria. During last year we worked on the design and development of an interactive digital whiteboard solution called INTOI – Interchange of Ideas. As students we saw the potential of a combination of analogous boards with digital projection technology. After some brainstorming sessions we came up with several ideas on how to enhance the presentation and collaboration workflow using digital whiteboards. Using the Microsoft .NET technology enabled us to build a well-structured, high-performance application. INTOI won the user interface design competition at the worldwide Microsoft Imagine Cup 2007 in Korea.
Designing a whiteboard solution
While commonly available flipcharts and whiteboards are regularly used in schools and universities, presenters are still subject to several restrictions. Flipping between pages and adding additional sheets is not just cumbersome but also time consuming. Once a word is written or a picture is drawn, there is no simple way to move the drawing from the place where it appears. Although people often use PowerPoint presentations in combination with analogous flipcharts, there is still no simple way to combine both to a powerful presentation tool. However, the main restriction of analogous flipcharts and whiteboards is the limited size of the available drawing area.
INTOI – Interchange of Ideas is a hardware and software concept that serves as a hybrid between a digital flipchart and a digital whiteboard. Pen-based multi user interaction on a rear projection surface is quite similar to working on an analogous whiteboard. INTOI allows its users to easily write, draw and, additionally, load images, Microsoft PowerPoint presentations, PDF’s and videos. An easy-to-use menu system and a simple remote control enable users to easily change properties and switch between slides. Navigation on the infinitely sized pages is performed by simple hand gestures.
INTOI combines the benefits of ordinary flipcharts and whiteboards with the potential of today’s computer systems. Furthermore INTOI introduces a unique user interface which is easy to understand even for people without any computer experience. In contrast to comparable, commercially available solutions, INTOI’s hardware setup is inexpensive to manufacture and easy to set up.
Usability concept
Unlike classical 2D user interfaces, drop down menus, tool bars and scroll bars have been omitted in our approach. Emphasis was placed on designing a unique user interface that meets the requirements of an interactive whiteboard for users who are not familiar with any computer application. Using INTOI is very easy, since one can use it like a classical flipchart with enhanced functionality.
A donut-shaped menu can be opened by tapping the pen on the display surface to allow the user to quickly change properties, tools or settings. The menu appears at the exact position where it was called and disappears once the user starts writing or painting in the drawing area. Different menu types are color coded, so one immediately knows the type of the menu that is currently shown.
Loading content, such as images, PowerPoint presentations and stored INTOI-projects can be done by simply browsing through the media library using the Curve Dial : The users can scroll with their pen very intuitively by simply drawing circles to the left or the right instead of using scrollbars. The Curve Dial paradigm was also used to navigate on the infinite sized page.
Where design meets development
Different physical input devices, non-standard user interfaces and working with different types of media required a special system structure. The main problem has been the development of a flexible event system, which supports different kinds of inputs coming from pens or the network.
Since INTOI does not use any classical drop down menus, a further challenge was to build a menu system, which facilitates the use of different pens and supports the rendering of our own graphical styles. Furthermore a fast communication between the other parts of the application is necessary to keep the user experience smooth.
Smith, G., Schraefel, M., and Baudisch, P. Curve Dial: Eyes-free Parameter Entry for GUIs.
In CHI 2005 Extended Abstracts (demo paper), Portland, OR, Apr 2005, pp. 1146-1147.
The Event Dispatcher is responsible for the communication between the main clusters. It handles incoming events and distributes them to all registered consumers. The Device Manager is the interface to all the external modules of INTOI, such as pens and web services. The Data Manager cluster acts as data storage component and manages the interoperability with third party applications, such as Microsoft PowerPoint or Microsoft Word. The menu management component is responsible for keeping track of the menu structure and states.
Rendering is encapsulated in our rendering classes, to facilitate a later change of the rendering technology. Since WPF was in its infancy when the development of INTOI started, OpenGL was chosen to be the graphics rendering engine.
As already described we built a multi-user application, where multiple pens can draw at the same time on the whiteboard. Therefore Windows mouse events couldn’t be used and had to build our own input device management and event system. As soon as a pen connects to INTOI by touching the surface, a pen management object is created, which handles all events coming from this single pen. Each pen object holds all available tools and their settings. In addition each pen also needs recognizers which detect clicks/taps or analyze the strokes for the Curve Dial. If a click is detected, the pen object sends the event via our Event Dispatcher directly to the menu. Clicks and selections are sent to the Menu Manager, which is responsible for opening the correct menu. The change of settings is sent back via the event dispatcher to the pen object or one of the related tools. Other menu functions called by the pen can also affect the whole project, page or page objects. Through the event dispatching model messages can be sent directly to the part of the application where the information is needed.
Example:
The recognizer detects a double click, so an “open menu” event has been fired. The Menu Manager can immediately set the affected menu visible, so that the renderer renders it. A further click is detected by the click detector - the “single click” event is fired. This event can be interesting for both a menu and a widget. So the Menu Manager checks all opened menus if the click affects one of their items. The user selected another color, thus a “tool changed” event is fired back to the pen. Not the pen, but the tool itself listened to the event and changed its property. After that the user tries to save the file. Now the pen isn’t affected any longer, but the File/Project Management has to store the whole pages and data.
So far only the pens are mentioned as input devices, but there is also the possibility to use hand tracking, upload files to INTOI from a remote computer or get screenshots back. All the additional functionality can be added easily by adding a further “device” or service to the Device Manager. This can be done during the start up of the application at run time. Through the event model each of them can communicate with the application without creating too many dependencies.
Each service must implement a very simple interface, so that it can be started/stopped through the device manager. But within the implementation of a service there are no limits, e.g. the upload service runs a server to handle the incoming files. In contrary the service to use a mouse as a pen device just listens to the windows mouse events.
Let’s have a look what’s happening when a user wants to upload a Microsoft PowerPoint file from a remote PC: When the user drags a file to the upload area, a TCP connection is established to transfer the file to the INTOI application. The server within INTOI stores the file on the local hard drive. After the transfer an event is thrown to call the Widget Manager, which is responsible within one page to create/arrange or delete all types of widgets. The widget can then been created them same way as the pen would have invoked the creational process. In particular a PowerPoint widget is created. Therefore the Widget calls the PowerPoint Interoperability, which opens the file within PowerPoint, and extracts the pages to be presented within INTOI.
Having a clear structure already at design time simplified the design and development collaboration significantly. Right now we are working on additional external client applications supporting the audience of INTOI presentations. Those applications are being developed based on WPF using Microsoft Expression and Microsoft Visual Studio 2008. Having those technologies already one year ago would have simplified the design and development process enormously.
Conclusion:
The success of INTOI is due to the good collaboration between the UI designers and the developers. Unique and powerful graphical user interfaces are the key value of next generation software applications. Microsoft Expression Blend is focusing on improving the workflow between UI designers and programmers, while it extends the possibilities for rich user interfaces.
