Kinect

Kinect™ for Windows® Software Development Kit (SDK) from Microsoft Research, a free beta release for noncommercial applications is avaialble. The SDK is designed to empower a growing community of developers, academic researchers and enthusiasts to create new experiences that include depth sensing, human motion tracking, and voice and object recognition using Kinect technology on Windows 7. The Kinect for Windows SDK can be downloaded today at no cost for development of noncommercial applications at http://research.microsoft.com/kinectsdk. We are looking forward to innovative ways of use of Kincet on following topics.

[Sub-Topics]
[1] Sign language recognition and translation based on Kinect

The aim of sign language recognition is to provide an efficient and accurate way to connect the hearing impaired community to our daily life. Additionally, we hope to connect the sign language recognition with machine translation to enable the smooth communication across language barriers. In addition, the result of this research can be applied to the teaching and education for the speaking and listening disabled people worldwide. It will bring unprecedented user experience to this kind of people powered by Kinect. This research will largely benefit from the great technologies of Kinect such as image processing, computer, vision, motion trace, gesture recognition, speech recognition, text-to-speech. The developed sign language recognition and translation technology will demonstrate the huge potential of Kinect.

[2] Combing Kinect and other devices to enable new interaction possibilities

Kinect provides an easy way to sense 3D information as well as body movements. Combined with other novel input/output devices, this may enable interesting new interaction possibilities (e.g. around 3D, tabletop, mobile, or projection-based interactions).

[3] Supporting Creativity with Kinect

Kinect is capable of capturing gestures, body movements, as well as 3D geometry of physical objects/environments. This might support a whole new vocabulary for people to explore and express their creativity. Creativity-support tools that leverage the full capabilities of Kinect would be a very rich space to explore.

[4] Kinect based natural 3D modeling interface

Kinect camera provides opportunities to create new 3D modeling interface. We plan to investigate how to integrate Kinect into the 3D modeling pipeline to replace the traditional mouse or glove based interface. This new interface will let user to control the 3D modeling through gesture. Application scenario may include car modeling, 3D assembly, and so forth.
Also Kinect camera is a powerful platform to create many new applications, include aesthetic computing oriented user interaction. We will explore how to use Kinect to create the computer aesthetic works, such as digital painting.

[5] Kinect based interaction for large scale data visualization

The large display wall and immersive environment is the next generation visualization setup for user to explore the large scale dataset. However, traditional user interactions (mouse + keyboard) is not efficient anymore in this scenario for navigation and data investigation. In this project, we will study how to combine the kinect and large display wall in Peking university for user to easily navigate and interact with the large data set, explore the underline connections between the data, and easily visualize the data. This kinect based solution will replace the traditional glove and hand tracking system used in the system.

[6] Kinect based human tracking and environment surveillance.

Traditional RGB camera is widely used in vision systems for object tracking and surveillance. The Kinect camera provides another robust and cheap solution for these tasks. Combined with high res RGB camera, the new tracking and surveillance system can be developed and applied in many applications. The large display wall and immersive environment is the next generation visualization setup for user to explore the large scale dataset. However, traditional user interactions (mouse + keyboard) is not efficient anymore in this scenario for navigation and data investigation. In this project, we will study how to combine the Kinect and large display wall in Peking university for user to easily navigate and interact with the large data set, explore the underline connections between the data, and easily visualize the data. This kinect based solution will replace the traditional glove and hand tracking system used in the system.

[7] Speech recognition

1) Microphone array based (like the 4 micophones in the Kinect) acoustic signal processing for speech enhancement, hands-free speech recognition, sound-source localization, voice-activity detection (VAD) applications.
2) Kinect based Avatar for 3D talking head, either in an animation or photo-realistic form with facial expressions rendering (synthesis), morphing and recognition.
3) Robust speech recognition for distant talking with Kinect

[8] eHealth application with Kinect

We focus on automatic detection of epilepsy and seizure events from patient video data. To support efficient modeling of the human body, there is a need to segment the patient from the background. This is a particularly difficult task due to challenges such as unconstrained poses of the patient, and interference from shadows and dynamically changing scenes where caregivers can enter the camera view. Here, we believe a combination of color and depth information could be valuable to help separate a patient from the cluttered background. Additionally, given that depth information offers more diverse segmentation cues as compared to widely used shape and appearance based data, we believe the fusion of these complementary features can yield improved segmentation results. We wish to investigate how input data from the Kinect camera can elevate scene monitoring performance in hospital settings.