The Microsoft SenseCam and Other Lifelogging Devices

The SenseCam is a personal, wearable camera developed by Microsoft Research in Cambridge, UK, and used as a lifelogging device in projects like MyLifeBits. Its use in applications like MLB and and the memory aid research for Alzheimer suffers at Addenbrooks hospital in the UK, is based on wearing the SenseCam for lifelogging of ‘events’ during your day, and generating a fast-forward movie of the event as the memory recall interface.

At Dublin City University we have been working with SenseCams for 15 months and our use of the device as a lifelogger is based on wearing it for the whole of your day (one of our users has over 1 million SenseCam images of his life). In the first part of this talk we will outline the work we are doing with SenseCam images, which begins with detecting the boundaries between events in a wearer’s day, for example moving from breakfast to travel to work, to sitting at a desk using a computer, to moving to a meeting, to having coffee with a colleague, to walking to another building, etc. To do this we use a variety of image-image similarities from global image features to regional SIFT features, combined with sensor readings from the SenseCam. Once event boundaries are detected, we then create a calendar of the wearer’s lifelog where s/he can browse his/her recorded lifelog. We allow a user to search for events similar to a given event, and in cases where a wearer is also carrying a GPS device we are able to populate the lifelog with other information culled from the internet such as images and place information. We are also starting work on automatic detection of (semantic) features from SenseCam images such as ‘in car’ or ‘outdoor’ or ‘people present’ with a view to categorising detected events into things like driving, shopping, in-meeeting, etc.

Our lifelogging work uses other recording devices besides the SenseCam, including off-the-shelf ones to take biometric readings, standard ones to record GPS and audio recorders, and RFID readers for objects tagged in the home or office. We also use some of our own wearable sensors which are the fabrics used in garments, and we can use these to measure any kind of body movement. We have further developed devices to measure the amount of bending or flexibility or the wearer which is useful for measuring things like back posture. The second part of the talk will include details of our work using these other lifelogging devices.

Speaker Details

Alan Smeaton is a full Professor of Computing at Dublin City University where he is Director of the Centre for Digital Video Processing, a University-designated research centre of 45 full-time researchers. He was Dean of the Faculty of Computing and Mathematical Sciences from 1998 to 2004 and was Head of the School of Computer Applications from January 1999 to December 2001. He holds the B.Sc., M.Sc. and PhD degrees in Computer Science from the National University of Ireland. His early research interests covered the application of natural language processing techniques to information retrieval (text) but this then broadened to cover the indexing and content-based retrieval of information in all media, text, image, audio and especially digital video and now the focus of his work is in information access for human digital memory applications. His major research funding is in the area of information analysis and access, particularly for digital video, and has also received funding for research in digital libraries, music IR and in web searching. His research has received funding from the European Union under the ESPRIT, LTR, Information Engineering, Language Engineering, VALUE, Libraries, FP5 and FP6 programmes as well as from national funding agencies (National Software Directorate, Forbairt/Enterprise Ireland Informatics Programme and Science Foundation Ireland) and from industry. With colleages from the Centre for Digital Processing, he currently holds research grants from Science Foundation Ireland (as PI for an SFI Investigator award), the Enterprise Ireland Technology Development fund 2003 (MediAssist project), the EU FP6 K-Space Network of Excellence, the EU FP6 Integrated Project aceMedia, the EU FP6 project MultiMatch, an Enterprise Ireland Basic Research Grant (GenIRL project with Stephen Blott), direct sponsorship and funding from Microsoft Research (Redmond) and with colleagues from University of Massachusetts and from State University of New York Buffalo, from Google (Mountain View, Calif.) as well as from two other multinational companies. Most of this work is in the area of analysis, indexing, searching, browsing and summarisation of information of all kinds. Most of Alan Smeaton’s research is carried out within the framework of the Adaptive Information cluster, an SFI-funded cluster of researchers in sensing technology, content extraction and information access, personalisation, agent technology, and middleware/infrastructure. Withn the AIC his work varies from key distribution on low-power sensor network nodes, to personality profiling for collaborative searching, and from efficient file structure design for large-scale information retrieval to managing up to a million images from a video diary. In 1994 Alan Smeaton was the chair of the 17th ACM SIGIR Conference which he hosted in Dublin and in 2001 he hosted an NSF-DELOS Workshop on Personalisation and Recommender Systems, also in Dublin. He was program co-chair of the ACM SIGIR Conference in Toronto in 2003 and general chair of the Conference on Image and Video Retrieval (CIVR) which he hosted in Dublin in 2004. In 2005 he hosted the European Summer School in Information Retrieval in Dublin. He has co-edited a book on Hypertext and Information Retrieval (Kluwer) and has published over 170 book chapters, journal and conference papers. He was an Associate Editor of the ACM Transactions on Information Systems for 8 years, and is a member of the Editorial Boards of Information Retrieval, Information Processing and Management, Foundations and Trends in Information Retrieval, the ACM Journal on Computers and Cultural Heritage and of the Journal on Digital Libraries. Professor Smeaton has graduated over twenty M.Sc. and PhD research students. He has acted as examiner for PhD theses from the Universities of Glasgow (three times), Sheffield, Ulster (twice), Sunderland, Trinity College Dublin (three times), RMIT Melbourne (twice), Tampere (Finland), ETH Zurich, City University London (twice), Southampton, Loughborough, Imperial College London, Universite Joseph Fourier (Grenoble) (twice), Robert Gordon University (Aberdeen), University of Surrey, University of Twente (NL) and the Australian National University. He has assisted the European Commission as an evaluator or reviewer in the ESPRIT, LRE, Language Engineering, Information Engineering, MLAP, LTR, INCO-Copernicus and IST FP5 and FP6 programmes as well as acting as a project reviewer for many specific projects. In 2002 Alan Smeaton was awarded the DCU President’s award for “outstanding research carried out in the field of Science and Engineering”. A slightly longer and more detailed CV can be found here. Alan Smeaton is a member of the ACM and the IEEE Computer Society and is a Fellow of the Irish Computer Society.

Date:
Speakers:
Alan Smeaton
Affiliation:
Dublin City University

    • Portrait of Jeff Running

      Jeff Running