Redesigning the Wearable Camera Pipeline to Detect Key Body Parts at Low Power

This paper addresses the problem of designing wearable devices suited for continuous mobile vision. Given power budgets, it is infeasible either to stream all video off board for analysis, or to perform all analysis on board. A compromise is to off-load only “interesting” frames for further analysis. We identify windows containing people and parts of their bodies as being interesting. We show that using a careful combination of emerging thermal sensors and ultra-lowpower coarse stereo enabled by modern low-power FPGAs, it is possible to detect the presence of these “key” body parts at well within wearable power budgets. We combine our (hardware) binary presence detector with a mobile CPU running a classifier to identify which body part is detected. Using this combination, we show how to detect faces, hands (of the video wearer) and bodies of those in the field of view at under 30mW; running on just the phone draws over 1W.