About

I am a Researcher at Microsoft Redmond Labs since 2013. Before that I finished my Ph.D. in Electrical Engineering at Princeton University, where I was advised by Profs. Niraj Jha and Naveen Verma. Even before that I completed my B. Tech. and M. Tech. Dual Degree in Electrical Engineering from IIT Madras, India.

I build computing systems to perceive human state that is expressed via spoken and body language. This work has led me to develop algorithms that analyze physiology, recognize gestures, activity, speech, pose etc. An important part of this research is the development and optimization of machine-learning algorithms such that they can be run on resource-constrained platforms.

At Microsoft, I have developed signal-processing and machine-learning algorithms to perceive human state within AR-VR and IoT applications. Specifically, I have formulated methods to process video on the edge for activity monitoring, track pose with wearables, detect gestures with multi-modal sensors, and analyze audio and speech data in diverse environments. My research has been leveraged by product teams in Azure, Windows and Windows Mobile. Take a look at my projects page for more information.

Research Interests:

Sparse and distributed signal processing
Resource constrained machine learning
Wearable devices and embedded systems
Processing of multimedia, radio and physiological signals

Projects

Publications

Videos

Downloads

Other

Work at Microsoft

I use Python and C# for most of my research. The following is what keeps me busy these days:

  1. Processing multi-modal sensor data to build better I/O interfaces in AR-VR devices
  2. Developing data-driven approaches to improve multimedia quality, analytics and retrieval

Take a look at my projects page for more details.


Interns and Collaborators

2018

  • Benjamin Elizalde, CMU Project: Learning Joint Audio-Text Embeddings for Effective Multimedia Search
  • Chen Song, SUNY Buffalo Project: Sensor Fusion to Track Controllers and Reconstruct Pose in VR
  • Jun Zhang, NYU (with Amol Ambardekar and Prof. Siddharth Garg). Project: Architecture-level Optimizations in Convolutional Neural Networks

2017

  • Han Zhao, CMU Project: Semi-supervised Methods to Improve Audio Quality
  • Xuesu Xiao, TAMU Project: Articulated Pose Tracking with Inertial Sensors
  • Rasool Fakoor, UTArlington (with Xiaodong He). Project: Reinforcement Learning for Adaptive Speech Enhancement
  • Jong Hwan Ko, GaTech (with Matthai Philipose). Project: Binarized Deep Neural Networks for Efficient Signal Processing
  • Yanhui Tu, USTC (with Ivan Tashev and Prof. Chin-Hui Lee). Project: Simultaneous Speech Enhancement and Recognition

2016

  • Amit Das, UIUC (with Ivan Tashev). Project: Accurate Recognition of Hand Gestures with Ultrasound Signals
  • Samantak Gangopadhyay, GaTech (with Krishna Chintalapudi). Project: Characterizing Performance of Long-range Radios in LPWANs
  • Josh Fromm, UW (with Matthai Philipose). Project: Efficient Implementation of DNNs on Embedded Devices for Object Recognition
  • Soham Desai, GaTech (remotely with Prof. Arijit Raychowdhury). Project: Low-energy RBM Architectures for Pose Detection

2015

  • Mark Gottscho, UCLA (with Sriram Govindan). Project: Modeling Performance Impact of DRAM Error Correction in Intel MCA
  • Barry Tassell, Princeton (with Prof. Daniel Steingart). Project: Self-sustaining Bluetooth Tags with RF-energy Harvesting and Flexible Batteries
  • Xin Zhang, Brown. (remotely with Prof. Sherief Reda). Project: Achieving Soft-heterogeneity in Data Centers with Firmware Reconfiguration
  • Swagath Venkataramani, Purdue (remotely with Prof. Anand Raghunathan). Project: Energy-scalable Classifiers for Efficient Machine Learning
  • Vahid Behravan, OSU. (remotely with Prof. Patrick Chiang). Project: Adaptive Compressed Sensing in Resource-constrained Networks

2014

  • Scott Wisdom, UW (with Jie Liu). Project: Indoor Localization with Frequency-modulated Continuous Wave Radar
  • Ashish Patro, UWisconsin (with Srikanth Kandula). Project: Inference Remapping for Vehicular Analytics on the Edge
  • Swagath Venkataramani, Purdue. Project: Efficient Implementation of Feature Extraction for Video Composition
  • Mark Gottscho, UCLA (with Sriram Govindan and Prof. Puneet Gupta). Project: DRAM Performance Study in Warehouse-scale Computers


Work before Microsoft

In my Ph.D., I have proposed to transform linear signal-processing computations (filtering, estimation etc.) so that they can be applied directly to signals that are compressed. This approach has improved performance in sensor networks by retaining just enough information for machine learning. I have also developed statistical signal-processing techniques to remove artifacts in spherically-coded lenses and reduce channel losses in UWB Radios.

Before graduate school, I developed Genetic Algorithms to learn structure in FIR filters with arbitrary frequency responses and to automatically determine digital-circuit topologies. I have also formulated performance models for multicast network switches and custom instructions in reconfigurable processors.

Background

Education

  • Ph.D. Electrical Engineering, Princeton University 2013
  • B.Tech. Electrical Engineering, IIT Madras, India 2008

Communities

I am a member of the IEEE and ACM. My interests are particularly aligned with the following groups:

  • IEEE Signal Processing, IoT, CAS, RAS, CEDA and Computer Societies
  • ACM SIGMOBILE, SIGBED

Service

I have been involved in the following professional activities:

  • Associate Editor, IEEE J. Design and Test (D&T) ’19 (along with Profs. Subhanshu Gupta, Christoph Studer and Roummel Marcia)
  • Associate Editor, IEEE J. Biomedical Health and Informatics (JBHI) ’18 (along with Dr. Georgia Tourassi and Prof. Chris Nugent)
  • Industry Chair, IEEE Int. Conf. Biomedical and Health Informatics and Body Sensor Networks (BHI-BSN) ’18
  • External Technical Committee Member, IEEE Design Automation Conference (DAC) ’15
  • Demo/Poster Chair, IEEE Cyber-physical Systems (CPS) Week ’15
  • TPC Member, IEEE Embedded Systems (ES) Week ’15: IoT Systems Track
  • Web Chair, IEEE/ACM Int. Conf. Information Processing in Sensor Networks (IPSN) ’14
  • External reviewer for Signal Proc. Letters (SPL), Trans. Circuits and Systems (TCAS), Trans. Biomed. Engg. (TBME), and Trans. Biomed. Circuits and Systems (TBCAS)

Biosketch

Shuayb Zarar (formerly Shoaib Mohammed, IEEE S’08,M’13) received the B.Tech. and M.Tech. dual degree from the Indian Institute of Technology Madras, India in 2008, and the MA and PhD degrees from Princeton University in 2010 and 2013, respectively, all in Electrical Engineering. Since 2013, he is a Researcher at Microsoft Redmond Labs, where he has contributed to product features in Azure, Windows, HoloLens and Windows Mobile. During his time at Microsoft, he has mentored/managed 20 Ph.D. students and led 4 team projects. His work focuses on building efficient computing systems, targeting optimizations in signal processing, machine learning and low-power sensing. He has filed/issued over 30 patents and published over 50 papers at venues like ICRA, NIPS, ICASSP, BSN and DAC. He has been a part of the organizing and technical committees of IPSN 2014, CPS Week 2015, ES Week (IoT track) 2015, DAC 2015, BSN and BHI 2018. He has served as an Associate Editor for the IEEE Journals of Design and Test of Computers in 2019 and Biomedical Health and Informatics in 2018, and as a fellow of the McGraw Center for Teaching and Learning NJ in 2013. He has received the Harold W. Dodds honorific fellowship and Gordon Wu Prize for Engineering Excellence from Princeton University in 2012. He has also received a Ph.D. graduate fellowship and the Roberto Padovani Scholarship from Qualcomm Inc. He is a member of the IEEE and ACM.