Over fourteen million people in the United States of America have some form of vision disability in which vision cannot be corrected with glasses alone. Many people within this population are unable to use extended reality (XR) head-mounted displays due to their inaccessible designs. Worse yet, best practices for accessibility in immersive XR are still a nascent area of research. In my dissertation, I develop an immersive visual impairment simulation that is both eye-tracked and data-driven. Furthermore, this testbed is validated by analyzing behavioral responses across both visually impaired and normally sighted users. My testbed will allow normally sighted individuals to better understand and design for people with visual impairments by allowing them to experience visual impairments firsthand.
The pace of global aging speaks to the urgency with which frailty must be addressed. Frailty is not an inevitable consequence of aging, but a preventable and even reversible vulnerable state. Multimodal approaches comprising early identification, risk prediction of pertinent outcomes, and targeted intervention protocols are needed. By bridging clinical expertise and AI, my research harnesses big data grounded in integrated information system to develop premorbid predictive risk algorithms that translate into meaningful standard practice. The algorithm predicts negative outcomes including delays in discharge or alternate level of care (ALC). Using Chi-squared automatic interaction detection (CHAID) and classification and regression tree (CART), results will be validated within home and community care and acute-care hospital cohorts. This lays the foundation for a collaborative framework involving real-time clinical exchange where decision support tools are seamlessly incorporated into routine workflows. This optimizes transitional care challenges like discharge planning and proactively tackles ALC and the woes of “hallway medicine”.
Sound provides rich information about our world, ranging from actionable cues (e.g., a microwave beep) to safety related cues (e.g., a fire alarm) to cues that make one feel present (e.g., a bird chirp). However, sounds are not always accessible to people who are d/Deaf and hard of hearing (DHH). In my work, I am leveraging advances in Human Computer Interaction, signal processing, and machine learning to sense and provide sound feedback to DHH people. Specifically, I am designing and evaluating three related systems: (1) a smarthome system, called HomeSound, to sense and visualize sounds in the home, (2) a smartwatch app, called SoundWatch, to notify about sounds in diverse contexts, and (3) an augmented reality head-mounted display system, called HoloSound, to display speech transcription and other conversation cues (e.g., speaker identification, location). All three systems are being deployed and tested with DHH users. My overarching vision is to transform how DHH people think about, experience, and engage with the sound.
Spaces and technologies are routinely and strategically used by women micro-business owners in the Global South and Global North to manage how they present themselves to the public, define and manage their family obligations, and turn a profit to make a living. To date, however, little research has focused on exactly how these ever-present yet somehow also invisible, overlooked workers do this. By noting the use, personalization, and integration of technologies, social practices, and spaces and time, this mixed-method dissertation examines how women micro-entrepreneurs in Guatemala and the U.S. manage their economic and home boundaries while shedding light on the general concept of “entrepreneur”, the work it entails, and the tangible consequences of this classification on women’s lives, tech and economic policies, and literature on work and computing. This dissertation will inform the design of digital technologies for women micro-entrepreneurs and the future of work-oriented policy.
Modeling realistic natural language scenarios requires dealing with long, noisy inputs and accounting for complicated structural dependencies. Traditionally, there have been two approaches for doing this. Structured prediction methods combine symbolic representations of knowledge and probabilistic inference, while neural approaches learn distributed representations that capture the underlying dependencies in a latent high-dimensional space. In my dissertation, I motivate the integration of these two modeling paradigms for natural language discourse scenarios, and propose a neural-symbolic framework to model both the language input, and the context surrounding the language event. The content consists of the different chunks of language and the way they flow together to convey meaning. The context is the frame that surrounds the communicative event and provides resources for its interpretation. I identify four key challenges and opportunities in the space of neural-symbolic methods for discourse: declarative modeling, computational considerations, deriving explanations and learning with humans in the loop.
My research addresses questions related to strategic behavior in Machine Learning (ML). These questions are of utmost importance nowadays, since ML algorithms are increasingly used in real-world, highly consequential decision-making that affects our everyday lives, ranging from online ad auctions guiding our purchasing behavior to complex algorithms deciding whether we should be approved for a loan or not. My dissertation serves as a building block towards establishing a theory of incentives for ML algorithms and studying their societal implications through the two paradigms of incentive-compatible and incentive-aware ML – two terms that are borrowed from the Game Theory literature while providing novel learning algorithms tailored to these strategic settings.
As robots are progressively used in the workplace, at home, and in public settings, design tools for creating social human-robot interactions become increasingly critical. Interaction designers and developers must coordinate individual behavioral modalities, such as speech and locomotion, to produce appropriate social behaviors, all while ensuring that the interaction logic, or how the robot responds to sensory input, produces a natural interaction flow. Furthermore, the success of a social human-robot interaction is defined by variable criteria, depending on the norms, constraints, and user preferences of the interaction context. The focus of my dissertation is on how human-robot interaction “authoring” tools can mitigate these challenges. Specifically, I seek to answer how authoring tools can help designers and developers create robust interactions by (1) filling in gaps in designer knowledge and expertise and (2) eliciting knowledge already possessed by designers and assisting with the integration of this knowledge into interaction designs.
Machine learning (ML) provides efficient solutions for a number of problems that were difficult to solve with traditional computing techniques. Deep neural networks have become a key workload for many computing systems due to their high inference accuracy. This accuracy, however, comes at a cost of long latency, high energy usage, or both. It also introduces many ML-specific bugs into software systems. My thesis aims to help developing accurate, efficient, and robust machine learning software systems. In the past few years, my work has focused on creating robust methods to incorporate neural networks into software systems to satisfy differing requirements and goals across a variety of users and applications. My past research has led to first-author papers published at ATC’20, ICML’20, and ICSE’21.
Cloud computing has emerged as the dominant platform for computing due to its ease of use and scalability advantages. To maintain high-performance services in a cost-effective manner, cloud providers need to efficiently administer a large number of management tasks including resource scheduling, power management, maintenance planning and more. Recent works have shown promising results with machine learning assisted resource management policies. To account for more dynamic changes in the cloud environment, online learning becomes necessary where the model is continuously updated online with live data. My dissertation research focuses on facilitating the integration of online learning into cloud resource managers. I propose to build a general framework that guides developers to reason through agent-specific issues while automatically catching and handling common problems shared across learning-based management agents during deployment. Such a framework will help deploy more safe and robust online learning-based resource managers in the cloud to improve resource efficiency.
Job search today involves digital elements such as applying online or conducting an interview. Yet, inmates have very limited opportunities to learn digital skills – at least in the United States, prisons do not provide internet access, and digital literacy trainings if offered at all are limited. My research focuses on understanding what mobile phone-based interventions could build on existing social support to improve formerly incarcerated individuals’ digital literacy skills? To explore this question, I will run a participatory design workshop where formerly incarcerated individuals and their support systems will design the elements of a mobile phone-based set of tools that can be used to learn digital literacy skills for job search. Next, I will develop a prototype and evaluate it by conducting a usability study. My empirical research will contribute to the much-needed conversation of how to support formerly incarcerated individuals in a world that increasingly relies on digital technology.
Carnegie Mellon University
Dissertation title: Active Vision: Autonomous Aerial Cinematography with Learned Artistic Decision-Making
Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints, such as entertainment, sports, and security. Fundamentally, it is a tool with immense potential to improve human creativity, expressiveness, and sharing of experiences. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple highly trained human operators to safely control a single vehicle. My dissertation focuses on building autonomous systems that can empower any individual with the full artistic capabilities of aerial cameras. I am developing intelligent cameras that actively reason about the cinematographic quality of viewpoints, and safely generate sequences of shots that avoid collisions and occlusions with obstacles. The theory and systems developed in this work can impact video generation for both real-world and simulated environments, such as professional and amateur movie-making, videogames, and virtual reality.
University of Maryland, College Park
Dissertation title: Practical Techniques for Leveraging Experts for Sequential Decisions and Predictions
My research addresses practical and theoretical sample complexity issues which arise in reinforcement learning by leveraging knowledge from experts, specified in the form of “Annotations.” As the impact of machine learning on all aspects of our lives continues to grow, having systems that can learn through interaction with users and the world becomes more pressing. Unfortunately, today’s approaches—typically driven by reinforcement learning algorithms—require a large number of interactions, which is feasible only for systems we can fully simulate. But for many real-world systems, simulators do not already exist. This means we need to develop algorithms that can interact with real systems and have very low sample complexity so that they can learn in a reasonable amount of time. I propose that for many such systems, we can leverage certain forms of expert knowledge, which I call “Annotations”, to reduce sample complexity and lead to more practical algorithms in real-world settings.
Mayara Costa Figueiredo
University of California, Irvine
Dissertation title: Self-Tracking for Fertility Care: A Holistic Approach
With the recent popularity of self-tracking technologies, individuals are increasingly using and interacting with their personal data. Fertility is a health matter that has been progressively assisted by consumer health technologies, which aim to support people in tracking diverse data potentially associated with their fertility cycles. However, fertility is a complex context for self-tracking, presenting many challenges, involving diverse goals and transitions, and entangled with social factors and taboos. My dissertation research combines multiple studies to investigate self-tracking for fertility care, focusing on understanding individuals’ data practices, how technology influences these practices, and the consequences of using such data. Based on these studies, I explore how design and technology can be used to reinforce positive experiences, avoid negative emotional burden, and support holistic tracking for fertility.
University of Washington
Dissertation title: Complex Analysis, Hierarchies, and Matroids—Improving Algorithms via a Mathematical Perspective
As a theoretical computer scientist, my goal is to design algorithms with provable guarantees for practically motivated problems. Some of the main topics in my thesis work are fair allocation, scheduling, and statistical reconstruction. These problems are united by the need to understand some underlying mathematical structure. In fair allocation, we discover a connection to matroids allowing for a simpler, more general, and more efficient approximation algorithm. Additionally, we make major progress on a scheduling problem using linear programming hierarchies and a partitioning algorithm for metric spaces. I also use tools from complex analysis and probability theory to study generating functions arising from various statistical reconstruction problems.
The University of British Columbia
Dissertation title: Placenta: Towards an Objective Pregnancy Screening System
The placenta forms the critical interface between the fetus and the mother. Placental insufficiency has adverse effects on the neonatal outcome and life-long health of the offspring as well as the mother. The current clinical practice of ultrasound-based pregnancy monitoring relies heavily on user interpretation. Additionally, conventional ultrasound suffers from low sensitivity to detect the underlying pathophysiological change resulting from placental insufficiency. There is a need to develop effective parameters that would be the predictor of the underlying pathophysiology, and therefore, would serve as potential biomarkers of placenta-mediated diseases. The objective of my dissertation research is to develop a system and user-independent pregnancy screening system that will provide a quantifiable measure of placental health. The quantitative ultrasound (QUS) based multiparametric screening will allow the identification of the early symptoms of the placental abnormalities with an opportunity of intervention to prevent clinical manifestation and long-term effects.
University of Massachusetts Amherst
Dissertation title: Enhancing Usability and Explainability of Data Systems
The growth of data science and its reach to an ever-growing user base of non-experts require data systems to be democratized and offer more transparency. Democratization entails a system that can be used by people with different skills and backgrounds alike, and transparency requires explanation mechanism for the users to understand the system behavior, especially when unexpected behavior occurs. Unfortunately, most data systems are usable only by the expert users. Furthermore, they lack explainability, making them unreliable for even the most expert users. The first aim of my work is to enhance usability of complex data systems for non-experts where I focus on example-driven techniques that complement traditional task-specification mechanisms. The second aim of my work is to provide causality-guided explanation frameworks to enable understanding of certain outcomes of complex data systems. Finally, my work aims at developing data profiling techniques for achieving trust and fairness in data-driven machine learning.
Dissertation title: Using Online Team Recommender Systems to Form Diverse Teams
Embracing diversity can yield measurable benefits for teams and organizations. Although society extols the benefits of diversity, particularly to teams, assembling diverse teams is not a simple task. Online team recommender systems may facilitate diverse team formation by enabling people to filter, curate, and review recommendations for prospective teammates from a wider pool and based on deep-level information. My dissertation sheds light on how team recommender systems shape users’ motivations and actions to assemble diverse teams. My goals are threefold: (i) characterize the diversity of teams assembled online, (ii) understand the basis of users’ decisions when they are forming teams online, and (iii) evaluate a team recommender system with regard to diversity. By tracking users’ prior actions and interactions, team recommender systems can leverage a combination of digital trace data, network information, and individual characteristics to assemble diverse teams in organizations.
University of Washington
Dissertation title: Low-power Communication for Environmental Sensing Systems
Over the last decade the Internet of Things (IoT) has been changing the world, from enabling connected electronics, smart homes, to smart agriculture. I am deeply passionate about the role that IoT plays when it comes to environmental monitoring, which is now increasingly relevant in the times of the climate change crisis and the need to achieve biodiversity conservation. My thesis focuses on enabling low-power communication for environmental sensing systems. In particular dealing with two major challenges: resources constraints and scale. For example, devices deployed in remote locations often lack power and Internet connectivity. This becomes even more challenging when considering the size of forest, farms, and oceans all requiring large-scale sensing systems. My past and ongoing research focuses on addressing these challenges by developing systems like FarmBeats to enable data-driven agriculture, enabling low-power communication using backscatter techniques, and using machine learning to improve the performance of mainstream IoT solutions.
Dissertation title: Understanding and Exploiting the Use of Linguistic Context by Neural Language Models
The meaning of any piece of text is determined by the words contained in it. Proper use of this linguistic context is an extremely important part of every task in the field of natural language processing (NLP). Yet, the black-box nature of neural networks makes it extremely unclear what contextual information is captured and how it can be used to further improve existing models. In my dissertation, I address this problem for neural language models—the basis of modern language understanding and generation tasks. First, I use analysis techniques to shed light on how these models use linguistic context and what features are learned from it. Then, using the knowledge gained from the analysis, I show how pre-trained neural language models can be improved and adapted to larger datasets and different domains without any additional training, by simply relying on the high-dimensional representations learned from linguistic context.
Dissertation title: Towards a More Inclusive Gig Economy: Examining Privacy, Security, and Safety Risks for Workers with Chronic Illnesses and/or Disabilities
Workers in the gig economy do not receive traditional labor protections, and the precarity of their work can lead them to prioritize financial incentives over protecting themselves from risk. Even more problematically, workers with chronic illnesses and/or disabilities may face disproportionate risks in terms of their privacy, security, and safety during the course of their work, in addition to navigating the economic precarity of gig work. Through a large qualitative study, my dissertation will develop a comprehensive account of the risks and opportunities posed by a range of gig platforms for this group of workers. Ultimately, this work aims to inform how gig platforms can be more inclusive for workers with chronic illnesses and/or disabilities while mitigating the risks they encounter during their work.
Rochester Institute of Technology
Dissertation title: Automatic Speech Recognition as a Captioning Tool: Enabling Greater Accessibility for Users who are Deaf or Hard-of-Hearing
As the accuracy of Automatic Speech Recognition (ASR) technology improves over time, it may become a viable method for transcribing audio input in real-time. Such technology can provide access to spoken language for people who are Deaf or Hard-of-Hearing (DHH). However, ASR is imperfect and will remain so for a while, thus there is a need for users to cope with errors in the output. Our proposed solution is to add markup to the captioning so the DHH viewer is able to discern when the ASR may be erroneous by utilizing the confidence values in the ASR output. My goal is to empower the DHH individual with greater autonomy in scenarios such as one-on-one meetings with hearing people when in-person interpreters are not available.
University of California, San Diego
Dissertation title: Inferring Country-Level Transit Influence of Autonomous Systems
Our work explores the country-level influence exerted by transit providers, a set of networking organizations that often have less direct contact with users, but who are nonetheless responsible for delivering an important fraction of transnational traffic into and out of many countries, and who may have the capability to observe, manipulate, or disrupt some of that traffic. For instance, an accidental misconfiguration or a state-ordered disconnection implemented by one of these operators may render popular services delivered on the Internet (such as email or social media) unreachable in entire regions. These concerns are not abstract, as previous instances of state-ordered disconnections have propagated to other countries and temporarily disabled some of the world’s most popular services there. By studying the ways in which these operators (Autonomous Systems) connect to one another and to the rest of the Internet, we aim to highlight each country’s relative risk exposure.
Dissertation title: Understanding the Limits of Computational Models and Learning Algorithms
The primary aim of my research has been studying the computation limits of and the role of randomness in space-bounded computational models. In particular, my work has focused on proving for a large class of learning problems the following: a low-memory learning algorithm requires an exponential number of samples to learn. These also give cryptographic protocols with unconditional security against low-space adversaries. The second aim of my research has been algorithmic fairness and investigating the sources of unfairness in classification algorithms. Machine learning algorithms are increasingly being used for making decisions about humans, which raises concerns that these might inadvertently discriminate against certain groups of society. In particular, my work has focused on understanding the role of information in fairness and exploring the mechanisms/incentives that lead to fair predictions.
University of Michigan, Ann Arbor
Dissertation title: Was That a Seizure? Understanding Everyday Ambiguity in the Clinical Diagnosis and Lived Experience of Epilepsy
Epilepsy is a chronic illness characterized by recurrent and unpredictable seizures, during which people lose voluntary control over body-mind function. Seizure symptoms often resemble bodily sensations such as muscle spasms and dizziness; but seizures require medical care whereas common muscle spasms require hydration or rest. This ambiguity between seizure-like symptoms and seizures troubles people with epilepsy (PWE), confounding the decision between three courses of action: report to ER, schedule a doctor’s appointment, or rest. This ambiguity also arises in clinical diagnosis, where neurologists must distinguish social and environmental factors from seizures: being patted on the back, for instance, resembles seizures. My dissertation addresses this ambiguity through comparative ethnographic fieldwork and machine learning to design a wearable interface that combines clinical expertise with experiential knowledge to detect seizures. In doing so, I problematize the gap between the subjective experience of seizures and objective representations thereof in clinical diagnosis.
Julian A. Ramos Rojas
Carnegie Mellon University
Dissertation title: Personalized and Context-Aware Behavioral Interventions using Artificial-Intelligence
In traditional behavioral interventions, a clinician personalizes a treatment to foster positive behaviors and replace unhealthy habits (eg., drinking, or smoking). Behavioral interventions are, however, limited to the number of available clinicians and their brief interactions with patients. For my PhD dissertation, I want to leverage the power of computing to scale behavioral interventions by creating a computational approach that fulfills the clinician’s role of personalization: finding a treatment that aligns with the patient’s goal, preferences, and desired outcomes. I will be using Artificial Intelligence (AI) methods and mobile sensors to personalize health interventions. As part of this work, I’m currently piloting an android app that gives advice on how to improve sleep based on Fitbit data and user feedback using AI.
Sonia Jawaid Shaikh
University of Southern California
Dissertation title: Bounded Technological Rationality: How Human-AI Collaboration Impacts Prosocial and Antisocial Behaviors
AI-enabled technologies such as intelligent assistants are increasingly being developed and deployed to make decisions that were previously made by humans alone. The widening use of intelligent assistants across physical and virtual spaces offers a unique opportunity to explore various issues pertaining to individual and group information exchange. This project empirically tests how and when intelligent assistants can serve as an intervention and nudge decision-makers to increase prosocial and decrease antisocial informational behaviors within various kinds of competitive and resource environments. The major goals of this project are: a) to unpack socio-psychological processes and behavioral outcomes in human-AI collaboration, b) to understand human rationality from a technological perspective, and c) to help develop AI-enabled technology for social good.
Dissertation title: Advancing Accessible 3D Design for the Blind and Visually Impaired via 2.5D Tactile Shape Displays
People with blindness and visual impairments (BVI) are experienced makers having to adapt the technology available to solve accessibility problems they face. This spirit of creative problem solving and tinkering has existed in parallel to the mainstream Maker Movement because most maker tools are inaccessible. Accessibility in making can not only provide access points for contextualized learning of STEM concepts but can also give the BVI community the tools to participate in the vibrant maker culture as designers themselves. My dissertation seeks to bridge this gap by increasing access to 3D design. It aims to create an accessible 3D design and printing workflow for BVI people through the use of 2.5D tactile displays, and to understand how BVI people use the workflow in the context of perception, interaction, and learning. At a broader level, it seeks to increase access to STEM concepts and give BVI people, a new medium for creative expression that others across the world already engage in.
University of California, San Diego
Dissertation title: Group Perception Methods to Support Human-Robot Teaming
The field of robotics is growing at a vast pace with robot deployments in everyday environments. In many of these environments, people are often in groups; therefore, robots need a high-level understanding of groups in order to fluently assist and interact with them. Yet much prior work in human-robot interaction (HRI) focuses on dyadic interaction. To address this gap, my research focuses on designing perception methods to enable robots to work seamlessly in a group. My research contributes novel perception methods that enable robots to effectively identify groups, track them over time, infer their future motion trajectories, and navigate and interact among them in real-world settings. Furthermore, it will enable robots to join and participate in group interactions, which will enable the next generation of artificially intelligent systems. My dissertation will enable more robust, realistic HRI, and support the safe operation of mobile robots in human-centered environments.
University of South Florida
Dissertation title: Hunting Mosquito Breeding Habitats using Drones and State-Dependent Measurement Models
To search for mosquito breeding habitats (MBHs) in forest-like areas, an autonomous robot such as a drone relies on sensor measurements to estimate its state and the state of the surrounding world; such states include the locations of the drone, tree trunks, branches, and MBHs. However, since measurements contain error (bias and uncertainty), state estimates are often inaccurate. My dissertation alleviates measurement error by modeling the errors using a novel concept called state-dependent measurement models, which estimate a measurement error probability distribution for each sensor measurement. While navigating in forest-like areas, drones can use such models to determine how confident they are about their state and the state of the world, which can help a drone safely maneuver amongst trees and competently perform tasks. My dissertation validates this concept by using it to build uncertainty-aware maps, autonomously navigate, and detect MBHs in dense forest-like areas with simulated and real drones.
Carnegie Mellon University
Dissertation title: Understanding AI as an HCI Design Material: Providing Methods and Tools for Sketching and Prototyping Human-AI Interaction
My work focuses on understanding and addressing the human-computer interaction (HCI) challenges brought by the advances of AI. I worked to articulate why and where human-AI interaction seems particularly difficult to design, in comparison to many other complex interactive technologies. I identified effective methods for sketching and prototyping human-AI interactions. I distill these methods into a Designing AI toolkit, helping more AI designers and teams to translate the technical advances of AI into human-centered, thoughtful, and creative real world applications.
University of Southern California
Dissertation title: Reducing User-Perceived Latency in Mobile Apps via Prefetching and Caching
Mobile devices have become the dominant computing platform and this trend is reflected in the billions of mobile devices and millions of mobile apps today. At the same time, user-perceived latency caused by network transfers remains a significant problem since mobile apps fetch data from the Internet constantly via unreliable wireless network. To tackle the network latency problem in mobile apps, my research focuses on prefetching and caching techniques as they can bypass the performance bottleneck and enable immediate response from a local store. However, such fundamental techniques are largely overlooked in the emerging mobile-app domain. Thus, my dissertation aims to establish the foundation for prefetching and caching techniques in the mobile-app domain by exploring the prefetching and caching opportunities in practice, and proposing a set of novel techniques that are suitable for mobile apps in order to reduce user-perceived latency.
University of Washington
Dissertation title: Toward Disability-Informed Human-Centered Design
Experience shows us that people with disabilities can positively impact interaction design for everyone. However, publishers of interaction design rubrics–such as Human-Centered Design–have tended to focus on supporting the design process for people with disabilities, rather than by them. My research focuses on developing an inclusive toolkit that augments current Human-Centered Design activities to be accessible to people with disabilities. Drawing from this toolkit, I will offer new ways to connect disability with design, all based on the life experiences of people with disabilities.
Georgia Institute of Technology
Dissertation title: Trust, Technology and Community Engagement
The work of community engagement performed by public officials in local government provides valuable opportunities for city residents to participate in governance. Technology stands to play an increasingly important role in mediating community engagement; however, the practices and relationships that constitute community engagement are currently understudied in human-computer interaction (HCI). Of particular importance is the role that trust plays in the success of community engagements—either establishing trust, or more frequently, overcoming distrust between public officials and city residents. To address this challenge, my research seeks to understand how trust could inform the design of technology to support the work of community engagement performed by public officials in local government. My research will culminate in a design framework that will inform development of technology for trust-based community engagement.
Ryan M. Corey
University of Illinois at Urbana-Champaign
Dissertation title: Array Signal Processing for Augmented Listening
Augmented listening technologies, such as hearing aids, smart headphones, and audio augmented- reality platforms, promise to enhance human hearing by processing the sound we hear to reduce unwanted noise and improve understanding. State-of-the-art listening devices perform poorly, however, in noisy environments that have many competing sound sources. Large microphone arrays with dozens or hundreds of sensors could allow listening devices to separate, process, and enhance multiple sound sources in real time while sounding natural to the user. My dissertation addresses several unique challenges of array processing for real-time listening applications, such as tracking human movement, preserving the user’s spatial awareness, estimating the dynamics of multiple simultaneous sound sources, and maintaining an imperceptible input-to-output delay. I am also developing first-of-their-kind wearable microphone array prototypes and data sets to help other researchers develop ambitious new augmented listening algorithms and applications.
Carnegie Mellon University
Dissertation title: Quantifying and Mitigating Risks of Algorithmic Decision Support
Machine learning is increasingly being used for decision support in critical settings, where predictions have potentially grave implications over human lives. Examples of such applications include child welfare, criminal justice, and healthcare. In these settings, the characteristics of available data and of deployment contexts give rise to challenges that have not been sufficiently addressed in the machine learning literature, including the presence of selective labels, unobservables, and the effects of omitted payoff bias. When left unaddressed, these challenges may lead to systemic biases, self-fulfilling prophecies, and loss of human trust in the systems. My research is focused on quantifying the performance and fairness risks of algorithmic learning in these settings, and on reducing these risks by developing novel algorithms.
Dissertation title: Artistic Vision: Providing Context for Capture-Time Decisions
As cameras become smarter and more pervasive, more people want to learn to be better content creators. People are willing to invest in expensive cameras as a medium for their artistic expression, but few have easy ways to improve their skills. Inspired by critique sessions common in in-person art practice classes, my dissertation research focuses on designing new interfaces and interactions that help people become better photo takers. Using contextual in-camera feedback, users can capture photos and videos in a way that is more informed and intentional, while still allowing for their aesthetic and creative decisions.
University of Rochester
Dissertation title: Computationally Efficient Modeling and Audio Enhancement Algorithms for Reverberant Acoustic Systems using Orthonormal Basis Functions
Highly interactive modeling methods and audio enhancement algorithms underlie the operation of modern acoustic systems. The capability of a system to produce lifelike acoustic experiences significantly depends on the accuracy and computational efficiency of the modeling and audio processing algorithms employed. Accordingly, my research has focused on the development of methods and algorithms that accurately model highly reverberant acoustic systems and process acoustic signals using as few parameters as possible. Such accurate yet computationally efficient modeling and processing algorithms are of essential interest in a wide variety of applications ranging from virtual acoustics to healthcare. My main contribution is the development of algorithms, which rely on orthonormal basis functions and time-frequency representation of an acoustic system, that provide high accuracy over a wide range of frequencies in real-time. As an early demonstration, I propose an efficient solution to adaptive feedback cancellation problems.
University of Florida
Dissertation title: Privacy Preserving Computational Cameras
Major advances in computer vision and mobile technologies have set the stage for widespread deployment of connected cameras, spurring increased concerns about privacy and security. To address these concerns, I’m building novel computational cameras that perform privacy processing, at the camera level, via optical filtering of the incident light-field and/or sensor-level electronics, and developing a data-driven framework to learn privacy-preserving encoding functions through adversarial optimization. Moving forward, I aim to leverage this framework to build low-power privacy-preserving computational cameras with camera-level implementations of learned encoding functions.
Massachusetts Institute of Technology
Dissertation title: Human-Guided Reinforcement Learning in Real-World Environments
Deploying AI systems safely in the real world is challenging. The rich and complex nature of the open world makes it difficult for machines trained on limited data to adapt and generalize well. The errors that can result from an imperfect model can be extremely costly (e.g., car accidents, incorrect diagnoses). My research focuses on using human feedback to help reinforcement learning agents better adapt to the real world, leading to safer deployment of these systems. This involves developing robust models that can accurately predict uncertainty in the world, use different forms of human input to learn, and adapt quickly in real-time to new changes in the environment. Developing such systems that learn from humans intelligently will move us closer towards more generalizable robots that perform a variety of tasks in such applications as assistive robotics, healthcare, and disaster response.
University of Pennsylvania
Dissertation title: Hierarchical Approaches to Improve the Flow, Style, and Coherence of Conversational Agents
There has been a renewed focus on dialog systems, including non-task driven conversational agents (i.e. “chit-chat bots”). Dialog is a challenging problem since it spans multiple conversational turns. To further complicate the problem, there are many contextual cues and valid possible utterances. We propose that dialog is fundamentally a multiscale process, given that context is carried from previous utterances in the conversation. Neural dialog models, which are based on recurrent neural network (RNN) encoder-decoder sequence-to-sequence models, lack the ability to create temporal and stylistic coherence in conversations. My thesis focuses on novel hierarchical approaches to improve the responses of neural chatbots.
Mina Tahmasbi Arashloo
Dissertation title: Programmable Network Monitoring and Control
Real-time and fine-grained network monitoring and control is crucial for operating networks that match the security and performance requirements of today’s online services. To that end, modern network devices offer programming interfaces for fine-grained specification of what information to maintain across packets, and how to process packets based on it. These interfaces, however, are quite low-level and suitable only for programming a single device, making them cumbersome to use in today’s large-scale networks. My thesis focuses on designing programming platforms that facilitate the use of programmable network devices for large-scale and real-time network monitoring and control. More specifically, these platforms consist of (i) domain-specific languages that are expressive enough for high-level specification of policies for end-to-end network transport, network-wide state-aware monitoring and control, and path-based network monitoring, and (ii) compilers that use efficient intermediate data structures to automatically distribute and implement these specifications on programmable network devices.
Dissertation title: Methods in Interpretability and Causal Inference for Better Understanding of Machine Learning Models
I aim to develop methods to help users of machine learning models increase both the trust in and understanding of their models. My dissertation is in the two fields of interpretability and causal inference. The two fields, seemingly disparate, actually share the common goals of revealing and adjusting for biases that can arise when building machine learning models. In interpretability, I am developing methods to probe tree ensembles and audit black-box risk scoring models such as COMPAS. In causal inference, I have worked on methods that use machine learning to more flexibly estimate treatment effects from observational data. To complete my dissertation, I plan to probe the definition of interpretability — still a subject of debate in machine learning — by conducting a large-scale comparison of different models claimed to be interpretable and augment this quantitative evaluation with human subject experiments using domain experts.
Johns Hopkins University
Dissertation title: Nanoengineering for Tunable Energy-Efficient Optoelectronics
Colloidal nanomaterials, such as semiconductor quantum dots, are of interest for various optoelectronic applications due to their size-tunable optical properties, distinctive electronic structure, and low-cost fabrication. Color-tuned and semi-transparent photovoltaics, devices with controlled and tunable reflection and transmission spectra, are of significant interest due to their potential applications in building-integrated photovoltaics, vehicular heat and power management, and multijunction photovoltaics. High-performance computing technologies coupled with advanced optimization methods have made it possible to rapidly and efficiently design and predict new device structures without having to rely on costly, time- and resource-intensive “trial-and-error” lab-based experiments in the field of optoelectronics. My project focuses on using nanoengineering techniques, including multi-objective optimization algorithms, plasmonic nanoparticle enhancements, and hybrid-materials-based surface modifications, to design and build colloidal quantum dot-based devices with controlled optical and electrical properties for the next generation of inexpensive and ubiquitous light harvesting, detection, and emission technologies.
Juan Camilo Gamboa Higuera
Dissertation title: Transfer of Robot Motor Behaviors from Low-Fidelity Domains
I’ve been working on algorithms for synthesizing controllers for a six-legged underwater autonomous vehicle, to perform a variety of navigation and pose control tasks. These algorithms allow us to specify data collection tasks, e.g. coral reef monitoring, from high level objectives encoded as numerical cost functions. To reduce the amount of data needed for each task, and since models of underwater dynamics are computationally expensive, we use model-based reinforcement learning techniques where the models are data-driven. A problem with these approaches is that, even if they are data efficient, collecting new data is expensive. I’m investigating techniques that mitigate this cost by re-using prior knowledge, from simulation or similar environments. Our current approach, which we call policy adjustments, allows us to transfer previously learned controllers by reasoning about the discrepancies between the source of the knowledge (a simulator) and the deployment environment (a physical robot in the ocean).
Dissertation title: Efficient, Privacy-Preserving, Secure Cloud Computation and Storage
Adopting cloud services to reduce operational, maintenance and storage costs, is becoming increasingly common. However, outsourcing data and computations, is opening up new challenges in terms of integrity and privacy of the data and the computations on them. Along with such important security and privacy concerns, availability, and scalability are major factors in such settings. My thesis addresses various problems in this space of secure storage and computation outsourcing. In summary, the main contributions of my thesis are the following.
- Designing models and protocols for outsourced queries on structured dynamic data with efficiency, integrity and privacy guarantees along with prototype implementations.
- Designing efficient (general) verifiable computation primitives for data-intense applications along with prototype implementations.
- Developing an expressive framework for efficient graph queries on encrypted networks along with prototype implementations.
- Designing efficient protocols to facilitate secure storage of encrypted data in the cloud while enabling deduplication.
University of Maryland, Baltimore County
Dissertation title: Smart Algorithms via Knowledge Management of Safe Physical Human-Robotic Care
The beginning of a new era in safe assistive robotics will occur when people with disabilities and seniors let intelligent software control a mobile robotic manipulator to safely reposition their body and limbs. Our goal is to explore the intersection between providing physical care and robotics, and how it is possible to translate safe patient handling and mobility guidelines into smart human-robotic interaction (HRI) algorithms. For a mobile manipulator with knowledge-managed algorithms. we propose to create an accessible low fidelity 3D Web interface for manipulating a high degree-of-freedom robot to safely reposition the human body and limbs. Our efforts seek to standardize protocols and regulations for how artificial intelligence agents related to physical HRI can achieve body and limb repositioning tasks. As assistive robotics become more mainstream, these best practices can improve safety in direct physical care in the process of repositioning the human body with a mobile robotic arm.
Dissertation title: Interpretable Machine Learning for Human Decision Making
My research primarily focuses on exploring how machine learning can help improve real world decision making in domains such as health care and criminal justice. To this end, my thesis addresses various challenges involved in developing and evaluating interpretable machine learning frameworks which can complement and provide insights into human decision making. More specifically, my thesis focuses on the following diverse yet related research directions: developing frameworks which can be used to compare the effectiveness of algorithmic and human decision making, building models for obtaining interpretable and diagnostic insights into the patterns of mistakes made by human decision makers, learning accurate and interpretable models (or approximations to existing machine learning models) which can complement human decision making. The main contribution of my thesis is to address these problems under realistic assumptions which hold in real world decision making such as presence of unmeasured confounders and limited availability of labeled data.
Indiana University, Bloomington
Dissertation title: Examining the Implementation of the Health Information System in Mozambique: Understanding the Experiences of Health Care Workers with ICTs
My study examines the implementation of the health information system (HIS) in Mozambique and the roletechnologies play in educating health professionals for better delivery of care. Through a comprehensive examination of the HIS, from development to roll-out, I analyze the relationship between colonial and (post)colonial governmental top-down policies and compare them to the on-the-ground reality of using information and communications technology (ICTs) to provide health education given social, economic, and political realities in Mozambique. Part of the problem with studies of technologies in poor parts of the world is that they are often conducted by highly educated researchers and are conducted in English. However, majority of the population in poor nations does not speak English. Such studies become irrelevant to the life experiences of those being studied. I will disseminate findings from this study in Portuguese and English through talks and publications in U.S., Mozambique, and other international venues.
Martez Edward Mott
University of Washington
Dissertation title: Accessible Touch Input for People with Motor Impairments
Touch-enabled devices such as smartphones, tablets, and interactive kiosks are some of the most pervasive technologies in the world today. As a result, touch has emerged as one of the most dominant forms of input for computing devices. Despite the overwhelming popularity of touch input, it presents significant accessibility challenges for millions of people with motor impairing conditions such as cerebral palsy, muscular dystrophy, and Parkinson’s disease. My dissertation research takes an ability-based design approach toward improving the accessibility of touch-enabled devices for people with motor impairments. I intend to create intelligent interaction techniques that allow people with motor impairments to touch in whichever ways are most comfortable and natural for them, and for the system to react as if it was touched precisely.
Shadi A. Noghabi
University of Illinois at Urbana-Champaign
Dissertation title: Building Large-scale Production Systems for Latency-sensitive Applications
In this era of increased engagement with technology, many latency-sensitive applications processing large amounts of data have emerged. For example, we expect social networks to show hashtag trends within minutes, data from IoT to be processed within seconds, and online gaming to react within milliseconds. In all these diverse areas, handling large scale data in a real-time fashion is crucial. At scale, providing low latency becomes increasingly challenging with many complexities in distribution, scaling, fault-tolerance, and load-balancing. My research has focused on developing techniques that broadly explore these issues with particular attention to end-to-end latency and building massive-scale solutions. Most of my work is deployed in large-scale production systems with hundreds of millions of users. My research contributions span a wide range of frameworks including: Ambry (LinkedIn’s mainstream geo-distributed media store), Apache Samza (a stream processing engine used by LinkedIn, Uber, TripAdvisor, etc.), and Freeflow (a high-performance container networking solution).
John R. Porter
University of Washington
Dissertation title: Understanding and Improving Real-World Video Game Accessibility
My dissertation work attends to the intersection of accessible human-computer interaction and video game design. Games continually grow more complex, pervasive, and significant in 21st century life. However, due to inaccessibility, games are often actively disabling experiences for many gamers with impairments, systematically excluding them from full participation in an increasingly important activity. Therefore, my work proposes to understand the play experiences of gamers with impairments and offer novel design solutions for mitigating the accessibility barriers they face. My proposed investigations seek to understand how accessibility barriers manifest in mainstream games, to empower gamers with impairments to better navigate the landscape of game accessibility through novel information design, and to address underlying institutional concerns that perpetuate systemic accessibility issues in the game development industry through education interventions.
Andrew S. Stamps
Mississippi State University
Dissertation title: Applications of Heterodox Rendering Methods to Visualization
Information visualization is an illustrative method to depict data, and the structure of this data is not necessarily known beforehand. The classic rendering via rasterization of visualization primitives tends to minimize extraneous details; every drawn pixel or glyph has a tight correspondence to the data on which it is based. A simple line chart for example. It is thought that a more expressive or artistic rendering of data might harness additional insight through abstraction, or even an emotional connection. These expressive methods which I have classified as Heterodox Visualization (HV) methods, include non-photorealistic rendering (NPR), stylized rendering processes like pixelization, and other rendering approaches, like those that mimic natural media e.g. painting or sketching. To date there has been little systematic guidance covering how these HV methods could be applied to information visualization. My research will help determine, through experiment, which techniques pose a benefit to different types of visualizations.
Vasuki Narasimha Swamy
University of California, Berkeley
Dissertation title: Real-time Ultra-reliable Wireless Communication
My research focuses on designing wireless communication protocols for Internet-of-Things (IoT) applications that require low-latency and high-reliability. This can enable exciting new interactive and immersive applications such as exoskeletons, inter-vehicle communication for self-driving cars, robotics & factory automation, virtual & augmented reality, high-performance gaming, and the smart grid. I am developing wireless communication protocols that employ simultaneous relaying by all radios in the network. This allows us to overcome bad channels and guarantee the latency requirements. My early work dealt with understanding the fundamental limits of using cooperative communication for high-performance applications. Currently, I am exploring the key physical layer requirements that are needed to implement these protocols. I am modeling how synchronization and channel estimation impacts the performance of these protocols. Ultimately, understanding the fundamental limits of high-reliability and low-latency wireless will enable us to engineer exciting applications.
University of California, Berkeley
Dissertation title: Hybrid Aesthetics – A New Media Framework for the Computational Design of Creative Materials, Tools, and Practices within Digital Fabrication
Technology plays an important role in both constraining and guiding how users explore, express, and innovate in a variety of creative tasks. Practices are emerging which blend both physical and computational techniques and materials providing new opportunities to expand the aesthetic repertoire available to creative practitioners. This thesis contributes a framework for understanding how to create these hybrid elements and develop materials, tools, and practices that stimulate the imagination to explore a wider gamut of creative expressions. Through a series of design tools, the thesis introduces data structures that break constrictive digital modes of practice, conceptual framings for guiding aesthetic exploration, and design principles for the adoption, sharing, and teaching of hybrid techniques. This work serves as a bridge between art and technology and challenges the narrative of who can participate and use digital fabrication technologies to include traditional artists, designers, and the broader community of creative practitioners.