Integrated Static and Dynamic Approaches to High-Assurance for Learning-Enabled Cyber-Physical Systems (DARPA ASSURED AUTONOMY) - Project page

The project develops approaches to reason about safety of autonomous vehicles that rely on machine learning for perception and control. Our approach combines design-time techniques with run-time monitoring techniques. One design-time technique is closed-loop verification of deep neural network (DNN) controllers, implemented in a tool VeriSig. Another run-time technique is calculation of robustness measures for classifiers and generation of run-time confidence monitors. Run-time monitoring techniques include detection of anomalous behaviors, evaluation of confidence in the outputs of learning-enabled components, and validation of operational assumptions made at design time. Finally, we develop new argument patterns for the construction of assurance cases for autnomous vehicles, which make use of design-time evidence of safety as well as dynamic evidence produced by monitors discussed above.

Robust Concept Learning and Lifelong Adaptation Against Adversarial Attacks (ARO MURI) - Project page

This proposal addresses the design and analysis of robust and adaptive machine learning based on how children learn. Learning systems are vulnerable to adversarial inputs that are malicious (e.g., image tampering) and non-malicious (e.g., dynamic environments). Moreover, these adversarial inputs can affect the learning systems at design-time (i.e., in the training data) and runtime (i.e., in the test data). Providing predictive analytics that deliver intelligent services in an adversarial setting requires cross-thrust research addressing fundamental adversarial learning challenges. First, modern machine learning generally lack adversarial robustness, both at design-time and run-time, where state-of-the-art approaches tend to perform substantially worse in the presence of adversarial examples such as sensor noise (e.g., dust on the camera lens), image transformations (e.g., translations, rotations, and scaling), and distribution shifts (e.g., daytime images to nighttime images). Second, concepts learned by both children and learning systems must be robust to adversarial examples in real, large-scale dynamic environments, requiring new concepts built upon previous knowledge. However, achieving this improved robustness requires increased reliance on prior concepts (e.g., relational models, causal models, and shapes) that may also be susceptible to changes in the environment. Third, safety critical autonomous systems require verifying and monitoring the robustness of concept-based learning, which is challenging in dynamically evolving systems based on the sensed environment.
In this project, we aim to develop the foundations for robust and adaptive learning based on childhood development consisting of three research thrusts: (Thrust I) Concept-based Learning Robust to Adversarial Examples; (Thrust II) Adaptive Learning in Dynamic Environments; and (Thrust III) Verification and Monitoring of Learning.

AutoWean: Foundations of Autonomous Medical CPS for Mechanical Ventilation Weaning (NIH R01) - Project page

While mechanical ventilators provide life-saving respiratory support, prolonged acute mechanical ventilation (PAMV) can lead to severe complications (e.g. , pneumonia) and increased health-care costs - predicted to be over $32 billion in 2020 and accounting for over 10% of all hospital costs. Thus, mechanical ventilation needs to be discontinued as early as possible, often by using a process known as "weaning". However, the best approach to weaning remains an open question and is subject to controversy, where estimated 170,000 preventable deaths per year in US intensive care units are a result of inappropriate ventilator weaning. Consequently, the Emergency Care Research Institute (ECRI) lists improper ventilator settings as a Top 10 Health Technology Hazard in 2019. Autonomous mechanical ventilation weaning exemplifies a medical cyber-physical systems (MCPS) that requires collections of interconnected medical devices (e.g., ventilators and patient monitors) that are coordinated for treating a patient (i.e. , performing safe autonomous weaning). This project aims to develop fundamental advances in safe and effective data-driven context-aware human-in-the-loop control that will enable autonomous mechanical ventilation weaning. While closed-loop control and data-driven techniques (e.g., system identification and machine learning) have been applied to MCPS, assuring the safety and reliability of using data-driven components that adapt in-the-loop with a human operator remains a challenge. We will demonstrate the impact of our closed-loop design and analysis techniques in autonomous MCPS for mechanical ventilation weaning.

Monitoring Safety and Adherence of Pain Management Through Remote Opioid Sensing and Analysis (NIH R01) - Project page

While mechanical ventilators provide life-saving respiratory support, prolonged acute mechanical ventilation (PAMV) can lead to severe complications (e.g. , pneumonia) and increased health-care costs - predicted to be over $32 billion in 2020 and accounting for over 10% of all hospital costs. Thus, mechanical ventilation needs to be discontinued as early as possible, often by using a process known as "weaning". However, the best approach to weaning remains an open question and is subject to controversy, where estimated 170,000 preventable deaths per year in US intensive care units are a result of inappropriate ventilator weaning. Consequently, the Emergency Care Research Institute (ECRI) lists improper ventilator settings as a Top 10 Health Technology Hazard in 2019. Autonomous mechanical ventilation weaning exemplifies a medical cyber-physical systems (MCPS) that requires collections of interconnected medical devices (e.g., ventilators and patient monitors) that are coordinated for treating a patient (i.e. , performing safe autonomous weaning). This project aims to develop fundamental advances in safe and effective data-driven context-aware human-in-the-loop control that will enable autonomous mechanical ventilation weaning. While closed-loop control and data-driven techniques (e.g., system identification and machine learning) have been applied to MCPS, assuring the safety and reliability of using data-driven components that adapt in-the-loop with a human operator remains a challenge. We will demonstrate the impact of our closed-loop design and analysis techniques in autonomous MCPS for mechanical ventilation weaning.

Smart Alarms 2.0: Foundations for Caregiver-in-the-loop Suppression of Non-Informative Alarms (NSF SCH) - Project page

Systems used to continuously monitor vital signs in hospitals and home settings generate alarms intended to warn caregivers, clinicians in the hospital, or parents at home of conditions that warrant their immediate attention. However, less than 1% of these alarms are considered actionable or informative, which has led to clinical alarm fatigue being ranged as one of the Top 10 Health Technology Hazards. While there have been advances in human factors, operational factors, and technical factors independently, their interdependencies have yet to be collectively considered. Consequently, current alarm suppression systems aimed at addressing clinical alarm fatigue are generally not extensible, suffer from low adoption rates, and/or have poor performance in practice. This project aims to develop foundations for the design of next generation smart alarms. Smart Alarms 2.0 will utilize caregiver-in-the-loop alarm suppression systems to reduce the frequency of non-informative alarms in-clinic and at home. Realizing an effective caregiver-in-the-loop alarm suppression system requires fundamental advances in robust alarm suppression design methodologies and caregiver-in-the-loop modeling and analysis. The potential impact of the proposed design and analysis techniques is evaluated in medical scenarios for alarm suppression applications, spanning pediatric wards and at-home monitoring of Bronchopulmonary dysplasia (BPD) patients. This project will lay the foundations for designing caregiver-in-the-loop alarm systems in which physicians, nurses, and parents monitor the right children, at the right times, using optimal alarm settings to maximize informativeness.