Towards Bounding the Behavior of Neural Networks
Primary Investigator (PI) Name
Arthur Choi
Department
CCSE - Computer Science
Abstract
Recent advances in Artificial Intelligence (AI) have unlocked many new possibilities but have also brought with it many new challenges. While modern AI systems have been continuously exceeding expectations, our ability to interpret and understand their behavior lags behind. For example, an AI model trained to detect pneumonia from X-rays may fail in new hospitals because it learned to recognize hospital logos instead of medical patterns. Why do some succeed while others fail? Do they truly understand their tasks, or are they relying on patterns that may not always hold?
To enumerate the most informative explanations of a neuron’s behavior, we developed an improved approach to bounding the behavior of individual neurons within artificial neural networks. In this paper we demonstrate, both theoretically and empirically, the utility of our approach.
.
Disciplines
Other Computer Engineering
Towards Bounding the Behavior of Neural Networks
Recent advances in Artificial Intelligence (AI) have unlocked many new possibilities but have also brought with it many new challenges. While modern AI systems have been continuously exceeding expectations, our ability to interpret and understand their behavior lags behind. For example, an AI model trained to detect pneumonia from X-rays may fail in new hospitals because it learned to recognize hospital logos instead of medical patterns. Why do some succeed while others fail? Do they truly understand their tasks, or are they relying on patterns that may not always hold?
To enumerate the most informative explanations of a neuron’s behavior, we developed an improved approach to bounding the behavior of individual neurons within artificial neural networks. In this paper we demonstrate, both theoretically and empirically, the utility of our approach.
.