Towards Bounding the Behavior of Neural Networks
Disciplines
Other Computer Engineering
Abstract (300 words maximum)
Recent advances in Artificial Intelligence (AI) have unlocked many new possibilities but have also brought with it many new challenges. While modern AI systems have been continuously exceeding expectations, our ability to interpret and understand their behavior lags behind. For example, an AI model trained to detect pneumonia from X-rays may fail in new hospitals because it learned to recognize hospital logos instead of medical patterns. Why do some succeed while others fail? Do they truly understand their tasks, or are they relying on patterns that may not always hold?
To enumerate the most informative explanations of a neuron’s behavior, we developed an improved approach to bounding the behavior of individual neurons within artificial neural networks. In this paper we demonstrate, both theoretically and empirically, the utility of our approach.
.
Academic department under which the project should be listed
CCSE - Computer Science
Primary Investigator (PI) Name
Arthur Choi
Towards Bounding the Behavior of Neural Networks
Recent advances in Artificial Intelligence (AI) have unlocked many new possibilities but have also brought with it many new challenges. While modern AI systems have been continuously exceeding expectations, our ability to interpret and understand their behavior lags behind. For example, an AI model trained to detect pneumonia from X-rays may fail in new hospitals because it learned to recognize hospital logos instead of medical patterns. Why do some succeed while others fail? Do they truly understand their tasks, or are they relying on patterns that may not always hold?
To enumerate the most informative explanations of a neuron’s behavior, we developed an improved approach to bounding the behavior of individual neurons within artificial neural networks. In this paper we demonstrate, both theoretically and empirically, the utility of our approach.
.