Document Type
Event
Start Date
1-12-2022 5:00 PM
Description
Neural networks have become increasingly powerful and commonplace tools for guiding decision-making. However, due to the black-box nature of many of these networks, it is often difficult to understand exactly what guides them to a certain prediction, making them dangerous to use for sensitive decision making, and making it difficult to ensure confidence in their output. For instance, a network which classifies images of dogs and cats may turn out to be flawed with little consequence, but a neural network that diagnoses the presence of diseases should be assured to make sound predictions. By understanding why a network makes the decisions it does, we can help to guarantee that the choices were made in a sensible way. However, part of the reason neural networks are considered a black-box is because it is very difficult computationally to explain how they work. In fact, individual neurons are known to be hard to explain already. In our research, we consider whether it is possible to learn an individual neuron that is explainable from the start. Unfortunately, our first result tells us that it is NP-hard to learn such a neuron. Fortunately, we have found new conditions under which we can learn an explainable neuron in pseudo-polynomial time.
Included in
GR-241 On Training Explainable Neurons
Neural networks have become increasingly powerful and commonplace tools for guiding decision-making. However, due to the black-box nature of many of these networks, it is often difficult to understand exactly what guides them to a certain prediction, making them dangerous to use for sensitive decision making, and making it difficult to ensure confidence in their output. For instance, a network which classifies images of dogs and cats may turn out to be flawed with little consequence, but a neural network that diagnoses the presence of diseases should be assured to make sound predictions. By understanding why a network makes the decisions it does, we can help to guarantee that the choices were made in a sensible way. However, part of the reason neural networks are considered a black-box is because it is very difficult computationally to explain how they work. In fact, individual neurons are known to be hard to explain already. In our research, we consider whether it is possible to learn an individual neuron that is explainable from the start. Unfortunately, our first result tells us that it is NP-hard to learn such a neuron. Fortunately, we have found new conditions under which we can learn an explainable neuron in pseudo-polynomial time.