USING COGNITIVE PSYCHOLOGY TO PROBE AI SOCIAL BIAS
Disciplines
Cognitive Psychology | Psychology
Abstract (300 words maximum)
Human rationality and decision making is heavily susceptible to social and cognitive biases. This irrationality in human nature poses an intriguing question: Does artificial intelligence display the same heuristics as humans? The current study seeks to examine social essentialism, the belief social groups possess natural or biological underpinnings, in GPT-4. This research builds upon recent studies that have tested prominent cognitive biases (e.g., anchoring and representative heuristics) using word vignettes by building on social essentialist bias. Our goal is to understand the differences between social essentialist thinking in large language models compared to humans. Specifically, we will examine two dimensions within social essentialism - Naturalness, or the belief in immutable and naturally occurring boundaries within social groups, and cohesiveness, or the belief in uniform characteristics within social groups. We utilized the social essentialism scale, a 9-point likert system, to observe whether GPT-4 would exhibit similar heuristic patterns in race, gender, nationality, religion, and social class domains. Our previous study that we are building upon showcased GPT-4 scoring lower than humans in terms of economic, race, and nationality social groups but higher in terms of the religious domain. This key finding prompted us to explore this phenomenon on a deeper level and enhance our methodologies and participant data. We will utilize additional methodology such as the switch-at-birth task which analyzes participants' essentialist thinking regarding individuals being born into social groups versus becoming a part of differing groups. We will also expand our human participant data collection methods by gathering a sample of U.S adult participants to test our methodologies on and later compare with GPT-4. Overall, we hope to gain a deeper command of artificial intelligences’ susceptibility to essentialist bias comparatively to humans and its understanding of social groupings.
Academic department under which the project should be listed
RCHSS - Psychological Science
Primary Investigator (PI) Name
Yian Xu
USING COGNITIVE PSYCHOLOGY TO PROBE AI SOCIAL BIAS
Human rationality and decision making is heavily susceptible to social and cognitive biases. This irrationality in human nature poses an intriguing question: Does artificial intelligence display the same heuristics as humans? The current study seeks to examine social essentialism, the belief social groups possess natural or biological underpinnings, in GPT-4. This research builds upon recent studies that have tested prominent cognitive biases (e.g., anchoring and representative heuristics) using word vignettes by building on social essentialist bias. Our goal is to understand the differences between social essentialist thinking in large language models compared to humans. Specifically, we will examine two dimensions within social essentialism - Naturalness, or the belief in immutable and naturally occurring boundaries within social groups, and cohesiveness, or the belief in uniform characteristics within social groups. We utilized the social essentialism scale, a 9-point likert system, to observe whether GPT-4 would exhibit similar heuristic patterns in race, gender, nationality, religion, and social class domains. Our previous study that we are building upon showcased GPT-4 scoring lower than humans in terms of economic, race, and nationality social groups but higher in terms of the religious domain. This key finding prompted us to explore this phenomenon on a deeper level and enhance our methodologies and participant data. We will utilize additional methodology such as the switch-at-birth task which analyzes participants' essentialist thinking regarding individuals being born into social groups versus becoming a part of differing groups. We will also expand our human participant data collection methods by gathering a sample of U.S adult participants to test our methodologies on and later compare with GPT-4. Overall, we hope to gain a deeper command of artificial intelligences’ susceptibility to essentialist bias comparatively to humans and its understanding of social groupings.