Finding the Proverbial Needle: Improving Minority Class Identification Under Extreme Class Imbalance
Department
Computer Science
Document Type
Article
Publication Date
4-1-2023
Abstract
Imbalanced learning problems typically consist of data with skewed class distributions, coupled with large misclassification costs for the rare events. For binary classification, logistic regression is a common supervised learning technique chosen to perform this task. Unfortunately, the model performs poorly on classification tasks when class distributions are highly imbalanced. To improve this generalization, we implement a novel instance-level weighting methodology for the minority class in the loss function. We build our method from a recently published, locally weighted log-likelihood objective function, where each of the minority class weights are learned from the data. We improve upon this previous approach by creating a convex and hyperparameter-free loss function that improves generalization performance for datasets exhibiting extreme class imbalance.
Journal Title
Journal of Classification
Journal ISSN
01764268
Volume
40
Issue
1
First Page
192
Last Page
212
Digital Object Identifier (DOI)
10.1007/s00357-023-09431-5