GRM-083-218 WISE: Whitebox Importance-based Subnetwork Extraction and the Privacy-Preserving Properties of Model Compression

GRM-083-218 WISE: Whitebox Importance-based Subnetwork Extraction and the Privacy-Preserving Properties of Model Compression

https://www.kennesaw.edu/ccse/events/computing-showcase/sp26-cday-program.php

WISE (Whitebox Importance-based Subnetwork Extraction) is a structured compression algorithm which extracts task-specific subnetworks by instrumenting a pretrained networks with learned gates on transformer components and optimizing on task loss and L0 sparsity regularization. WISE maintains high task performance at high sparsity levels (81-88% accuracy at 85%) where other SOTA methods collapse to near random chance. We present the first evaluation of model compression along privacy dimensions: attribute inference resistance, training data memorization, and extraction attack vulnerability. Structured compression via learned gates produces subnetworks with favorable privacy-utility balance without any explicit privacy mechanism. WISE masks also transfer to fresh models with recovery ratios exceeding 1.0, supporting the Lottery Ticket Hypothesis