Evaluate Inference Attacks: Attack and Defense against 2D Semantic Segmentation Models
Abstract
Evaluate Inference Attacks: Attack and Defense against 2D Semantic Segmentation Models YIHAN LIAO, City University of Hong Kong, China JACKY KEUNG, City University of Hong Kong, China JINGYU ZHANG∗ , City University of Hong Kong, China YUROU DAI, Lehigh University, USA SHUO LIU, City University of Hong Kong, China Deep learning (DL)-based 2D semantic segmentation (SS) plays a vital role in the perception task of autonomous driving. However, the SS model relies on DL, which makes it vulnerable to inference attacks. Recent research has discovered that SS models are susceptible to the membership inference attack, yet other inference attacks remain underexplored. Our study fills this gap by comprehensively investigating the vulnerabilities of two widely used RGB image-based 2D SS models (DeepLabV3 and DeepLabV3+) against three inference attacks: membership inference, attribute inference, and model inversion. We evaluate the attack effectiveness on three backbones (MobileNetV2, ResNet50, and ResNet101) across three datasets (VOC2012, CityScapes, and ADE20K), where the attack accuracy can reach up to 95% (membership inference), 40% (attribute inference), and 70% (model inversion), revealing that deeper networks are more prone to privacy leakage in inference attacks. Consequently, we introduce differential privacy and model pruning as defensive mechanisms, significantly reducing attack performance, where the average accuracy drops 20% among the three inference attacks. Our findings reveal critical privacy vulnerabilities in SS tasks and offer practical guidance for developing more robust SS models in autonomous driving. CCS Concepts: • Security and privacy → Privacy-preserving protocols; • Computing methodologies → Computer vision; Machine learning. Additional Key Words and Phrases: Semantic segmentation, Inference attacks, Differential privacy, Model pruning