Synthetic and (Un)Secure: Evaluating Generalized Membership Inference Attacks on Image Data

Coscia P., Ferrari S., Piuri V., SALMAN A.

22nd International Conference on Security and Cryptography, SECRYPT 2025, Bilbao, İspanya, 11 - 13 Haziran 2025, cilt.1, ss.287-297, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 1
  • Doi Numarası: 10.5220/0013657700003979
  • Basıldığı Şehir: Bilbao
  • Basıldığı Ülke: İspanya
  • Sayfa Sayıları: ss.287-297
  • Anahtar Kelimeler: Fréchet Coefficient, Generative Models, Membership Inference Attack
  • Maltepe Üniversitesi Adresli: Evet

Özet

Synthetic data are widely employed across diverse fields, including computer vision, robotics, and cybersecurity. However, generative models are prone to unintentionally revealing sensitive information from their training datasets, primarily due to overfitting phenomena. In this context, membership inference attacks (MIAs) have emerged as a significant privacy threat. These attacks employ binary classifiers to verify whether a specific data sample was part of the model’s training set, thereby discriminating between member and nonmember samples. Despite their growing relevance, the interpretation of MIA outcomes can be misleading without a detailed understanding of the data domains involved during both model development and evaluation. To bridge this gap, we performed an analysis focused on a particular category (i.e., vehicles) to assess the effectiveness of MIA under scenarios with limited overlap in data distribution. First, we introduce a data selection strategy, based on the Frechet Coefficient, to filter and curate the evaluation datasets, followed by ´ the execution of membership inference attacks under varying degrees of distributional overlap. Our findings indicate that MIAs are highly effective when the training and evaluation data distributions are well aligned, but their accuracy drops significantly under distribution shifts or when domain knowledge is limited. These results highlight the limitations of current MIA methodologies in reliably assessing privacy risks in generative modeling contexts.