Conference Paper
Machine Learning

Amnesiac Machine Learning

Laura M. Graves(University of Waterloo), Vineel Nagisetty(University of Waterloo), Vijay Ganesh(University of Waterloo)
May 18, 2021AAAI Conference on Artificial Intelligence458 citations
arXivPDFDOI

458

Citations

62

Influential Citations

AAAI Conference on Artificial Intelligence

Venue

2021

Year

Abstract

The Right to be Forgotten is part of the recently enacted General Data Protection Regulation (GDPR) law that affects any data holder that has data on European Union residents. It gives EU residents the ability to request deletion of their personal data, including training records used to train machine learning models. Unfortunately, Deep Neural Network models are vulnerable to information leaking attacks such as model inversion attacks which extract class information from a trained model and membership inference attacks which determine the presence of an example in a model's training data. If a malicious party can mount an attack and learn private information that was meant to be removed, then it implies that the model owner has not properly protected their user's rights and their models may not be compliant with the GDPR law. In this paper, we present two efficient methods that address this question of how a model owner or data holder may delete personal data from models in such a way that they may not be vulnerable to model inversion and membership inference attacks while maintaining model efficacy. We start by presenting a real-world threat model that shows that simply removing training data is insufficient to protect users. We follow that up with two data removal methods, namely Unlearning and Amnesiac Unlearning, that enable model owners to protect themselves against such attacks while being compliant with regulations. We provide extensive empirical analysis that show that these methods are indeed efficient, safe to apply, effectively remove learned information about sensitive data from trained models while maintaining model efficacy.

Analysis

Why This Paper Matters

With the enactment of GDPR, organizations holding personal data of EU residents must honor deletion requests, including data used to train machine learning models. This paper addresses a critical gap: simply removing training records does not prevent the model from retaining and leaking that information through attacks like model inversion (extracting class information) and membership inference (determining if a specific example was in the training set). The authors demonstrate that without proper unlearning, model owners remain vulnerable to privacy breaches and potential non-compliance.

The significance is heightened by the growing deployment of deep neural networks in sensitive domains such as healthcare, finance, and personalized services. The paper provides a practical path forward by introducing two concrete methods that balance privacy protection with model utility, making it highly relevant for AI practitioners who need to operationalize the Right to be Forgotten.

Technical Contributions

  • Threat Model: Clearly defines a real-world scenario where an attacker can exploit a model trained on data that was supposedly deleted, highlighting the inadequacy of naive data removal.
  • Unlearning Method: A straightforward approach that retrains the model from scratch on the dataset excluding the deleted records. While computationally expensive, it serves as a baseline for effectiveness.
  • Amnesiac Unlearning: A more efficient method that incrementally updates model weights to reverse the influence of specific training examples without full retraining. This is achieved by storing per-example gradients and applying negative updates during deletion.
  • Empirical Validation: Extensive experiments on multiple datasets (e.g., MNIST, CIFAR-10) demonstrate that both methods reduce attack success rates for model inversion and membership inference to near-random levels while maintaining classification accuracy within 1-2% of the original model.

Results

The paper reports that after applying either unlearning method, membership inference attack success rates drop from over 70% (on the original model) to approximately 50% (random guessing). Model inversion attacks similarly become ineffective, with reconstructed images no longer resembling the deleted class. Accuracy on the remaining test set remains within 1-2% of the original model's performance. Amnesiac Unlearning achieves these results with orders of magnitude less computation than full retraining, making it practical for frequent deletion requests.

Significance

This research directly enables GDPR compliance for machine learning systems, reducing legal liability for organizations. It also advances the field of model unlearning, providing a foundation for future work on efficient, provable data deletion. For AI practitioners, the methods offer a clear trade-off: full Unlearning for maximum safety, or Amnesiac Unlearning for efficiency. The paper's emphasis on empirical attack resistance sets a standard for evaluating unlearning techniques beyond simple accuracy metrics.

machine-learninggdprright-to-be-forgottendata-privacymodel-unlearningdeep-neural-networksmembership-inferencemodel-inversion
Neural Networks

Related Papers