Subspace based Federated Unlearning

Suppose that there are $N$ clients, denoted as $C_{1},...,C_{N}$, respectively. Client $C_{i}$ has a local dataset $\mathcal{D}^{i}$. The goal of traditional FL is to collaboratively learn a machine learning model $w$ over the dataset $\mathcal{D}\triangleq\bigcup_{i\in[N]}\mathcal{D}^{i}$ :

where $L_{i}(w)=\mathbb{E}_{(x,y)\sim\mathcal{D}^{i}}[\ell_{i}(w;(x,y))]$ is the empirical loss of $C_{i}$ and during federated training, each client minimizes their empirical risk $L_{i}(w)$, $w^{*}$ is the final model trained by the FL process.

Now we consider how to forget the contribution of the target client $C_{I}$. A natural idea is to increase the empirical risk $L_{C_{I}}(w)$ of the target client $C_{I}$, which is equivalent to reversing the learning process. However, simply maximizing the loss can influence the effect of the model on other clients. Federated unlearning needs to forget the contribution of the target client $C_{I}$ while ensuring the overall model performance. Thus, the objective of federated unlearning is defined below:

where $\delta$ is a small change in the empirical loss, $\mathcal{L}^{ul}()$ is the empirical loss of the FL system after removing the target client.

where $\mathcal{D}^{un}\triangleq\bigcup_{i\in[N\backslash C_{I}]}\mathcal{D}^{i}$ is the remaining data set after removing the target client.

Comparing the difference between the unlearned model and the retrained model is one of the criteria used to measure the effect of unlearning. Common dissimilarity metrics include model test accuracy difference (Bourtoule et al., 2021), $\ell_{2}$-distance (Wu et al., 2020) or, and Kullback-Leibler (KL) divergence (Sekhari et al., 2021). However, in the FL scenario, it is difficult to intuitively indicate whether the contribution of a given client can be removed from the evaluation method based on model differences. Other metrics include privacy leakage in the differential privacy framework  (Sekhari et al., 2021) and membership inference attacks (Graves et al., 2021; Baumhauer et al., 2022). In this paper, we use the backdoor triggers (Gu et al., 2017) as an effective way to evaluate the performance of unlearning methods, similar to  Wu et al. (2022). In particular, the target client uses a dataset with a certain fraction of images that have a backdoor trigger inserted in them. Thus, the global FL model becomes susceptible to the backdoor trigger. Then, a successful unlearning process should produce a model that reduces the accuracy of the images with the backdoor trigger, while maintaining good performance on regular (clean) images. Note that we use the backdoor triggers as a way to evaluate the performance of unlearning methods; we do not consider any malicious client (Xie et al., 2020; Bagdasaryan et al., 2020; Fung et al., 2020)

We introduce a novel Subspace-based federated unlearning framework, named SFU. The main insight of the SFU is that we constrain the gradients generated by the target client’s gradient ascent to the input subspace of the other clients to remove the contribution of the target client from the global model. As shown in Fig. 1, SFU participants can be divided into three kinds of roles: the target client to be forgotten, the remaining clients, and the server. The target client performs gradient ascent to upload the gradient to the server. Other clients compute the local representation matrix and upload it to the server. The server is responsible for the computation of other client input Spaces and the unlearning update of the global model. Next, we will introduce the specific tasks of the three participants respectively.

Datasets: We evaluate SFU using three popular datasets: MNIST(Xiao et al., 2017), CIFAR10, and CIFAR100 (Krizhevsky et al., 2009) as described below:

• •

  MNIST: It is a dataset that contains a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28×28 grayscale image associated with a label from 10 classes.

• •

  CIFAR10: It consists of 60,000 32 × 32 color images in 10 classes, with 6000 images per class. There are 50,000 training images and 10,000 test images.

• •

  CIFAR100: It has 100 classes. Each class has 600 color images of size 32 × 32, of which 5000 are used as a training set, and 100 are used as a test set.

The training difficulty of the dataset is increasing from MNIST, CIFAR10 to CIFAR100. We adopt two data distribution Settings, including IID (Independent-Identically-Distributed) as well as Non-IID (Non Independent-Identically-Distributed). The Non-IID setting that we adopt is Dirichlet($\mathcal{\beta}$): Label distribution on each device follows the Dirichlet distribution, where $\mathcal{\beta}$ is a concentration parameter ($\mathcal{\beta}>0$).

Baselines. We chose three typical federated unlearning algorithms as our baseline:

(1) Retraining: This method retrains the entire FL system without the target client being forgotten, which is an effective but computationally and communicationally expensive algorithm;

(2) “UL” Wu et al. (2022): This method forgets the target client by subtracting historical parameter updates of the target client from the global model. Then, “UL” uses the knowledge distillation method to remedy the skew of the unlearning model caused by the subtraction without using any data from the target clients.

(3) “GA”: GA uses gradient ascent information on the target client to update the global model. Gradient ascent makes the loss of the model increase, which is the inverse process of learning and can achieve unlearning. However, the loss of gradient ascent is unbounded so we set the gradient clip norm when the global model is updated to reduce the probability of producing a random model.

Models. We employ three neural network architectures in our experiments.

• •

  MLP: This is a fully-connected neural network architecture with 2 hidden layers. The number of neurons in the layers is 200 and 100, respectively.

• •

  CNN: This network architecture consists of 2 convolutional layers with 64 5 × 5 filters followed by 2 fully connected layers with 800 and 500 neurons and a Relu layer.

• •

  ResNet: we use a smaller version of ResNet18  (He et al., 2016), with three times fewer feature maps across all layers.

We used two networks MLP and CNN for the MNIST dataset, while we used the above three network structures for the CIFAR10 and CIFAR100 datasets. We use PyTorch (Oord et al., 2018) to implement those models.

Evaluation metric. We use backdoor attacks in the target client’s updates to the global model as described before, so that we can intuitively investigate the unlearning effect based on the attack success rate of the unlearned global model. In Tab. 1 and Tab. 2, we record the attack success rate as “atk acc”. A lower “atk acc” represents a cleaner contribution removal from the target client. In our experiments, we implement the backdoor attack using a ”pixel pattern” trigger of size 2 × 2 and change the label to ”9”. Because of the prediction error of the model, We can consider an error rate of less than 10% as successful forgetting of the target client contribution. We also use the accuracy metric on the clean test data to measure the performance of the model after unlearning which is denoted “test acc” in Tab. 1 and Tab. 2. A high accuracy indicates that unlearning has little impact on the performance of the model. Current unlearning methods usually adopt certain methods to recover the model accuracy after unlearning, so we divide our evaluation of the unlearning approach into two respects: the removal of specific client contributions and the recovery of model performance.

Implementation details. We consider two scenarios: (i) We have ten clients with one target client, and all clients participate fully during each training round. (ii) We have 100 clients with one target client and only 10% of the clients participate during each training round. In the experiment of removal of specific client contributions, We conduct unlearning experiments on the FL model after 100 rounds of training; In the experiment of recovery of model performance, We start FL training with the stochastic model for full retraining. we start FL training on the unlearned local model without the involvement of the target client for SFU and GA. We use the knowledge of the public data on the server for distillation learning to recover the model accuracy for UL. Tab.  2 and Tab.  1 only show the results of scenario (i). Results of scenario (ii) and other implementation details are shown in Appendix A due to limited space.

We conducted extensive experiments to determine the advantage of SFU in removing the contribution of a specific client in the global model. In addition, we demonstrate the robustness and superiority of SFU in different training degrees of the model and different data heterogeneity. All results are reported based on the global model after performing unlearning. The results show that SFU can effectively remove the contribution of target clients under the premise of ensuring model accuracy, and has strong robustness to data heterogeneity and the training degree of the model.

Efficient forgetting by SFU. Tab. 1 reports the unlearning effects of SFU and the other baselines on different IID datasets and model architectures. The results show that SFU can efficiently complete the contribution removal of target clients. We observe that SFU successfully eliminates the contribution of target customers and achieves low precision for backdoor data. For example, SFU can reduce the backdoor attack success rate to or close to 0% in all datasets. Other baselines also can achieve similar results. This shows that the current unlearning methods can effectively remove the contribution of the target client.

High model accuracy for SFU. The results in Tab.  1 also report the accuracy of the model on the clean test set data after unlearning for each baseline. The results show that SFU achieves the best results on various datasets and different model architectures. However, the UL method will cause great damage to the performance of the model, and the direct gradient ascent has a large probability of producing a low-accuracy model. For example, on the CNN architecture of the MNIST dataset, the accuracy of the UL model is only 42.76%, which is 56.55% lower than the accuracy of the original FL model 99.31%. The accuracy of the model generated by GA is 92.33%, which is 7% lower than the 99.29% accuracy of the model generated by SFU, which indicates that restricting the gradient in the orthogonal subspace of the input space can reduce the loss of the original model performance.

Robustness of the original model maturity. Federated unlearning can occur at any stage of FL, so it is possible that the model does not converge when federated unlearning is performed. We define the maturity of the model in terms of the number of rounds of model training, where the more rounds the model is trained, the more mature it is. Fig. 3 reports the effect of each baseline with unlearning at different rounds of the FL model training. The results show that SFU can ensure the performance of the model to the greatest extent at any round of model training, and the accuracy of SFU is similar to that of FL model training. UL is easy to remove in the early stage of model training. However, as the model continues to train, the integration degree of each client model increases, simply subtracting the historical weight of the client is easy to cause a significant decline in the accuracy of the model. For example, the UL accuracy in the MNIST dataset dropped from 97% to 66% as the model trained. GA has similar results to SFU, but still has lower accuracy than SFU, which indicates that SFU is robust to the original model maturity.

Robustness on heterogeneous data. We test different unlearning algorithms on MNIST datasets with different degrees of heterogeneity, and ten clients are selected to form the FL system. Fig. 3 shows that enhancing data distribution heterogeneity will reduce the effect of various unlearning algorithms. It is because the aggregation of the global model becomes more complex with the increase of model heterogeneity, which makes it more challenging to separate the contributions of specific clients under the premise of ensuring model accuracy. However, the accuracy of SFU is close to that of the original FL model and the retrained model in all Settings, which indicates that SFU is robust to data heterogeneity.

Below, we report the accuracy of the final model and the recovery efficiency of different methods. All results are reported based on the global model after unlearning.

High final accuracy of the SFU. Tab. 2 reports the model performance of each algorithm after 10 rounds of accuracy recovery training and after 100 rounds of full retraining. We find that almost all methods can achieve a low success rate of backdoor attacks except for GA. The reason why GA has a high success rate of backdoor attacks is that the algorithm after unlearning is still near the original model, and it is easy to converge to the original model with high accuracy and a high success rate of backdoor attacks. The final accuracy of SFU is the highest among all the methods, and sometimes it can even exceed the accuracy of retraining, which proves that updating in the subspace will produce a better-initialized model to achieve higher accuracy.

High-speed precision recovery for SFU. To compare the model accuracy recovery speed of SFU with each baseline, we calculate the accuracy and backdoor attack success rate of different methods in terms of the number of FL training rounds after unlearning, and the results are shown in Fig. 4. We observe that SFU is able to achieve high accuracy after one round of training, while other methods require 5 or even more rounds of training to achieve. SFU is more efficient in terms of computation and communication cost on the retained clients than the baseline of retraining while achieving comparable backdoor accuracy.