A Principled Approach to Data Valuation for Federated Learning

A typical FL process executes the following steps repeatedly until some stopping criterion is met: (1) The coordinator samples a subset of participants; (2) The selected participants download the current global model parameters from the coordinator; (3) Each selected participant locally computes an update to the model by training on the local data; (4) The coordinator collects an aggregate of the participant updates; (5) The coordinator locally updates the global model based on the aggregated update computed from the participants that participate in the current round.

Let $I$ be the set of participants that participate in at least one round of the FL process. Our goal is to assign a real value to each participant in $I$ to measure its contribution to learning the model. Suppose the learning process lasts for $T$ rounds. Let the participants selected in round $t$ be $I_{t}$ and we have $I=I_{1}\cup\cdots\cup I_{T}$.

In FL, different participants contribute to the learning process at different time and the performance of the federated model depends on the participation order of participants. Clearly, the symmetric utility assumption of the SV does not hold. Moreover, FL is designed to maintain the confidentiality of participants’ data and in each round, only a subset of participants are selected and upload their model updates. Hence, the coordinator can only know the model performance change caused by adding a participant’s data into the subset of participants’ data selected earlier. However, computing the SV requires the ability to evaluate the model performance change for every possible subset of participants. Unless the participants are able to bear considerable extra communication cost, the combinatorially evaluable utility assumption is invalid for FL. Hence, the SV cannot be used to value the data of different participants in the federated setting.

We propose a variant of the SV amenable to the federated setting. The key idea is to characterize the aggregate value of the set of participants in the same round via the model performance change caused by the addition of their data and then use the SV to distribute the value of the set to each participant. We will call this variant the federated SV and its formal definition is given below. We use $\nu(\cdot)$ to denote the utility function which maps any participants’ data to a performance measure of the model trained on the data. Note that unlike in the canonical SV definition where $\nu(\cdot)$ takes a set as an input, the argument of $\nu(\cdot)$ is an ordered sequence. For instance, $U(A+B)$ means the utility of the model that is trained on A’s data first, then B’s data. Furthermore, let $I_{1:t-1}$ be a shorthand for $I_{1}+\cdots+I_{t-1}$ for $t\geq 2$ and $\emptyset$ for $t=1$.

We will suppress the dependency of the federated SV $s^{\nu}(i)$ on $\nu$ whenever the underlying utility function is self-evident.

Due to the close relation between the canonical SV and the federated SV, one can expect that the federated variant will inherit the desirable properties of the canonical SV. Indeed, Theorem 1 shows that the federated SV preserves the group rationality, fairness, as well as additivity.

The proof of the theorem follows from the fact that the federated Shapley value calculates the Shapley value for the players selected in each round which distributes the performance difference from the previous round.

By aggregating the instantaneous group rationality equation over time, we see that the federated SV also satisfies the long-term group rationality:

The long-term group rationality states that the set of players participates in a $T$-round FL process will divide up the final yield of their coalition.

Similar to the canonical SV, computing the federated SV is expensive. Evaluating the exact federated SV involves computing the marginal utility of every participant to every subset of other participants selected in each round (see Eqn. 3). To evaluate $U(I_{1:t-1}+S)$, we need to update the global model trained on $I_{1:t-1}$ with the aggregate of the model updates from $S$ and calculate the updated model performance. The total complexity is $\mathcal{O}(T2^{m})$, where $m$ is the maximum number of participants selected per round. In this section, we present efficient algorithms to approximate the federated SV. We say that $\hat{s}\in\mathbb{R}^{N}$ is a $(\epsilon,\delta)$-approximation to the true SV $s=[s_{1},\cdots,s_{N}]^{T}\in\mathbb{R}^{N}$ if $Pr[||\hat{s}_{i}-s_{i}||_{\infty}\leq\epsilon]\geq 1-\delta$. These algorithms utilize the existing approximation methods developed for the canonical SV [23, 12] to improve the efficiency of per-round federated SV calculation.

The idea of the first approximation algorithm is to treat the Shapley value of a participant as its expected contribution to the participants before it in a random permutation using Eqn. 2 and use the sample average to approximate the expectation. We will call this algorithm permutation sampling-based approximation hereinafter and the pseudocode is provided in Algorithm 2. An application of Hoeffding bound indicates that to achieve $(\epsilon,\delta)$-approximation in each round of updating, the number of utility evaluations required for $T$ rounds is $Tm(\frac{2r^{2}}{\epsilon^{2}})\log(\frac{2m}{\delta})$.

The second approximation algorithm makes use of the group testing technique [11] to estimate the per-round federated SV and we will call this algorithm group testing-based approximation. In our scenario, each ”test” corresponds to evaluating the utility of a subset of participant updates. The key idea of the algorithm is to intelligently design the sampling distribution of participants’ updates so that we can calculate Shapely differences between the selected participants from the test results with high-probability bound on the error. Based on the result in [11], the number of tests required to estimate the Shapley differences up to $(\frac{\epsilon}{C_{\epsilon}},\frac{\delta}{C_{\delta}})$ is $T_{1}=\frac{4}{(1-q_{tot}^{2})h(\frac{2\epsilon}{ZrC_{\epsilon}(1-q_{tot}^{2})})}\log(\frac{C_{\delta}(m-1)}{2\delta})$, where $h(u)=(1+u)\log(1+u)-u$ and other variables are defined in Algorithm 3. We can then take any participant $i$ and estimate the corresponding SV using the permutation sampling-based approximation, denote it as $s_{*}$. Then, the SV of all other $m-1$ users can be estimated using the estimated difference of the SV with participant $i$ (we choose the $m$th participant as the pivot participant in the pseudo-code in Algorithm 4). The number of utility evaluation required for estimating $s_{*}$ up to $(\frac{(C_{\epsilon}-1)\epsilon}{C_{\epsilon}},\frac{(C_{\delta}-1)\delta}{C_{\delta}})$ is $T_{2}=\frac{4r^{2}C_{\epsilon}^{2}}{(C_{\epsilon}-1)^{2}\epsilon^{2}}\log(\frac{2C_{\delta}}{(C_{\delta}-1)\delta})$. $C_{\epsilon},C_{\delta}$ are chosen so that $T_{1}+T_{2}$ are minimized. Algorithm 1, 2, 3, and 4 present the pseudo-code for both permutation sampling and group testing.

If we treat $\epsilon,\delta$ as constant, $T_{1}+T_{2}\sim\mathcal{O}((\log m)^{2})$ while permutation sampling-based approximation is $\mathcal{O}(m\log m)$. Therefore, when the number of selected participants in each round is large, group testing-based approximation is significantly faster than permutation sampling-based one. One the other hand, when the number of selected participants is small, permutation sampling-based approximation is more preferable since its utility evaluation complexity tends to have a smaller constant.

We will compare the federated SV with the following two baselines.

Federated Leave-One-Out One natural way to assign the contribution to a participant update $i$ at round $t$ is by calculating the model performance change when the participant is removed from the set of participants selected at round $t$, i.e., $loo_{t}(i)=U(I_{1:t})-U(I_{1:t-1}+I_{t}/\{i\})$, and $loo_{t}(i)=0$ if participant $i$ is not selected in round $t$. The Leave-One-Out (LOO) value for FL takes the sum of the LOO values of all rounds: $loo(i)=\sum_{t=1}^{T}loo_{t}(i)$.

Random The random baseline does not differentiate between different participants’ contribution and just randomly selects participants to perform a given task.

In the figures, we will use Fed. LOO and Fed. SV to denote federated leave-one-out and federated Shapley Value approach, respectively.

For each task, we perform experiments on the MNIST [20] as well as the CIFAR10 dataset [19]. Following [24], we study two ways of partitioning the MNIST data over participants: IID, where the data is shuffled, and then partitioned into 100 participants each receiving 600 examples, and Non-IID, where we first sort the data by digit label, divide it into 200 shards of size 300, and assign each of 100 participants 2 shards. For MNIST, we train a simple multilayer-perceptron (MLP) with 2-hidden layers with 200 neurons in each layer and ReLu activations as well as a simple CNN. For all experiments on CIFAR10, we train a CNN with two 5x5 convolution layers (the first with 32 channels, the second with 64, each followed by 2x2 max pooling), a fully connected layer with 512 neurons with ReLu activation, and a final softmax output layer. In each round of training, we randomly select 10 participants out of 100, unless otherwise specified. We run 25 rounds for training on MNIST, achieving up to 97% and 92% global model accuracy for the IID and the non-IID setting, respectively. For CIFAR10, we run up to 50 to 200 rounds of training. We achieve up 77% and 70% test accuracy in IID and non-IID setting, respectively, for 200 rounds of training. As a side note, the state-of-the-art models in [17] can achieve test accuracy of 99.4% for CIFAR10; nevertheless, our goal is to evaluate the proposed data value notion rather than achieving the best possible accuracy. We use the permutation sampling approach in Algorithm 2 to estimate the Shapley value in all experiments since the number of participants is small.

Labels in the real world are often noisy due to annotators’ varying skill-levels, biases or malicious tampering. We show that the proposed data value notion can help removing the noisy participants. The key idea is to rank the participants according to their data value, and drop the participants with the lowest values.

We set 20 participants’ local data to be noisy where noise flipping ratio is 10% for MNIST, and 3% for CIFAR10. The performances of different data value measures are illustrated in Figure 1a and 1b. We inspect the label of participant’s local training instances that have the lowest scores, and plot the change of the fraction of detected noisy participants with the fraction of the inspected participants. We can see that when the training data is partitioned in IID setting, federated LOO and federated SV perform similarly. However, in the Non-IID setting, the federated SV outperforms federated LOO. We conjecture that this is because for Non-IID participants, the trained local models tend to overfit, diverge from the global model, and exhibit low accuracy. In comparison with the federated SV, federated LOO only computes the marginal contribution of a participant to the largest subset of other selected participants and therefore the noisy participants are harder to be identified by federated LOO.

We also find that, with the number of training rounds increases, the total contribution of participants in each round will decrease, as shown in Figure 2a. This makes sense since the federated SV satisfies instantaneous group rationality in Theorem 1, and the improvement of global model’s utility will slowdown when it is close to convergence. That is, it is relatively easy to improve the global model’s utility in earlier rounds, while harder to further improve the utility in later rounds. Hence, the contribution of participants selected in early rounds is inflated. This inspires us to consider a variant of data value measures, which normalize the per-round data values by their norms and then aggregate them across all rounds. The performance of noisy label detection with the normalized versions of federated SV and federated LOO is shown in Figure 1c and 1d. As we can see, it is much easier to separate noisy participants from benign participants with the normalized version of data value notions. However, the normalized federated SV no longer preserves the group rationality and additivity property. We leave developing more detailed analysis of different variants of data value as future work.

Motivated by privacy concerns, in FL, the coordinator is designed to have no visibility into a participant’s local data and training process. This lack of transparency in the agent updates can be exploited so that an adversary controlling a small number of malicious participants can perform a backdoor attack. The adversary’s objective is to cause the jointly trained global model to misclassify a set of chosen inputs, i.e. it seeks to poison the global model in a targeted manner, while also ensures that the global model has a good performance on the clean test data. We focus on backdoor attacks based on the model replacement paradigm proposed by [2].

For CIFAR10, following the settings in [2], we choose the feature of vertically stripped walls in the background (12 images) as the backdoor. For MNIST, we implement pixel-pattern backdoor attack in [7]. We set the ratio of the participants controlled by the adversary to be 30%. We mix backdoor images with benign images in every training batch (20 backdoor images per batch of size 64) for compromised participants, following the settings in [2].

In Figure 3a and 3b, we show the success rate of backdoor detection with respect to the fraction of checked participants. Both of the figures indicate that federated SV is a more effective than federated LOO for detecting compromised participants. In the Non-IID setting, both compromised participants and some benign participants tend to have low contribution on the main task performance, which makes the compromised participants more difficult to be identified by the low data values. Hence, we also test the performance of normalized version of federated SV/LOO for this task and Figure 3c and 3d show that the performance improves a lot compared with the original definitions.

In our data summarization experiments, we investigate whether the federated SV can facilitate federated training by identifying the most valuable participants. Per communication round, a percentage of the selected participants is ignored for the update of the global model. We use data value measures to dismiss participants that are expected to contribute the least to the model accuracy. The data values are calculated on a separate validation set, which contains 1000 and 800 random samples for MNIST and CIFAR10, respectively. During each communication round of FL, we compute the data value summands. After training has finished, we compute the total data value.

We then repeat training, while maintaining an identical selection of participants per round. During each round, we dismiss a certain fraction $q\in[0,0.1,\ldots,0.9]$ of the selected participants. We compute and average the results for the random baseline three times per run.

We train a small CNN model on the MNIST dataset. The CNN consists of two 5x5 convolution layers, each followed with 2x2 max pooling and ReLu activations. Two fully connected layers (input, hidden and output dimensions of 320, 50, 10, respectively) with intermediate ReLu activation follow the second convolution layer. We apply dropout on the second convolution and first fully connected layer. For CIFAR10, we operate on 1000-dimensional feature vectors extracted with an imagenet-pretrained MobileNet v2 mode.¹¹1We use preprocessing and the pretrained model as provided by PyTorch Hub We train a MLP with 2-hidden layers with 1000 neurons in each layer and ReLu activations.

We evaluate our algorithm for FL of 10 rounds on MNIST and 100 rounds on CIFAR10. The results of our experiments are shown in Figure 4. For the MNIST IID case, the federated SV approach outperforms both baselines. While it also consistently outperforms the random baseline in the non-IID setting, federated LOO achieves higher test accuracies for lower fractions of dismissed samples. Here, analysis of the federated SV per participant shows that it tends to be higher for participants that are selected throughout the FL. Furthermore, we observe that participants that were sampled few times also are more likely to have a negative federated SV, compared to the IID setting. We hypothesize that this bias negatively affects the performance of the federated SV-based summarization in the non-IID setting.

We also observe that both federated SV and LOO perform worse on the CIFAR10 dataset summarization. We hypothesize that selection of good participant subsets is more effective on the MNIST dataset, as it contains a larger portion of redundant samples. Consequently, valuable information is less likely to be lost by dismissal of a fraction of participants.