Social Welfare Maximization in cross-silo Federated Learning

In Federated Learning (FL), clients cooperatively train a Machine Learning (ML) model with their decentralized datasets under the coordination of a central server[1]. One of the typical settings of FL is cross-silo FL[2] where a neutral third-party agent acts as the central server and clients are a group of organizations, aiming to jointly train an optimal ML model for their respective use. In this case, these organizations are also the owners of the global model and can utilize the well-trained global model to further process tasks for their own interests. ¹¹1This work is partly supported by the US NSF under grant CNS-2105004. ²²footnotetext: ${\star}$ Corresponding author. An optimal global model with high performance requires the organizations in cross-silo FL to collaborate efficiently so as to bring considerable benefits to all participants, which can be regarded as the maximization of the social welfare. In fact, there are many studies on optimizing the social welfare in cross-silo FL to directly improving the model performance[3, 4, 5, 6], increasing the convergence speed[7], reducing the communication cost[8], protecting privacy[9, 10, 11] and security[12], etc.

However, since every organization participating in cross-silo FL can obtain the final global model regardless of its contribution, the well-trained model becomes a public good, which is non-excludable and non-rivalrous for all organizations[13]. This leads to the emergence of selfish behaviors that some organizations may only consider their own interests via inactively participating in local training while obtaining the final global model for free or at a lower cost. The spread of this behavior can result in huge loss of the social welfare, and then none of the organizations can get the optimal model, which compromises the long-term stability and sustainability of cross-silo FL.

The existing studies motivate organizations to fully contribute to cross-silo FL by designing an incentive mechanism[13]. However, this requires extra negotiation costs since organizations need to reach a consensus on the mechanism in advance, and demands additional running costs where a distributed algorithm runs over all organizations, clearly adding more burden to organizations.

In this paper, we take a brand-new approach using the Multi-player Multi-action Zero-Determinant (MMZD) strategy[14] to maximize the social welfare in cross-silo FL without causing additional costs for organizations. Another outstanding advantage of our method is that it can control the social welfare and be applied to any cross-silo FL scenario no matter what strategies or actions organizations perform. In summary, our contributions include:

We consider a cross-silo FL scenario with a set of organizations, denoted as $\mathcal{N}=\{1,2,...,N\}$. All organizations rely on a central server to collaboratively conduct global model training for a specific task, where each of them has their own data for local training. The goal of organizations is to obtain an optimal global model, minimizing the loss with all datasets. The central server collects the results of local model updates from all organizations, aggregates to obtain the global model, and then distributes it to everyone for the next round of training.

In each round of local training, every organization performs $K$ iterations of model training. We denote the number of global aggregations as $r$. For the current task, the strategy of organization $i\in\mathcal{N}$, denoted as $y_{i}\in\{0,1,...,r\}$, represents the number of global aggregation rounds it participates in the task. Then, $\mathbf{y}=(y_{1},...,y_{i},...,y_{N})$ denotes the strategy vector of all organizations. Here we assume that all organizations in this cross-silo FL may participate in fewer global aggregations due to laziness or selfishness, but they do not carry out malicious attacks, such as model poisoning attack.

According to the cross-silo FL model, all organizations get the same model in return. Inspired by [13], we define the revenue of organization $i$ as

where $m_{i}$ denotes the unit revenue of organization $i$ by using the returned final model, $\chi_{0}$ denotes the precision of the untrained model, and $\chi(\mathbf{y})$ denotes the precision of the trained global model with corresponding strategy vector $\mathbf{y}$. $\chi(\mathbf{y})$ can be modeled as $\chi(\mathbf{y})=\frac{\theta_{0}}{\theta_{1}+K\sum_{i\in\mathcal{N}}y_{i}}$, with positive coefficients $\theta_{0}$ and $\theta_{1}$ [15] being derived based on the loss function, neural network, and local datasets. In particular, we have $\chi_{0}=\frac{\theta_{0}}{\theta_{1}}$. The revenue of each organization is proportional to the difference between the expected loss after $r$ rounds aggregation (i.e.,$\chi(\mathbf{y})$) and the minimum expected loss (i.e.,$\chi_{0}$)[13]. As the number of total participation rounds increases, the marginal decrease of the difference reduces.

We define the cost of organization $i$ as $\Psi_{i}(y_{i})=C^{i}_{p}+C^{i}_{m}$. The cost is composed of the organization’s computation cost $C^{i}_{p}$ and its communication cost $C^{i}_{m}$. On the one hand, the computation cost $C^{i}_{p}=\beta_{i}Ky_{i}$, where $\beta_{i}$ is a positive parameter, denoting the computation cost of each iteration in organization $i$’s local training¹¹1As [16] shows, $\beta_{i}=\frac{\alpha_{i}}{2}f^{2}_{i}D_{i}S_{i}$, where $\frac{\alpha_{i}}{2}$ is the effective capacitance coefficient of organization $i$’s computing chipset, $f_{i}$ denotes the calculation processing capacity, $D_{i}$ denotes the number of data units, and $S_{i}$ denotes the number of CPU cycles required by organization $i$ to process one data unit.. On the other hand, the communication cost $C^{i}_{m}$ is defined as a parameter since we assume that every organization uploads its updates in each global aggregation round. If it chooses not to participate in global aggregation, it will submit a zero vector as updates.

Then the utility of organization $i$ is defined as the difference between its revenue and cost:

According to previous statements, we model the interactions among organizations as a cross-silo FL game.

The cross-silo FL game can be iterative since these organizations in cross-silo FL usually cooperate for a long time to finish multiple FL tasks. Each game round in the cross-silo FL game corresponds to a certain FL task. Moreover, the social welfare in cross-silo FL game can be denoted as the total utility of all organizations $i$, namely $\Sigma^{N}_{i=1}U^{i}(\mathbf{y})$. In the cross-silo FL game, we find that the social dilemma occurs if $\Phi_{i}(\mathbf{y})-C^{i}_{p}<0$, which can be summarized as below.

The condition $\Phi_{i}(\mathbf{y})-C^{i}_{p}<0$ in the above theorem indicates that if any organization $i\in\mathcal{N}$ only trains the local model using its local dataset, the utility is negative. In fact, this condition strengthens organizations’ motivation to participate in global training in cross-silo FL game.

According to the analysis above, we can see that the underlying cause of the social dilemma is selfishness, leading to the loss of all organizations, namely the low social welfare. Aiming to solve this problem, we resort to the Multi-player Multi-action Zero-Determinant (MMZD) strategy for the social welfare maximization in this section. In each game round, any organization can choose the action $y_{i}\in\{0,1,...,r\}$, so there are $(r+1)^{N}$ possible outcomes for each game round. We assume that the organizations have one-round-memory since a long-memory player has no priority against others with short memory[14]. For arbitrary organization $i\in\mathcal{N}$, its mixed strategy $\mathbf{p}^{i}$ is defined as:

where $p^{i}_{j,g}(j\in\{1,2,...,(r+1)^{N}\},g\in\{0,1,...,r\})$ represents the probability of organization $i$ choosing action $y_{i}=g$ in the current game round and other organizations’ choosing the same actions as $j$-th outcome of the previous game round. In addition, the corresponding utility vector $\mathbf{u}^{i}$ is denoted as:

where each utility $u^{i}_{j,g}$ of organization $i$ choosing action $y_{i}=g$ in the $j$-th outcome can be calculated by $u^{i}_{j,g}=U^{i}(\mathbf{y}^{(j,g)})$, with $\mathbf{y}^{(j,g)}$ denoting the action vector $\mathbf{y}$ corresponding to the $j$-th outcome but $y_{i}=g$.

In the cross-silo FL model, an organization’s current move depends only on its last action and the strategy vector $\mathbf{y}$ in the last game round. We can construct a Markov matrix $\mathbf{M}=[M_{vw}]_{(r+1)^{N}\times(r+1)^{N}}$, with each element $M_{vw}$ denoting the one-step transition probability from state $v$ to $w$.

As demonstrated in[14], a certain column $\hat{\mathbf{p}}^{i}$ of $\mathbf{M}^{\prime}\equiv\mathbf{M}-\mathbf{I}$ is only controlled by organization $i$’s strategy. We assume the stationary vector of $\mathbf{M}$ is $\mathbf{v}$, then organization $i$’s expected utility in the stationary state is:

which makes a linear combination of all organizations’ expected utilities yielding the following equation:

In the above equation, $\alpha_{0}$ and $\alpha_{x}(x\in\mathcal{N})$ are constants. Thus, when organization $i$ chooses a strategy satisfying $\hat{\mathbf{p}}^{i}=\phi(\sum_{x=1}^{N}\alpha_{x}\mathbf{u}^{x}+\alpha_{0}\mathbf{1})$, where $\phi$ is a non-zero constant and $\hat{\mathbf{p}}^{i}$ is under the control of organization $i$, the corresponding column of $\hat{\mathbf{p}}^{i}$ and the last column of $\det(\mathbf{p}^{1},\dots,\mathbf{p}^{N},\sum_{x=1}^{N}\alpha_{x}\mathbf{u}^{x}+\alpha_{0}\mathbf{1})$ will be proportional. (5) becomes:

We further study the social welfare maximization problem with the help of the MMZD in this circumstance. Take organization $1$ performing the MMZD strategy as an example. According to (6), by setting $\alpha_{x}=1(x\in\mathcal{N})$, the social welfare can be calculated as $\sum_{x=1}^{N}E^{x}=-\alpha_{0}$. Thus, the issue of maximizing the social welfare is equivalent to the following optimization problem:

We denote $u^{x}_{k},k\in\{1,2,...,(r+1)^{N+1}\}$ as the $k$th element in $\mathbf{u}^{x}$, then we can solve the above optimization problem by considering the following two cases:
1) $\phi>0$. To meet the constraint $p^{1}_{j,g}\geq 0$, we can get the lower bound of $\alpha_{0}$ as follows:

To meet the constraint $p^{1}_{j,g}\leq 1$, we can get the upper bound of $\alpha_{0}$ as follows:

Only if ${\alpha_{0}}_{min}\leq{\alpha_{0}}_{max}$, can $\alpha_{0}$ have a feasible solution, which is equivalent to $\max(\Lambda_{k})\leq\min(\Lambda_{l}),\forall{k}\in\{1,2,...,(r+1)^{N+1}\},\forall{l}\in\{(r+1)^{N}+1,...,2(r+1)^{N+1}\}$. If there exists $\phi>0$ satisfying the above constraint, we can obtain the minimum value of $\alpha_{0}$ as follow:

2) $\phi<0$. Similarly, when $p^{1}_{j,g}\geq 0$, we have ${\alpha_{0}}_{min}=\max(\Lambda_{l}),\forall{l}\in\{(r+1)^{N+1}+1,...,2(r+1)^{N+1}\}$; considering $p^{1}_{j,g}\leq 1$, we have ${\alpha_{0}}_{max}=\min(\Lambda_{k}),\forall{k}\in\{1,2,...,(r+1)^{N+1}\}$. In addition, $\alpha_{0}$ is feasible only when ${\alpha_{0}}_{min}\leq{\alpha_{0}}_{max}$, i.e., $\max(\Lambda_{l})\leq\min(\Lambda_{k}),\forall{k}\in\{1,2,...,(r+1)^{N+1}\},\forall{l}\in\{(r+1)^{N+1}+1,...,2(r+1)^{N+1}\}$. Finally, we can get the following result:

In summary, by (7) and (8), organization $1$ can unilaterally set the expected social welfare $\sum_{x=1}^{N}E^{x}$ with the MMZD strategy $\mathbf{p}^{1}$ meeting $\hat{\mathbf{p}}^{1}=\phi(\sum_{x=1}^{N}\mathbf{u}^{x}+\alpha_{0}\mathbf{1})$, with each element of $\mathbf{p}^{1}$ calculated by:

where $p^{1}_{h}$ denotes the $h$-th element in $\mathbf{p}^{1}$.

In this section, we evaluate the performance of the MMZD strategy to maximize the social welfare based on simulation experiments. All experiments are implemented using Matlab R2021a on a laptop with 2.3 GHz Intel Core i5-8300H processor. We set $N=10$ since the number of organizations in cross-silo FL is usually small. We consider that $K=200$, $r=33$. $\theta_{0}=23271.584$ and $\theta_{1}=50193.243$ are derived from the simulation dataset. In addition, we set $\phi=0.01$. For every control group with different strategy settings, we repeat the experiment for 100 times.

In order to verify the effectiveness of the MMZD strategy on maximizing the social welfare, we compare it with other five classical strategies, by simulating the entire cross-silo FL process for 20 rounds of game. The numerical results of maximum social welfare denote the sum of all organizations’ utilities. In Fig. 1, organization 1 adopts MMZD, all-defection (ALLD)[17], all-cooperation (ALLC)[17], and random (Rand) strategies[17]. Other organizations use ALLD, ALLC, Rand, Tit-For-Tat (TFT)[18], and mixed (Mixed) strategies[17]. Specifically, ALLD strategy is defined as: the organization does not perform local training at all, only submits a zero vector in global aggregation. While ALLC strategy means that the organization participates in all $r$ global aggregation with their local updates in every game round. Organizations which adopt Rand strategy randomly participate in global aggregation from $0$ to $r$ rounds with the probability of $\frac{1}{r+1}$. TFT strategy is defined as the organizations randomly choose the number of participating global aggregation from $0$ to $\frac{\lfloor r\rfloor}{2}$ when the sum of global aggregation rounds in last game round is less than $\frac{Nr}{2}$, otherwise they randomly choose the number of participating global aggregation from $\frac{\lfloor r+1\rfloor}{2}$ to $r$. We define the mixed strategy as adopting a specific strategy chosen from ALLC, ALLD, Rand, and TFT. By comparing Fig. 1(a) with the other three figures, we can find that the MMZD strategy can effectively control the social welfare. This can prove that free-riding behavior is avoided in some ways.

Fig. 2 plots the expected social welfare changes in each game round, as the organization 1 adopts the MMZD strategy and other organizations employ different strategies. Fig. 1(a) displays the final result in Fig. 2, which indicates that no matter what kind of strategies other organizations adopt, the social welfare finally converges to a fixed value, verifying the power of the proposed social welfare maximization game.

In this paper, we define the cross-silo FL game among organizations as a public goods game, revealing the social dilemma in cross-silo FL theoretically. In order to overcome the social dilemma, we propose a brand-new method using the MMZD to solve the social welfare maximization problem. By the means of the MMZD, an individual organization can unilaterally control social welfare at a certain level, regardless of other organizations’ strategies. Moreover, our approach can maintain the stability and sustainability of the system without extra cost. Simulation results prove that the MMZD strategy can efficiently and effectively control social welfare, which reduces the loss from selfish behaviors.