A Mean-Field Team Approach to Minimize the Spread of Infection in a Network

Networks are ubiquitous in today’s world, connecting people and organizations in various ways to improve the quality of day-to-day life in terms of, for example, health services [1], consumers demand [2], energy management [3], and social activities [4], to name only a few. There has been a growing interest in the literature recently on network analysis, and in particular, on enhancing network reliability and security [5, 6]. This problem has been on the spotlight ever since the dramatic influence of social media on public opinion was observed in a number of major events.

Controlling the spread of undesirable phenomena such as disease and misinformation over a network is an important problem for which different approaches are proposed in the literature [7, 8]. The dynamics of an infection propagating in a network of $n$ nodes, where each node has a binary state (susceptible and infected), can be modeled by a Markov chain with a $2^{n}\times 2^{n}$ transition probability matrix. Since the computational complexity of such a model is exponential with respect to $n$, mean-field theory has proved to be effective in approximating a large-scale dynamic network by an infinite population one. For this purpose, the dynamics of the probability distribution of infected nodes can be described by a differential equation (called diffusion equation) [9, 10].

In the analysis of infection spread, the main objective is to study the dynamics of the states of nodes, specially after a sufficiently long time, in order to determine the rate of convergence to the steady state [11, 12, 13]. It is shown in [11] that if the rate of spread of infection over the rate of cure is less than the inverse of the largest eigenvalue of the adjacency matrix, the infinite-population network reaches to an absorbing state in which all states are healthy, i.e., the infection is eventually cleared.

In the control of infection spread, on the other hand, the objective is to derive the transition probabilities such that a prescribed performance index, which is a function of implementation cost and the number of infected nodes, is minimized [14, 15, 16, 17, 18]. This problem is computationally difficult to solve, in general. However, in the special case when the diffusion equation and cost function have certain structures, the optimal strategy can be obtained analytically. For example, in [14, 15, 16] it is assumed that the network dynamics and cost are linear in control action (that is immunization or curing rates), which leads to bang-bang control strategy. The interested reader is referred to [17, 18] for more details on optimal resource allocation methods.

This paper studies the optimal control of a network consisting of an arbitrary number of nodes that are influenced (coupled) by the empirical distribution of infected nodes (such couplings are not necessarily linear). The infectious process is assumed to be persistent, in the sense that the infection does not disappear after the initial time. In contrast to the papers cited in the previous paragraph which consider a continuous action set, in this paper it is assumed that there is a limited number of resources available, which means that the action set is finite. In addition, we raise a practical question that when the solution of an infinite-population network constructs a meaningful approximation for the finite-population one. Inspired by existing techniques for mean-field teams [19, 20, 21, 22, 23, 24], we first compute the optimal solution of a finite-population network for the case where the empirical distribution of infected nodes is observable. Next, we derive an infinite-population Bellman equation that requires no observation of infected nodes, and identify a stability condition under which the solution of the infinite-population network constitutes a near-optimal solution for the finite-population one.

The paper is structured as follows. In Section II the problem is formulated and the objectives are subsequently described. The optimal control strategies, as the main results of the paper, are derived on micro and macro scales in Section III. An illustrative example of a social network is presented in Section IV. The results are finally summarized in Section V.

Prior to solving Problems 1 and 2, it is necessary to understand the dynamics of the empirical distribution of the infected users, and more importantly, the way it evolves over time according to each option of the network manager and state of the infectious process. To this end, the following theorem is needed.

According to Theorem 2, the optimal strategy does not depend on the history of infected users and infectious process, i.e., it is sufficient to know the current values in order to optimally control the network.

In the special case of $n=\infty$, the probability mass function of $m_{t+1}$ becomes a Dirac measure. In such a case, there is no loss of optimality in restricting attention to the dynamics of the controlled differential equations. More precisely, define

$$p_{t}:=\lim_{n\rightarrow\infty}\frac{1}{n}\sum_{n=1}^{\infty}\mathbb{1}\left(x^{i}_{t}=I\right).$$

(13)

According to [20, Lemma 4], the following equality holds with probability one for any trajectory $z_{1:\infty}$,

$$p_{t+1}=(1-p_{t})f^{0}(u_{t},p_{t},z_{t})+p_{t}(1-f^{1}(u_{t},p_{t},z_{t})),$$

(14)

with $p_{1}=\mathbb{E}\big{[}m_{1}\big{]}$. By incorporating the macro-scale (infinite-population) dynamics (14) into the Bellman equation (11), one arrives at the following Bellman equation:

$$\tilde{V}(p,z)=\min_{u\in\mathcal{U}}\Big{(}c(u,p,z)+\beta\sum_{\tilde{z}\in\mathcal{Z}}\mathbb{P}\big{(}\tilde{z}\mid z,u\big{)}\\ \times\tilde{V}\big{(}(1-p)f^{0}(u,p,z)+p(1-f^{1}(u,p,z)),\tilde{z}\big{)}\Big{)},$$

(15)

for any $p\in[0,1]$ and $z\in\mathcal{Z}$. Let $\tilde{g}:[0,1]\times\mathcal{Z}\rightarrow\mathcal{U}$ be a minimizer of the right-hand side of equation (15), and define the following action at time $t$:

$$\tilde{u}_{t}:=\tilde{g}(p_{t},z_{t}).$$

(16)

Notice that $p_{1:t+1}$ is a stochastic process adapted to the filtration $z_{1:t}$ for any $t\in\mathbb{N}$, i.e.,

$$p_{t+1}=(1-p_{t})f^{0}(\tilde{g}(p_{t},z_{t}),p_{t},z_{t})+p_{t}(1-f^{1}(\tilde{g}(p_{t},z_{t}),p_{t},z_{t})).$$

(17)

To establish the convergence result, the following assumptions are imposed on the model.

Since the optimization in (15) is over an infinite space $[0,1]$, it is computationally difficult to find the exact solution. However, it is shown in [20, Corollary 1] that if the optimization problem is carried out over space $\mathcal{M}_{n}$, the resultant solution will be a near-optimal solution for the finite-population case under Assumptions 1, 2 and 3.

Nowadays, many people get their daily news via social media, where a small piece of false information may propagate and lead to a widespread misinformation and potentially catastrophic consequences. As a result, it is crucial for network managers as well as governments to prevent large-scale misinformation on social media. Inspired by this objective, we present a simple rumor control problem, where the goal of a network manager is to minimize the number of misinformed users in the presence of a false rumor.

Example 1: Consider $n\in\mathbb{N}$ users and a matter of public interest with uncertain outcome such as an election. Let $x^{i}_{t}=S$ mean that user $i\in\mathbb{N}_{n}$ at time $t\in\mathbb{N}$ is correctly informed about the topic and $x^{i}_{t}=I$ mean otherwise. Denote by $m_{t}=\frac{1}{n}\sum_{t=1}^{n}\mathbb{1}\left(x^{i}_{t}=I\right)$ the empirical distribution of the misinformed users at time $t$.

Let $z_{t}\in\{0,1,2,3,4\}$ be the number of fake news that the source of the rumor publishes on social media at time $t$. The source is assumed to be persistent, i.e., it will find a way to spread the rumor unless it is constantly blocked.

The network manager has three options at each time instant, i.e. $u_{t}\in\{1,2,3\}$, where:

• •

  $u_{t}=1$ means that the network manager does not intervene;

• •

  $u_{t}=2$ means that the network manager blocks the source of the rumor, and

• •

  $u_{t}=3$ means that the network manager broadcasts authenticated information to the users and addresses the issue publicly for transparency.

Let $w^{z}_{t}\in\{0,1\}$ be a random one-unit increment with success probability $0.3$ such that at time $t\in\mathbb{N}$,

$$z_{t+1}=\begin{cases}z_{t}+w^{z}_{t},&z_{t}<4,u_{t}=1,3,\\ 4,&z_{t}=4,u_{t}=1,3,\\ 0,&u_{t}=2.\end{cases}$$

(24)

The number of fake news may be viewed as the severity level of the misinforation induced by the rumor. In this example, we have implicitly assumed that when option “block” is taken by the manager, the source of rumor starts producing new fake news cautiously from zero again. The initial states of users are identically and independently distributed with probability mass function $(0.85,0.15)$, where $0.15$ is the probability of being initially misinformed. At any time $t\in\mathbb{N}$, given the empirical distribution of misinformed users $m_{t}$, the number of fake news $z_{t}$ and the option $u_{t}$ taken by the network manager, an informed user is misled by the rumor with the following probability:

$$f^{0}(u_{t},m_{t},z_{t})=\begin{cases}0.2m_{t}(z_{t}+1),&u_{t}=1,\\ 0.2m_{t},&u_{t}=2,\\ 0.1m_{t}^{2},&u_{t}=3,\end{cases}$$

(25)

where a larger number of misinformed users and fake news means a higher probability that a user becomes misinformed. On the other hand, a misinformed user becomes informed and convinced by the authenticated information provided by the network manager with a high probability. More precisely,

$$f^{1}(u_{t})=\begin{cases}0,&u_{t}=1,2,\\ 0.8,&u_{t}=3.\end{cases}$$

(26)

Denote by $\ell:\mathcal{U}\times\mathcal{Z}\rightarrow\mathbb{R}_{\geq 0}$ the implementation cost of each option, and let:

$$\ell(1,z)=0,\quad\ell(2,z)=0.2z+1,\quad\ell(3,z)=5,$$

(27)

for any $z\in\{0,1,2,3,4\}$. It is desired to minimize the following cost function:

$$\mathbb{E}\big{[}\sum_{t=1}^{\infty}0.9^{t-1}\left(3.8m_{t}+\ell(u_{t},z_{t})\right)\big{]}.$$

(28)

To determine the optimal strategy, we first compute the transition probability matrix in Theorem 1 and then solve the Bellman equation (11) in Theorem 2 by using the value-iteration method. The optimal strategy for $n=200$ is displayed in Figure 1 as a function of the empirical distribution of misinformed users and the number of fake news. Under this optimal strategy, one realization of Example 1 is depicted in Figure 2.

To verify the results of Theorem 3 and Corollary 1, let the Bellman equation (15) be quantized with a step size $1/n$, and denote the resultant value function by $V^{Q}(q_{1},z_{1})$, where $q_{1}$ is the closest number in $\mathcal{M}_{n}$ to $p_{1}=0.15$. Subsequently, it is shown in Figure 3 that $\mathbb{E}\big{[}V^{Q}(q_{1},z_{1})\big{]}$ converges to $J^{\ast}_{n}=\mathbb{E}\big{[}V(m_{1},z_{1})\big{]}$, as $n$ increases.

A stochastic dynamic control strategy was introduced over a homogeneous network to minimize the spread of a persistent infection. It was shown that the exact optimal solution can be efficiently computed by solving a Bellman equation whose state space increases linearly with the number of nodes. In addition, an approximate optimal solution was proposed based on the infinite-population network, where the approximation error was shown to be upper bounded by a term that decays to zero as the number of users tends to infinity. An example of a social network was then presented to verify the theoretical results.

As a future research direction, the obtained results can be extended to partially homogeneous networks wherein the nodes are categorized into several sub-populations of homogeneous nodes such as low-degree and high-degree nodes. In addition, various approximation methods may be used to further alleviate the computational complexity of the proposed solutions. The development of reinforcement learning algorithms based on the Bellman equations provided in this paper can be another interesting problem for future work.