Mean-Field Game Analysis of SIR Model with Social Distancing

The focus of this work is on understanding the effect of social distancing behavior and the corresponding utility trade-offs on the epidemiological dynamics, so we will keep the epidemiology as simple as possible as a baseline model. We will devote a section to including an exposed class (SEIR), but leave further expansion on the epidemiology as future work.

We consider an SIR model without births or deaths. The recovered (R) is an absorbing state, where individuals gain permanent immunity from the disease. For now, we do not consider exposed period or asymptomatic or presymptomatic transmissions.

Without birth or death, the population is fixed, and we let $x_{z}$ be the fraction of the population in compartment $z\in\{S,I,R\}$. Then the dynamics of the SIR system is

$\beta$ is the likelihood that infection happens given a contact between S and I, and $\mu$ is the constant rate of recovery from infection. $C(\cdot)$ is the rate that an S individual and an I individual make contact, which is dependent on whether individuals choose to socially distance or not. Let us write the number of contacts made by a $z\in\{S,I,R\}$ individual at time $t$ as $c_{z}(t)$.

It is evident that $C(\cdot)$ should increase as $c_{S}(t)$ and $c_{I}(t)$ increase, but there are two intuitive ways of writing an expression for $C(\cdot)$, analogous to frequency-dependent or density-dependent disease transmission.

i) Frequency-dependent
Looking only at the transmission term (suppressing notation for dependence on $t$ for brevity),

where a contact by S has probability $\frac{c_{I}x_{I}}{c_{S}x_{S}+c_{I}x_{I}+c_{R}x_{R}}$ of being with I. Therefore, the transmission term depends on the relative frequency between the contact strategies.

ii) Density-dependent
Similarly the transmission term is

Here, the transmission term depends on not only the relative frequency, but also the absolute density of the contact strategies. If the whole population is doubling or halving contacts, the transmission should increase or decrease accordingly, and this density-dependent formulation is consistent with such intuitions. For the rest of this paper, we will take the density-dependent formulation. (Note that $\beta^{\prime}$ term here is technically different from the $\beta$ term above, but for simplicity, any $\beta$ referred to for the rest of this paper is $\beta^{\prime}$ of the density-dependent formulation.)

$c_{z}(t)$ is the contact strategy which is explicitly chosen by an individual. Large $c_{z}(t)$ means going out to work and socializing, while smaller $c_{z}(t)$ means refraining from those activities (i.e. social distancing). $u_{z}(c_{z})$ denotes the utility received by a $z$ individual from making $c_{z}$ contacts, which is a combination of economic gains and personal well-being. The realistic conditions that we impose are the following: Starting from zero contacts, increasing the number of contacts results in increased utility, which is the baseline universal need for social interactions, as well as monetary gain from economic activities. We assume that the marginal increase in utility decreases with more contacts, until eventually, more contacts become detrimental.

Therefore, the appropriate functional form of $u_{z}(c_{z})$ must be a concave function with an interior maximum. One appropriate function form with interpretations of the parameters is $u_{z}(c_{z})=(b_{z}c_{z}-c_{z}^{2})^{\gamma}-a_{z}$ where $c_{z}\in[0,b_{z}]$[8]. $b_{z}$ is a parameter of how the disease impacts the marginal economic productivity of the individual. Large $b_{z}$ means that the individual has the choice to make more contacts, as well as receive higher marginal utility from making contacts. $a_{z}$ is a parameter of the baseline cost of being infected, such as an individual’s general propensity to be healthy. These two parameters decompose the effect of the disease into economic cost and health cost. Some assumptions on these parameters are that $b_{S}=b_{R}>b_{I}$ and $a_{S}=a_{R}<a_{I}$, since the infected become impaired in economic productivity, as well as suffer the health cost compared to the S and R. $\gamma$ changes the concave shape of the function, and $\gamma\in(0,1]$ ensures that $u_{z}(c_{z})$ is concave everywhere in the domain.

From this utility function, we see that each individual has some optimal level of social contacts. The utility of a $z$ individual is maximized at $\frac{b_{z}}{2}$, which is each individual’s optimal contact strategy in the absence of adaptive behavior in response to the risk of infectious disease.

Over time period, $t\in[0,T]$, the total utility of an individual is the sum of utility gained at each time point. For example, a recovered individual with continuous contact decision $c(t)$ receives total utility $\int_{0}^{T}u_{R}(c(t))dt$. Typically, a discounting term, $\delta$ which discounts the future utility compared to the present utility, is included, but we set $\delta=1$ for simplicity. The general results here do not change with $\delta<1$.

Let us define $V_{R}(t)$ to be the total future utility expected by the R individual at time $t$ until $T$. From this formulation, it follows that terminal condition is $V_{R}(T)=0$, and

Similarly for S and I individuals, we define $V_{S}(t)$ and $V_{I}(t)$, which depend on the rate at which the individuals move between the SIR states. (Fig. 2)

From these transition rates, we can write the Bellman equations, which give the value functions for individuals in S, I, and R states, respectively, as:

with terminal conditions, $V_{S}(T)=V_{I}(T)=V_{R}(T)=0$.

The SIR dynamics and the Bellman equations are coupled by the contact strategies, $c_{S},c_{I},c_{R}$, and the population, $x_{S},x_{I},x_{R}$. The solution to this problem is the mean-field equilibrium (MFE), which is the fixed point $(c_{S}^{eq},c_{I}^{eq},c_{R}^{eq},x_{S},x_{I},x_{R})$ such that 1) the strategies $c_{S}^{eq}(t)$, $c_{I}^{eq}(t)$, $c_{R}^{eq}(t)$ are the optimal solutions in equations 38 given $x_{S},x_{I},$ and $x_{R}$ and 2) $x_{S}$, $x_{I}$, and $x_{R}$ are solutions to the system of ODEs in equation 35 given the optimal strategies.

Additionally, we can characterize the socially optimal solution. Similar to the second-best equilibrium, discussed by Lipsey and Lancaster[17][18], if the infected individuals do not pursue the most utility-maximizing decisions, there is another equilibrium, in which the population can attain higher utility on average. If a central planner chooses the contact strategies of the population to maximize the utility of the entire population, the resulting socially optimal solution solves the dynamic optimization problem,

We can also define modified version of the socially optimal problem, in which there is cost to move away from the MFE solution. Given the MFE solution, $(c_{S}^{eq},c_{I}^{eq},c_{R}^{eq})$, the modified problem is

The MFE solution is characterized through Proposition 1, 2, 3.

In this model, the recovered individuals gain total immunity, so they always choose optimal contact rates.

The I individuals, similar to the R individuals, do not change from the optimal contact rate. From the above formulation, we see that $V_{I}(t)$ is bounded by $V_{R}(t)$, and $V_{R}(t)-V_{I}(t)$ increases as $\mu$ decreases, since it implies that I individuals spend longer time as an infected before recovering.

Note that $c_{S}^{eq}$ is smaller for bigger values of $b_{I}\beta x_{I}(V_{S}(t)-V_{I}(t))$, which means that the susceptible individuals should decrease contact if 1) infected population gets large, 2) the disease spreads easily, 3) cost of being infected is large, or 4) if the disease minimally affects the ability of the infected.

We use discrete time steps $\Delta t$ to find the numerical solutions to the equations. With some initial $c^{0}=(c_{S}^{0},c_{I}^{0},c_{R}^{0})$, we compute $x^{0}=(x_{S}^{0},x_{I}^{0},x_{R}^{0})$ using the ODE forward equations. Then, $c^{1}$ can be computed via backward induction with the given $x^{0}$. We continue this until we find $c^{k}$ and $x^{k}$ such that $c^{k}=c^{k-1}$ and $x^{k}=x^{k-1}$, which is the MFE solution.

Figure 3 shows the MFE numerical solution with disease parameters $\beta=0.03$ and $\mu=0.1$, and utility parameters $b_{S}=b_{R}=10$, $b_{I}=6$, $a_{S}=a_{R}=0$, $a_{I}=4$, and $\gamma=0.25$. As we showed in Proposition 1 and 2, the recovered and infected individuals have no incentive to lower their contact rates, and so they continue with optimal level of contact rates. The susceptible individuals lower their contact rate to balance their immediate utilities and their expected cost of possibly getting infected. In result, the spread of infection is mitigated by the behavioral changes by the susceptible population, compared to the classical case in which no adaptive behavior is considered. The infection curve has lower peak due to the behavioral changes, but also a longer tail, because there are more remaining susceptible individuals to get the disease after the peak.

Figure 4 shows the cumulative epidemic curve of the MFE solution and the classical SIR for different parameter values of $\beta,\mu,b_{I},$ and $a_{I}$. For any set of parameter values, the MFE solution, because of the adaptive behaviors, results in smaller final size as well as more gradual spread of the epidemic.

If the disease is more infectious (large $\beta$), the MFE solution shows faster spread as well as larger final size, although mitigated compared to what the classical SIR model would predict (Figure 4a). If the disease has longer recovery time (small $\mu$), the classical SIR model predicts faster spread. However, the MFE solution predicts a different result. For example, when $\mu=0.02$, the classical case predicts that because the average infectious period is long, the number of infected population rises faster. However, the behavioral decision of individuals are considered in the MFE solution, and so a more gradual epidemic is predicted, since it is optimal to suppress contacts to avoid getting infected in the first place (Fig. 4b).

$b_{I}$ and $a_{I}$ are new parameters introduced to consider the effect of behavioral change. $b_{I}$ denotes the economic utility, depending on social activity, and $a_{I}$ denotes the health utility of the individual. If the disease does not affect the productivity of the infected individual (high $b_{I}$), the disease will spread as if its transmission rate $\beta$ is higher (Fig. 4c). In the case of COVID-19, the symptoms of many have been mild to moderate, which might have contributed to the perceived higher transmission rate.

The epidemic spread as predicted by the classical SIR model would not change with changes in $a_{I}$, but the MFE solution shows smaller epidemic and flatter growth rate for high $a_{I}$, as individuals choose to make less contacts to avoid the high cost of becoming infected (Fig. 4d).

The MFE solution, discussed above, considers the case in which individuals maximize their own utility. For the socially optimal solution, we pose a centralized control problem, in which we find $(c_{S}^{opt},c_{I}^{opt},c_{R}^{opt})$ which maximizes the average utility of the entire population. Therefore we solve

In order to solve the optimal control problem, we use Pontryagin’s maximum principle, which gives the necessary conditions for the optimal controls, given the evolving dynamics of the system.

This proposition implies that the selfish behavior of the infected is never optimal for the population, and so the appropriate policy in all cases is to decrease the contacts of the infected below the level that they want to selfishly make.

For the same set of parameters as in Figure 3, we can compute the socially optimal solution.

The optimal solution as shown in Figure 5b is to completely suppress the contacts of the I, so that no additional infection can take place, while the susceptible population resumes normal activities. In fact, this is most often the socially optimal strategy. An exception is if a large part of the population was already infected initially (Figure 5c,d). Here, 70 percent of the population is initially infected, and so the optimal contact strategy is to completely isolate the susceptible until the number of infected decreases to some level, at which, we go back to complete quarantining of the I and the resuming of normal activities by the S. This type of policy is the complete shutdown implemented to various degrees in the United States, in which it is better to completely insulate the susceptibles until the number of infected population decrease back to a manageable level.

A key piece missing in our characterization of the socially optimal solution is the various costs to controlling the contact rate of the population. The assumption in Section 3.2 that the central planner can freely choose contact rates is far from realistic. For example, in the optimal solutions of Figure 5b, complete suppression of contact rates of the infected must be done through quarantining or providing proper incentives to keep infected people from making social contacts, which all cost resources. Additionally, people do not generally like their choices to be decided by an authority, so this cost also includes the ”loss in freedom,” which the central planner of wants to minimize. Therefore, we add to the problem in 3.2, a cost for deviating away from the population’s MFE.

We add a new parameter, $k\geq 0$, which balances the competing objectives of maximizing total utility and minimizing the deviations from selfish strategies, similar to a regularization parameter in machine learning. When $k=0$, the solution to the objective function is the socially optimal solution, and when $k$ is large, the solution is the MFE solution. In the intermediate values of $k$, we can find the solution which balances the trade-off.

As we see in Figure 6, for different values of $k$, we compute the socially optimal contact rates of the population. Another interpretation is that given limited resources to control contact rates, we find how it should be distributed during $t\in[0,T]$. We see a common result among the range of values for $k$, which is that when susceptibles have low contact rates, it is less important to keep the contact rates of the infected as low. Instead, it is more optimal use of resources to make sure the contact rates of the infected after the peak of the epidemic is lowered because this is when the susceptibles start ramping up to normal activities, thus posing higher risk of a second epidemic. Another reason is that because the number of infected is smaller at this time and the imposed cost is fixed per capita, the same amount of resources is more efficiently used by controlling the smaller population of infected rather than the larger infected population during the peak.

Here we show the numerical solutions to the MFE problem when exposed class (E) is added to the model. First, we consider the case in which the exposed individuals are not yet infectious, but incubating the disease. Second, we consider the case in which the exposed individuals are presymptomatic, meaning that they are infectious before they show symptoms, although at a lower rate compared to the infectious individuals.

The incubation period is a random variable which has exponential distribution with parameter $\nu$ (The average incubation period is $\nu^{-1}$). Another variable which is introduced is $\beta_{E}$, the presymptomatic transmission rate of individuals in the exposed class. If there is no presymptomatic transmission, $\beta_{E}=0$. The SEIR model is

Because the susceptible and the exposed classes do not have visible symptoms, an individual cannot differentiate between being in the S or E class. Therefore, we define the asymptomatic class, which includes both the susceptible class and the exposed class. The value function of this class will be taken as the weighted average between the S and E classes,

where the corresponding value functions are given as