On the Probability of Immunity

Let $X$ and $Y$ denote an exposure and its outcome, respectively. Let $X$ and $Y$ be binary taking values in $\{x,x^{\prime}\}$ and $\{y,y^{\prime}\}$. Let $Y_{x}$ and $Y_{x^{\prime}}$ denote the counterfactual outcome when the exposure is set to level $X=x$ and $X=x^{\prime}$. Let $y_{x}$, $y^{\prime}_{x}$, $y_{x^{\prime}}$ and $y^{\prime}_{x^{\prime}}$ denote the events $Y_{x}=y$, $Y_{x}=y^{\prime}$, $Y_{x^{\prime}}=y$ and $Y_{x^{\prime}}=y^{\prime}$. For instance, let $X$ represent whether a patient gets treated or not for a deadly disease, and $Y$ represent whether she survives it or not. Individual patients can be classified into immune (they survive whether they are treated or not, i.e. $y_{x}\land y_{x^{\prime}}$), doomed (they die whether they are treated or not, i.e. $y^{\prime}_{x}\land y^{\prime}_{x^{\prime}}$), benefited (they survive if and only if treated, i.e. $y_{x}\land y^{\prime}_{x^{\prime}}$), and harmed (they die if and only if treated, i.e. $y^{\prime}_{x}\land y_{x^{\prime}}$).

In general, the average treatment effect (ATE) estimated from a randomized controlled trial (RCT) does not inform about the probability of benefit (or of any of the other response types, i.e. harm, immunity, and doom). However, it may do it under certain conditions. For instance,

	$\displaystyle ATE=p(y_{x})-p(y_{x^{\prime}})$	$\displaystyle=p(y_{x},y_{x^{\prime}})+p(y_{x},y^{\prime}_{x^{\prime}})-[p(y_{x},y_{x^{\prime}})+p(y^{\prime}_{x},y_{x^{\prime}})]$
		$\displaystyle=p(\text{benefit})-p(\text{harm})$		(1)

and thus $p(\text{benefit})=ATE$ if $p(\text{harm})=0$ (a.k.a. monotonicity). Necessary and sufficient conditions are derived by Mueller and Pearl [1] to determine from observational and experimental data if monotonicity holds. In this work, we derive similar conditions for non-immunity, i.e. $p(\text{immunity})=p(y_{x},y_{x^{\prime}})=0$. These are interesting because under non-monotonicity, they turn an RCT informative about the probabilities of benefit and harm. To see it, consider

$$ATE=p(y_{x})-p(y_{x^{\prime}})$$

where the terms on the right-hand side of the equation are estimated from an RCT. Moreover,

	$\displaystyle p(y_{x})$	$\displaystyle=p(y_{x},y_{x^{\prime}})+p(y_{x},y^{\prime}_{x^{\prime}})=p(\text{immunity})+p(\text{benefit})$		(2)
	$\displaystyle p(y_{x^{\prime}})$	$\displaystyle=p(y_{x},y_{x^{\prime}})+p(y^{\prime}_{x},y_{x^{\prime}})=p(\text{immunity})+p(\text{harm})$		(3)

and thus $p(\text{benefit})=p(y_{x})$ and $p(\text{harm})=p(y_{x^{\prime}})$ if $p(\text{immunity})=0$.

In some cases, non-immunity is assured. For instance, when evaluating the effect of advertising on the purchase of a new product. The control group not being exposed to the ad has no way of purchasing the product, i.e. $p(y_{x^{\prime}})=0$ and thus $p(y_{x},y_{x^{\prime}})=0$. In other cases, non-immunity cannot be assured. For instance, when evaluating the effect of a drug. An individual may carry a gene variant that makes her recover from the disease regardless of whether she takes the drug or not, i.e. $p(y_{x},y_{x^{\prime}})\geq 0$. However, it may still be bounded as $p(y_{x},y_{x^{\prime}})\leq\epsilon$ from expert knowledge. We show that our necessary and sufficient conditions for non-immunity can trivially be adapted to $\epsilon$-bounded immunity. Moreover, we show that the knowledge of $\epsilon$-bounded immunity may tighten the bounds of the probabilities of benefit and harm by Tian and Pearl [2]. We also introduce the concepts of indirect benefit and harm (i.e., through a mediator) and repeat our previous analysis for them. Finally, we propose a method for sensitivity analysis of immunity under unmeasured confounding. We illustrate our results with concrete examples.

Consider the bounds of $p(\text{benefit})$ derived by Tian and Pearl [2]:

$$\max\left\{\begin{array}[]{cc}0,\\ p(y_{x})-p(y_{x^{\prime}}),\\ p(y)-p(y_{x^{\prime}}),\\ p(y_{x})-p(y)\end{array}\right\}\leq p(\text{benefit})\leq\min\left\{\begin{array}[]{cc}p(y_{x}),\\ p(y^{\prime}_{x^{\prime}}),\\ p(x,y)+p(x^{\prime},y^{\prime}),\\ p(y_{x})-p(y_{x^{\prime}})+\\ p(x,y^{\prime})+p(x^{\prime},y)\end{array}\right\}.$$

(4)

Then, combining Equations 2 or 3 with 4 gives

$$\max\left\{\begin{array}[]{cc}0,\\ p(y_{x})-p(y^{\prime}_{x^{\prime}}),\\ p(y_{x})-p(x,y)-\\ p(x^{\prime},y^{\prime}),\\ p(y_{x^{\prime}})-p(x,y^{\prime})-\\ p(x^{\prime},y)\end{array}\right\}\leq p(\text{immunity})\leq\min\left\{\begin{array}[]{cc}p(y_{x}),\\ p(y_{x^{\prime}}),\\ p(y_{x})-p(y)+\\ p(y_{x^{\prime}}),\\ p(y)\end{array}\right\}.$$

(5)

A sufficient condition for $p(\text{immunity})=0$ to hold is that some argument to the min function in Equation 5 is equal to 0, that is

$$p(y_{x})=0\text{ or }p(y_{x^{\prime}})=0\text{ or }p(y_{x})+p(y_{x^{\prime}})=p(y)\text{ or }p(y)=0.$$

(6)

Likewise, a necessary condition for $p(\text{immunity})=0$ to hold is that all the arguments to the max function are non-positive, that is

		$\displaystyle p(y_{x})+p(y_{x^{\prime}})\leq 1\text{ and }$
		$\displaystyle p(y_{x})\leq p(x,y)+p(x^{\prime},y^{\prime})\text{ and }$
		$\displaystyle p(y_{x^{\prime}})\leq p(x,y^{\prime})+p(x^{\prime},y).$		(7)

In the previous sections, the causal graph of the domain under study was unknown. In this section, we assume that the graph is available (e.g., from expert knowledge) and discuss two advantages that follow with it. Specifically, suppose that the domain under study corresponds to the following causal graph:

and thus $p(y_{x})=p(y|x)$ and $p(y_{x^{\prime}})=p(y|x^{\prime})$. Then, $p(y_{x})$ and $p(y_{x^{\prime}})$ can be estimated from observational data and thus, unlike in the previous sections, no RCT is required. A further advantage is that we can now compute the probabilities of benefit and harm mediated by $Z$. We elaborate on this below.

The effect of $X$ on $Y$ mediated by $Z$ (a.k.a. indirect effect) corresponds to the effect due to the indirect path $X\rightarrow Z\rightarrow Y$, i.e. after deactivating the direct path $X\rightarrow Y$. Different ways of deactivating the direct path have resulted in different indirect effect measures in the literature. Pearl [3] proposes deactivating the direct path by setting $X$ to non-exposure and comparing the expected outcome when $Z$ takes the value it would under exposure and non-exposure:

$$NIE=E[Y_{x^{\prime},Z_{x}}]-E[Y_{x^{\prime}}]$$

which is known as the average natural (or pure) indirect effect. Geneletti [4] also proposes deactivating the direct path by setting $X$ to non-exposure but instead, she proposes comparing the expected outcome when $Z$ is drawn from the distributions $\mathcal{Z}_{x}$ and $\mathcal{Z}_{x^{\prime}}$ of $Z_{x}$ and $Z_{x^{\prime}}$:

$$IIE=E[Y_{x^{\prime},\mathcal{Z}_{x}}]-E[Y_{x^{\prime},\mathcal{Z}_{x^{\prime}}}]$$

which is known as the interventional indirect effect. Although $NIE$ and $IIE$ do not coincide in general, they coincide for the causal graph above [5]. Finally, Fulcher et al. [6] proposes deactivating the direct path by setting $X$ to its natural (observed) value and comparing the expected outcome when $Z$ takes its natural value and the value it would under no exposure:

$$PIIE=E[Y_{X,Z_{X}}]-E[Y_{X,Z_{x^{\prime}}}]$$

which is also known as the population intervention indirect effect. This measure is suitable when the exposure is harmful (e.g., smoking), and thus one may be more interested in elucidating the effect (e.g., disease prevalence) of eliminating the exposure rather than in contrasting the effects of exposure and non-exposure.

We propose an alternative way of deactivating the direct path $X\rightarrow Y$ and measuring the indirect effect of $X$ on $Y$ through $Z$. Specifically, we assume that the direct path $X\rightarrow Y$ is actually mediated by an unmeasured random variable $U$ that is left unmodelled. This arguably holds in most domains. The identity of $U$ is irrelevant. Let $G$ denote the causal graph below, i.e. the original causal graph refined with the addition of $U$.

Now, deactivating the direct path $X\rightarrow Y$ in the original causal graph can be achieved by adjusting for $U$ in $G$, i.e. $\sum_{u}E[Y|x,u]p(u)$. Unfortunately, $U$ is unmeasured. Instead, we propose the following way of deactivating $X\rightarrow Y$. Let $H$ denote the causal graph below, i.e. the result of reversing the edge $X\rightarrow U$ in $G$.

The average total effect of $X$ on $Y$ in $H$ can be computed by the front-door criterion [7]:

	$\displaystyle TE$	$\displaystyle=E[Y_{x}]-E[Y_{x^{\prime}}]$
		$\displaystyle=\sum_{z}p(z|x)\sum_{\dot{x}}E[Y|\dot{x},z]p(\dot{x})-\sum_{z}p(z|x^{\prime})\sum_{\dot{x}}E[Y|\dot{x},z]p(\dot{x}).$

Note that $G$ and $H$ are distribution equivalent, i.e. every probability distribution that is representable by $G$ is representable by $H$ and vice versa [7]. Then, evaluating the second line of the equation above in $G$ or $H$ gives the same result. If we evaluate it in $H$, then it corresponds to the part of association between $X$ and $Y$ that is attributable to the path $X\rightarrow Z\rightarrow Y$. If we evaluate it in $G$, then it corresponds to the part of $TE$ in $G$ that is attributable to the path $X\rightarrow Z\rightarrow Y$, because $TE$ in $G$ equals the association between $X$ and $Y$, since $G$ has only directed paths from $X$ to $Y$. Therefore, the second line in the equation above corresponds to the part of $TE$ in the original causal graph that is attributable to the path $X\rightarrow Z\rightarrow Y$, thereby deactivating the direct path $X\rightarrow Y$. We propose to use the second line in the equation above as a measure of the indirect effect of $X$ on $Y$ in the original causal graph.

The reasoning above can be extended to the probabilities of benefit and harm, and thereby measure the benefit and harm mediated by $Z$. As mentioned above, the causal graphs $G$ and $H$ represent different data generation mechanisms but the same probability distribution over $X$, $Y$ and $Z$. Therefore, the mechanisms agree on observational probabilities but may disagree on counterfactual probabilities. We use $p()$ to denote observational probabilities obtained from either mechanism, and $q()$ to denote counterfactual probabilities obtained from the mechanism corresponding to $H$. The probabilities of benefit and harm of $X$ on $Y$ mediated by $Z$ in $G$ and thus in the original causal graph (henceforth indirect benefit and harm, or $IB$ and $IH$) can be computed by applying Equation 2 to $H$. That is,

$$IB=q(\text{benefit})=q(y_{x})=\sum_{z}p(z|x)\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})$$

where the second equality holds if $q(\text{immunity})=0$, and the third is due to the front-door criterion on $H$. Likewise for $IH$ simply replacing $x$ by $x^{\prime}$. Applying Equation 5 to $H$ yields necessary and sufficient conditions for $q(\text{immunity})=0$. That is,

		$\displaystyle\sum_{z}p(z|x)\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})=0\text{ or }$
		$\displaystyle\sum_{z}p(z|x^{\prime})\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})=0\text{ or }$
		$\displaystyle\sum_{z}[p(z|x)+p(z|x^{\prime})]\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})=p(y)\text{ or }$
		$\displaystyle p(y)=0$		(10)

is a sufficient condition, whereas

		$\displaystyle\sum_{z}[p(z|x)+p(z|x^{\prime})]\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})\leq 1\text{ and }$
		$\displaystyle\sum_{z}p(z|x)\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})\leq p(x,y)+p(x^{\prime},y^{\prime})\text{ and }$
		$\displaystyle\sum_{z}p(z|x^{\prime})\sum_{\dot{x}}p(y|\dot{x},z)p(\dot{x})\leq p(x,y^{\prime})+p(x^{\prime},y)$		(11)

is a necessary condition. Necessary and sufficient conditions for $\epsilon$-bounded immunity on $H$ (i.e., $q(\text{immunity})\leq\epsilon$) can be obtained much like in Section 2.1. That is, it suffices to add $\epsilon$ to the right-hand sides of the conditions above and replace $=$ with $\leq$. Finally, we can adapt accordingly the equations in Section 2.2 to obtain $\epsilon$-bounds on $IB$ and $IH$. Note that the analysis of indirect benefit and harm presented here does not require an RCT, i.e. all the expressions involved can be estimated from just observational data.

In this section, like in the previous section, we assume that the causal graph of the domain under study is available, e.g. from expert knowledge. We also assume that we only have access to observational data, i.e. no RCT is available. Specifically, consider the following causal graph:

which includes potential unmeasured exposure-outcome confounding. Since $p(y_{x})=\sum_{z}p(z|x)\sum_{\dot{x}}E[Y|\dot{x},z]p(\dot{x})$ by the front-door criterion, we can proceed as in the previous section to derive necessary and sufficient conditions for non-immunity. Suppose now that $Z$ is unmeasured or that the effect of $X$ on $Y$ is direct rather than mediated by $Z$. Then, $p(y_{x})$ is unidentifiable from observational data [7], and thus we cannot proceed as in the previous section. We therefore take an alternative approach to inform the analyst about the probability of immunity and thereby help her in decision making. In particular, we propose a sensitivity analysis method to bound the probability of immunity as a function of the observed data distribution and some intuitive sensitivity parameters. Our method is an straightforward adaption of the method by Peña [9], originally developed to bound the probabilities of benefit and harm.

Let $U$ denote the unmeasured exposure-outcome confounders. For simplicity, we assume that all these confounders are categorical, but our results also hold for ordinal and continuous confounders.²²2If $U$ is continuous then sums/maxima/minimima over $u$ should be replaced by integrals/suprema/infima. For simplicity, we treat $U$ as a categorical random variable whose levels are the Cartesian product of the levels of the elements in the original $U$.

Note that

$$p(y_{x})=p(y_{x}|x)p(x)+p(y_{x}|x^{\prime})p(x^{\prime})=p(y|x)p(x)+p(y_{x}|x^{\prime})p(x^{\prime})$$

where the second equality follows from counterfactual consistency, i.e. $X=x\Rightarrow Y_{x}=Y$. Moreover,

$$p(y_{x}|x^{\prime})=\sum_{u}p(y_{x}|x^{\prime},u)p(u|x^{\prime})=\sum_{u}p(y|x,u)p(u|x^{\prime})\leq\max_{u}p(y|x,u)$$

where the second equality follows from $Y_{x}\!\perp\!X|U$ for all $x$, and counterfactual consistency. Likewise,

$$p(y_{x}|x^{\prime})\geq\min_{u}p(y|x,u).$$

Now, let us define

$$M_{x}=\max_{u}p(y|x,u)$$

and

$$m_{x}=\min_{u}p(y|x,u)$$

and likewise $M_{x^{\prime}}$ and $m_{x^{\prime}}$. Then,

$$p(x,y)+p(x^{\prime})m_{x}\leq p(y_{x})\leq p(x,y)+p(x^{\prime})M_{x}$$

and likewise

$$p(x^{\prime},y)+p(x)m_{x^{\prime}}\leq p(y_{x^{\prime}})\leq p(x^{\prime},y)+p(x)M_{x^{\prime}}.$$

These equations together with Equation 5 give

$$\max\left\{\begin{array}[]{cc}0,\\ p(x^{\prime})m_{x}+p(x)m_{x^{\prime}}-p(y^{\prime}),\\ p(x^{\prime})m_{x}-p(x^{\prime},y^{\prime}),\\ p(x)m_{x^{\prime}}-p(x,y^{\prime})\end{array}\right\}\leq p(\text{immunity})\leq\min\left\{\begin{array}[]{cc}p(x,y)+p(x^{\prime})M_{x},\\ p(x^{\prime},y)+p(x)M_{x^{\prime}},\\ p(x^{\prime})M_{x}+p(x)M_{x^{\prime}},\\ p(y)\end{array}\right\}$$

(12)

where $m_{x}$, $M_{x}$, $m_{x^{\prime}}$ and $M_{x^{\prime}}$ are sensitivity parameters. The possible regions for $m_{x}$ and $M_{x}$ are

$$0\leq m_{x}\leq p(y|x)\leq M_{x}\leq 1$$

(13)

and likewise for $m_{x^{\prime}}$ and $M_{x^{\prime}}$.

Our lower bound in Equation 12 is informative if and only if³³3Note that the second row in the maximum equals the third plus the fourth rows.

$$0<p(x^{\prime})m_{x}-p(x^{\prime},y^{\prime})$$

or

$$0<p(x)m_{x^{\prime}}-p(x,y^{\prime}).$$

Then, the informative regions for $m_{x}$ and $m_{x^{\prime}}$ are

$$p(y^{\prime}|x^{\prime})<m_{x}\leq p(y|x)$$

and

$$p(y^{\prime}|x)\leq m_{x^{\prime}}<p(y|x^{\prime}).$$

On the other hand, our upper bound in Equation 12 is informative⁴⁴4Note that we already know that $p(\text{immunity})\leq p(y)$ by Equation 5. if and only if⁵⁵5Note that the third row in the minimum equals the first plus the second minus the fourth rows.

$$p(x,y)+p(x^{\prime})M_{x}<p(y)$$

or

$$p(x^{\prime},y)+p(x)M_{x^{\prime}}<p(y)$$

which occurs if and only if $p(y|x)<p(y|x^{\prime})$ or $p(y|x^{\prime})<p(y|x)$.⁶⁶6To see it, rewrite $p(y)=p(x,y)+p(x^{\prime},y)$ and recall Equation 13. Therefore, our upper bound is always informative, and thus the informative regions for $M_{x}$ and $M_{x^{\prime}}$ coincide with their possible regions.

The analysis in this work can be repeated for $p(\text{doom})$ instead of $p(\text{immunity})$ by simply swapping $y$ and $y^{\prime}$, and $p(\text{benefit})$ and $p(\text{harm})$. Additionally, the analysis of indirect benefit can be repeated for $p(\text{harm})$ instead of $p(\text{immunity})$ due to Equation 1, and thereby extend the analysis by Mueller and Pearl [1].