Posterior risk of modular and semi-modular Bayesian inference

We observe a sequence of data $z_{1:n}=(z_{1},\ldots,z_{n})$, $z_{i}\in\mathcal{Z}$ for each $i$. The observations $z_{1:n}$ are considered an observation of a random vector $Z$, and wish to conduct Bayesian inference on the unknown parameters $\theta$ in the assumed joint likelihood $f(z_{1:n}\mid\theta)$, where $\theta=(\theta_{1}^{\top},\theta_{2}^{\top})^{\top}$, $\theta\in\Theta_{1}\times\Theta_{2}\subseteq\mathbb{R}^{d}$, and $\Theta_{1}\subseteq\mathbb{R}^{d_{1}}$, $\Theta_{2}\subseteq\mathbb{R}^{d_{2}}$. Our prior beliefs over $\Theta$ are expressed via a prior density $\pi(\theta)=\pi(\theta_{1})\pi(\theta_{2}\mid\theta_{1})$.

In modular Bayesian inference, the joint likelihood $f(z_{1:n}\mid\theta)$ can often be expressed as a product, with terms for data sources from different modules. The simplest case is the two module system described in Plummer (2015), where the random variables are $Z=(X,Y)$. The first module consists of a likelihood term depending on $X$ and $\theta_{1}$, given by $f_{1}(\cdot\mid\theta_{1})$, and the prior $\pi(\theta_{1})$. The second module consists of a likelihood term depending on $Y$ and $\theta_{1},\theta_{2}$, given by $f_{2}(\cdot\mid\theta_{1},\theta_{2})$, and the conditional prior $\pi(\theta_{2}\mid\theta_{1})$. An example of such a model is given below. A model with this structure leads to the following full posterior

$\displaystyle\pi_{\mathrm{full}}(\theta_{1},\theta_{2}\mid z_{1:n})$

$\displaystyle=\pi(\theta_{1}\mid x_{1:n},y_{1:n})\pi(\theta_{2}\mid y_{1:n},\theta_{1}),$

(1)

in which the conditional posterior for $\theta_{2}$ given $\theta_{1}$ does not depend on the observations for the random variable $X$, denoted as $x_{1:n}=(x_{1},\dots,x_{n})$. The parameter $\theta_{1}$ is shared between the two modules, while $\theta_{2}$ is specific to the second module.

While the full posterior in (1) delivers reliable inference when the model is correctly specified, in the presence of model misspecification Bayesian inference can sometimes be unreliable and not “fit for purpose”; see, e.g., Grünwald and Van Ommen (2017) for examples in the case of linear models, and Kleijn and van der Vaart (2012) for a general discussion. Following the literature on cutting feedback methods, we restrict our attention to settings where misspecification is confined to the second module, while specification of the first module is not impacted. Because the parameter $\theta_{1}$ is shared between modules, inference about this parameter can be corrupted by misspecification of the second module. This can also impact the interpretation of inference about $\theta_{2}$, which can be of interest even if the second module is misspecified provided this is done conditionally on values of $\theta_{1}$ consistent with the interpretation of this parameter in the first module.

Rather than use the posterior (1), in possibly misspecified models it has been argued that we can cut the link between the modules to produce more reliable inferences for $\theta_{1}$. This is the idea of cutting feedback; see Figure 1 for a graphical depiction of this “cutting” mechanism where, for simplicity, we assume $\pi(\theta_{2}\mid\theta_{1})$ does not depend on $\theta_{1}$. In the case of the two module system, the cut (indicated by the vertical dotted line in Figure 1) severs the feedback between the modules and allows us to carry out inferences for $\theta_{1}$ based on module one, using the likelihood $f_{1}(\cdot\mid\theta_{1})$, and then inference for $\theta_{2}$ can be carried out conditional on $\theta_{1}$. In the case of Bayesian inference, this philosophy has led researchers to conduct inference using the cut posterior distribution (see, Plummer, 2015, Jacob et al., 2017).

As shown in Carmona and Nicholls (2020) and Nicholls et al. (2022), the cut posterior is a “generalized” posterior distribution (see, e.g., Bissiri et al., 2016) that restricts the information flow to guard against model misspecification (Frazier and Nott, 2024). In the canonical two module system, the cut posterior takes the form

$$\pi_{\mathrm{cut}}(\theta\mid z_{1:n}):=\pi_{\mathrm{cut}}(\theta_{1}\mid x_{1:n})\pi(\theta_{2}\mid y_{1:n},\theta_{1}),\quad\pi_{\mathrm{cut}}(\theta_{1}\mid x_{1:n})\propto f_{1}(x_{1:n}\mid\theta_{1})\pi(\theta_{1}).$$

The common argument given for the use of $\pi_{\mathrm{cut}}(\theta\mid z_{1:n})$ instead of $\pi(\theta\mid z_{1:n})$ is the assumption that misspecification adversely impacts inferences for $\theta_{1}$; e.g., Liu et al. (2009), and Jacob et al. (2017). The cut posterior uses only information from the data $x_{1:n}$ in making inference about $\theta_{1}$, ensuring inference is insensitive to misspecification of the model for $y_{1:n}$. However, uncertainty about $\theta_{1}$ can still be propagated through for inference on $\theta_{2}$ via the conditional posterior $\pi(\theta_{2}\mid\theta_{1},y_{1:n})$.

In the remainder we consider a generalization of the canonical two-module system by assuming that the joint likelihood takes the form

$$f(z_{1:n}\mid\theta)=f_{1}(z_{1:n}\mid\theta_{1})f_{2}(z_{1:n}\mid\theta_{1},\theta_{2}),$$

where the individual models may or may not depend on the entire dataset $z_{1:n}$. In the expressions above, $f_{1}(z_{1:n}\mid\theta_{1})$ and $f_{2}(z_{1:n}\mid\theta_{1},\theta_{2})$ need not be densities as functions in $z_{1:n}$ so long as $f(z_{1:n}\mid\theta)$ itself is a density; the terms $f_{1}$ and $f_{2}$ describe a decomposition of the likelihood having the dependence indicated by the arguments. The goal of modular Bayesian inference is to conduct inference on $\theta=(\theta_{1}^{\top},\theta_{2}^{\top})^{\top}$ in the specific setting where inference for $\theta_{1}$ based only on $f_{1}(z_{1:n}\mid\theta_{1})$ is reasonable, but where the module $f_{2}(z_{1:n}\mid\theta_{1},\theta_{2})$ contains additional useful information on $\theta_{1}$; see the regularity conditions in Appendix C.1 for specific details. Modular inference on $\theta$ can be carried out using the cut posterior

$$\pi_{\mathrm{cut}}(\theta\mid z_{1:n}):=\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})\pi(\theta_{2}\mid z_{1:n},\theta_{1}),\;\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})\propto{f_{1}(z_{1:n}\mid\theta_{1})\pi(\theta_{1})}.$$

(2)

The cut posterior is beneficial when misspecification of $f_{2}(\cdot\mid\theta_{1},\theta_{2})$ makes full Bayesian inference on $\theta_{1}$ less accurate compared to Bayesian inference using only $f_{1}(\cdot\mid\theta_{1})$. The benefits of cutting feedback can be formalized by assuming that misspecification is limited to $f_{2}(\cdot\mid\theta_{1},\theta_{2})$ and that the true data generating process (DGP) has a density of the form¹¹1Strictly speaking, this analysis extends beyond density functions but we maintain the use of densities to simplify our discussion.

$$h(z\mid\theta_{1,0},\delta_{0})=f_{1}(z\mid\theta_{1,0})\delta_{0}(z),$$

for some unknown component $\delta_{0}(z)$ that controls the amount of model misspecification. The form of $h(z\mid\theta_{1,0},\delta_{0})$ allows us to capture both gross model misspecification, and situations where misspecification is ambiguous. We first restrict our attention to gross model misspecification by imposing the following assumption.

Under Assumption 1 and regularity conditions, the cut posterior $\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})$ can be shown to deliver accurate inferences for $\theta_{1,0}$, while the full posterior $\pi_{\mathrm{full}}(\theta_{1}\mid z_{1:n})$ does not. To state this result, denote the cut posterior distribution for $\theta_{1}$ as $\Pi_{\mathrm{cut}}(\theta_{1}\in\cdot\mid z_{1:n})$, and let $\Pi_{\mathrm{full}}(\theta_{1}\in\cdot\mid z_{1:n})$ denote the full posterior distribution for $\theta_{1}$. For any $\varepsilon>0$, define $\Theta_{1}(\varepsilon):=\{\theta_{1}\in\Theta_{1}:\|\theta_{1}-\theta_{1,0}\|\leq\varepsilon\}$. For $P_{0}^{(n)}$ the true data generating process, we say that $X_{n}\xrightarrow{P}a$ (or $X_{n}=a+o_{p}(1)$) if for all $\varepsilon>0$, $P_{0}^{(n)}(\|X_{n}-a\|\geq\varepsilon)=o(1)$ as $n\rightarrow\infty$.

Assumption 1 embodies cases where a practitioner can confidently determine that the second module is misspecified. Lemma 1 then shows that, in these cases, the cut posterior concentrates on the true value $\theta_{1,0}$, while the full posterior does not. By restricting the flow of information across modules, cutting feedback produces inferences for $\theta_{1,0}$ that are more accurate than full Bayesian inference. Although Lemma 1 implies that cut posterior inference is more accurate than full posterior inference when the second module is grossly misspecified, it is of limited use when $\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})$ and $\pi_{\mathrm{full}}(\theta_{1}\mid z_{1:n})$ are similarly located, which can occur when misspecification of the second module is not severe. In such cases, the full posterior will have less uncertainty than the cut posterior, while also being similarly located, making it unclear whether to prefer the cut or full posterior.

To capture the empirically relevant setting where, for any fixed $n$, it is unclear which posterior to use, we investigate the behavior of these posteriors under a certain class of locally misspecified DGPs. Following the literature of robust asymptotic statistical analysis (see, e.g., Ch 3. Rieder, 2012), we approximate the empirical situation where neither method is clearly preferable using a local perturbation about the assumed model:

$\displaystyle h(z\mid\theta_{0},\delta_{n}):=f_{1}(z\mid\theta_{1,0})\{1+\psi^{\top}\zeta(z)/\sqrt{n}\}f_{2}(z\mid\theta_{0}),\quad\delta_{n}=\psi^{\top}\zeta/\sqrt{n},$

(3)

which depends on a (random) direction of misspecification $\zeta\in\mathbb{R}^{d}$, and magnitude $\psi$ that takes values in $\Delta\subset\mathbb{R}^{d}$. Under the local perturbation framework in (3) the cut and full posteriors have similar locations in $\Theta$ for small-to-moderate sample sizes, and conventional specification testing methods cannot consistently detect which method delivers more accurate inferences for $\theta_{0}$. As such, this class of perturbations allows us to compare cut and full posterior inference when correct specification of the second module is ambiguous.

Under Assumption 2, cut posterior inference for $\theta_{1}$ is not impacted by misspecification of the second module. To show this, we require further notation. First, note that

$$\ell(\theta)=\log f(z_{1:n}\mid\theta)=\log f_{1}(z_{1:n}\mid\theta_{1})+\log f_{2}(z_{1:n}\mid\theta)\equiv\ell_{p}(\theta_{1})+\ell_{c}(\theta_{1},\theta_{2}),$$

where $\ell_{p}(\theta_{1})$ denotes the partial log-likelihood for $\theta_{1}$ used in the cut posterior, and $\ell_{c}(\theta)$ denotes the log-likelihood used in the conditional posterior. Denote the derivative of the full log-likelihood as $\dot{\ell}(\theta):=\partial\ell(\theta)/\partial\theta$, and the second derivative by $\ddot{\ell}(\theta):=\partial^{2}\ell(\theta)/\partial\theta\partial\theta^{\top}$. For $j,k\in\{1,2\}$, define the first and second partial derivatives as $\dot{\ell}_{(j)}(\theta):=\partial\ell(\theta)/\partial\theta_{j}$, and $\ddot{\ell}_{(ij)}(\theta):=\partial^{2}\ell(\theta)/\partial\theta_{j}\partial\theta_{i}^{\top}$. Similar notations will be use to denote derivatives of $\ell_{u}(\theta)$, for $u\in\{c,p\}$. Let $\mathbb{E}_{n}[g(z)]$ denote the expectation of $g:\mathcal{Z}\rightarrow\mathbb{R}^{d}$, under $h(z\mid\theta_{0},\delta_{n})$ in Assumption 2, and define the following information matrices: $\mathcal{I}:=-\lim_{n}n^{-1}\mathbb{E}_{n}[\ddot{\ell}(\theta_{0})]$, $\mathcal{I}_{jk}:=-\lim_{n}n^{-1}\mathbb{E}_{n}[\ddot{\ell}_{(jk)}(\theta_{0})]$, and $\mathcal{I}_{p(11)}:=-\lim_{n}n^{-1}\mathbb{E}_{n}[\ddot{\ell}_{p}(\theta_{1,0})]$. Let $\overline{\theta}_{\mathrm{cut}}:=\int_{\Theta}\theta\pi_{\mathrm{cut}}(\theta\mid z_{1:n})\mathrm{d}\theta$, $\overline{\theta}_{\mathrm{full}}=\int_{\Theta}\theta\pi_{\mathrm{full}}(\theta\mid z_{1:n})\mathrm{d}\theta$, and let $\Rightarrow$ denote weak convergence under $P^{(n)}_{0}$.

Lemma 2 implies that the user is faced with a trade-off between conducting inference using the cut or full posterior: inferences under $\pi_{\mathrm{full}}(\theta\mid z_{1:n})$ have the smallest variability possible (via Cramer-Rao) but exhibit a bias of unknown magnitude, whereas inferences under $\pi_{\mathrm{cut}}(\theta\mid z_{1:n})$ are guaranteed to have smaller bias than those under $\pi_{\mathrm{full}}(\theta\mid z_{1:n})$ but have (weakly) larger variability. Lemma 2 is the first result to formally show that a bias-variance trade-off exists between the cut and full posteriors. Since the bias due to misspecification ($\eta$) is unobservable, Lemma 2 exemplifies the situation where it is ambiguous as to whether we should base inferences on the posterior that exhibits low bias - the cut posterior - or the posterior that exhibits larger bias but which has much smaller variability - the full posterior.

To build intuition as to how $\pi_{\omega}(\theta\mid z_{1:n})$ can utilize the bias-variance trade-off between $\pi_{\mathrm{cut}}$ and $\pi_{\mathrm{full}}$, we first focus on the behavior of the SMP for $\theta_{1}$, i.e., $\pi_{\omega}(\theta_{1}\mid z_{1:n})$ in (4), and analyze the behavior of $\pi_{\omega}(\theta\mid z_{1:n})$ in subsequent sections.²²2Differences between the cut, full, and SMP posteriors for $\theta_{2}$ are attributable to differences in the posterior for $\theta_{1}\mid z_{1:n}$ since each posterior shares the same conditional posterior for $\theta_{2}$ given $\theta_{1}$. Focusing on $\pi_{\omega}(\theta_{1}\mid z_{1:n})$ in (4), note that the SMP point estimator for $\theta_{1}$ is

$\displaystyle\overline{\theta}_{1}(\omega)$

$\displaystyle:=(1-\omega)\overline{\theta}_{1,\mathrm{cut}}+\omega\overline{\theta}_{1,\mathrm{full}}\equiv\overline{\theta}_{1,\mathrm{cut}}-\omega(\overline{\theta}_{1,\mathrm{cut}}-\overline{\theta}_{1,\mathrm{full}}),$

(5)

where $\overline{\theta}_{1,\mathrm{cut}}:=\int_{\Theta_{1}}\theta_{1}\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})\mathrm{d}\theta_{1}$, and $\overline{\theta}_{1,\mathrm{full}}:=\int_{\Theta}\theta_{1}\pi_{\mathrm{full}}(\theta_{1},\theta_{2}\mid z_{1:n})\mathrm{d}\theta$. From Lemma 2, the asymptotic mean of $\sqrt{n}(\overline{\theta}_{1,\mathrm{cut}}-\theta_{1,0})$ is zero, while that of $\sqrt{n}(\overline{\theta}_{1,\mathrm{full}}-\theta_{1,0})$ depends on the misspecification bias $\eta$. Hence, the SMP posterior mean $\overline{\theta}_{1}(\omega)$ combines an asymptotically unbiased estimator with high variance, $\overline{\theta}_{1,\mathrm{cut}}$, and an asymptotically biased estimator with small variance, $\overline{\theta}_{1,\mathrm{full}}$. Therefore, if we are willing to tolerate some bias it will be possible to choose $\omega$ to deliver inferences on $\theta_{1,0}$ that are more accurate than either the cut or full posterior alone, at least so long as our measure of “accuracy” accounts for both bias and variance.

The idea of combining biased and unbiased estimators has a long history in statistics, with the most commonly encountered estimators of this kind being shrinkage and James-Stein estimators, see Chapter 5 of Lehmann and Casella (2006) for a textbook discussion. Indeed, the form of $\overline{\theta}_{1}(\omega)$ in (5) is reminiscent of those encountered in the shrinkage literature. For two normally distributed estimators, one biased and the other unbiased, Green and Strawderman (1991) demonstrated how to optimally combine such estimators to deliver a shrinkage estimator that is optimal in terms of expected squared error loss. The approach of Green and Strawderman (1991) was extended to more general settings and loss functions by Kim and White (2001), and Judge and Mittelhammer (2004).

While $\overline{\theta}_{1,\mathrm{cut}}$ and $\overline{\theta}_{1,\mathrm{full}}$ are not normally distributed, they are asymptotically normal, and so one could choose $\omega$ in the SMP using existing shrinkage estimation proposals. Following ideas similar to Green and Strawderman (1991), and Kim and White (2001), we could choose $\omega$ in (4) as

$$\widetilde{\omega}_{+}=\min\{1,\widetilde{\omega}\},\quad\frac{\gamma}{n(\overline{\theta}_{1,\mathrm{cut}}-\overline{\theta}_{1,\mathrm{full}})^{\top}\Upsilon(\overline{\theta}_{1,\mathrm{cut}}-\overline{\theta}_{1,\mathrm{full}})},$$

(6)

for some $\gamma>0$, and $\Upsilon$ a positive definite $(d_{1}\times d_{1})$-matrix. Since $\overline{\theta}_{1,\mathrm{full}}$ is asymptotically biased, while $\overline{\theta}_{1,\mathrm{cut}}$ is not, using $\widetilde{\omega}_{+}$ within the SMP would allow us to interpret the SMP as shrinking cut posterior inferences towards those of the biased full posterior, and so using $\widetilde{\omega}_{+}$ would deliver a type of “shrinkage” SMP (S-SMP).

Semi-modular Bayesian inference is clearly much more general than the linear Gaussian models analyzed in Green and Strawderman (1991), Kim and White (2001), and Judge and Mittelhammer (2004). Nevertheless, applying shrinkage estimation ideas within semi-modular Bayesian inference should allow us to effectively combine the cut and full posteriors. In the following sections, we show that this is indeed the case: empirically and theoretically, SMPs based on weights similar to (6) deliver inferences that can be shown to be optimal according to a general notion of asymptotic risk. Before presenting any formal analysis, we first illustrate the behavior of the S-SMP based on $\widetilde{\omega}_{+}$ empirically in a simple example from the cutting feedback literature.

To demonstrate the benefits of the SMP, we consider a minor modification of the biased mean example in Section 2.1 of Liu et al. (2009). We observe two datasets generated from independent random variables with the same unknown mean $\theta_{1}$, but where the assumed model for the second dataset is incorrect resulting in biased estimation of the parameter of interest. The first dataset corresponds to $n_{1}$ observations on a $(d_{1}\times 1)$-dimensional, $d_{1}\geq 1$, random vector that we assume is generated from the model:

$$y_{1,i}=\theta_{1}+\epsilon_{1,i},$$

where $\theta_{1}=(\theta_{1,1},\dots,\theta_{1,d_{1}})^{\top}$ and $\epsilon_{1,i}$, $i=1,\dots,n_{1}$, are iid $N(0,I_{d_{1}})$. However, we also observe an additional dataset, comprised of $n_{2}>n_{1}$ observations which is assumed to be from the model:

$$y_{2,i}=\theta_{1}+\sigma\epsilon_{2,i},$$

where $\epsilon_{2i}$, $i=1,\dots,n_{2}$, are iid $N(0,I_{d_{1}})$ for unknown $\sigma>0$. The prior density for $\theta_{1}$ is $N(0,I_{d_{1}})$, and for $\theta_{2}:=\sigma^{2}$ is $\pi(\sigma^{2})\propto 1/\sigma^{2}$. The parameter of interest is $\theta_{1}$, and we wish to determine how much the second dataset should influence inference about $\theta_{1}$, when its assumed model is incorrect, leading to biased estimation of $\theta_{1}$.³³3The original example of Liu et al. (2009) is such that for even moderate values of the bias we always prefer the cut posterior. Given this, we have slightly modified the original example to ensure a meaningful trade-off exists between the cut and full posteriors. Without this modification, the S-SMP simply returns the cut posterior in the vast majority of cases.

Suppose that in the actual data-generating process $\epsilon_{2,i}$ is not $N(0,I_{d_{1}})$, but

$$\epsilon_{2,i}\stackrel{{\scriptstyle iid}}{{\sim}}\begin{cases}N(-0.25\cdot\iota_{d_{1}},0.10\cdot I_{d_{1}})&\text{ with prob. }\delta\\ N(0,0.5I_{d_{1}})&\text{ with prob. }1-\delta\end{cases}$$

for $i=1,\dots,n_{2}$, where $\iota_{d_{1}}$ denotes a $d_{1}\times 1$ dimensional vector of ones. When $\delta=0$, this reduces to the assumed model, but when $\delta>0$ we will obtain biased estimation of $\theta_{1}$ under the assumed model.

For our experiments, we assume $\sigma^{2}=1/2$ and use an equally spaced grid of values for the contamination $\delta\in\{0:0.05:0.9\}$. For each value in this grid, we generate 1000 replications from the above process in the case where $n_{1}=100$ and $n_{2}=1000$, and consider two different values of $d_{1}\in\{1,5\}$. For each dataset, the S-SMP is based on the following version of the weights in (6): for $\widehat{\mathcal{I}}_{p(11)}^{-1}=\text{Cov}_{\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})}(\theta_{1})$ and $\widehat{\mathcal{I}}_{11.2}^{-1}=\text{Cov}_{\pi(\theta_{1}\mid z_{1:n})}(\theta_{1})$

$$\widetilde{\omega}_{+}=\min\left\{1,\widetilde{\omega}\right\}\,\;\widetilde{\omega}=\frac{\text{\rm tr}[\widehat{\mathcal{I}}_{p(11)}^{-1}-\widehat{\mathcal{I}}_{11.2}^{-1}]}{\|\overline{\theta}_{1,\mathrm{cut}}-\overline{\theta}_{1,\mathrm{full}}\|^{2}}\mathbb{I}\left\{\text{\rm tr}[\widehat{\mathcal{I}}_{p(11)}^{-1}-\widehat{\mathcal{I}}_{11.2}^{-1}]>0\right\}.$$

To compare the impact of misspecification on the different modular posteriors we plot a Monte Carlo estimate of the expected squared error for the point estimators, based on 1000 replicated samples, across the grid of values for $\delta$. The results are presented in Figure 3 and show that for relatively small levels of contamination, the full posterior has lower expected squared error than the cut posterior, due to having a much smaller variance, while at higher levels of contamination the cut posterior has lower expected squared error. However, the expected squared error for the S-SMP is always lower than the cut posterior, which demonstrates that the SMP is able to ‘trade-off’ between the two posteriors to minimize squared error risk across all levels of contamination. However, we note that when $d_{1}=1$ and $\delta=0.90$, the S-SMP and cut posterior give very similar results.⁴⁴4Appendix D in the supplementary material contains additional experiments for this example. These results show that the S-SMP delivers reasonable results for all choices of $d_{1}$ and becomes more accurate as the dimension $d_{1}$ increases.

As we have already seen in Section 3.1, weights of the form presented in (6) deliver a SMP whose expected squared error was smaller than the cut posterior. Thus, in the remainder we focus our theoretical analysis on SMPs with pooling weights that are general versions of those in (6): for $\gamma_{n}$ a user-chosen sequence such that $\gamma_{n}\xrightarrow{P}\gamma$, $\gamma>0$, define the pooling weight

$\displaystyle\widehat{\omega}_{+}:=\min\{1,\widehat{\omega}\},\quad\widehat{\omega}=\frac{\gamma_{n}}{n(\overline{\theta}_{\mathrm{cut}}-\overline{\theta}_{\mathrm{full}})^{\top}\Upsilon(\overline{\theta}_{\mathrm{cut}})(\overline{\theta}_{\mathrm{cut}}-\overline{\theta}_{\mathrm{full}})}.$

(7)

In contrast to the weight $\widetilde{\omega}_{+}$ suggested in (6), the weight $\widehat{\omega}_{+}$ in (7) depends on the entire vector $\theta$, and the curvature of the loss function, as captured by the matrix $\Upsilon(\theta)=\partial^{2}g(\theta,\delta)/\partial\delta\partial\delta^{\top}|_{\delta=\theta}$. Incorporating the curvature of the loss within the pooling weight is necessary for the SMPs to deliver inferences that are accurate according to the chosen loss; see Theorem C.3 of Appendix C.3 for further discussion.

The pooling weight in (7) yields the shrinkage SMP (S-SMP):

$\displaystyle\pi_{\widehat{\omega}_{+}}(\theta\mid z_{1:n}):=\{(1-\widehat{\omega}_{+})\pi_{\mathrm{cut}}(\theta_{1}\mid z_{1:n})+\widehat{\omega}_{+}\pi(\theta_{1}\mid z_{1:n})\}\pi(\theta_{2}\mid\theta_{1},z_{1:n}).$

(8)

Different choices of $\gamma_{n}$ in (7) deliver different weights and ultimately different posteriors. However, the following result shows that across a range of choices for $\gamma_{n}$, the P-risk of $\pi_{\widehat{\omega}_{+}}$ in (8) is dominated by that of the cut posterior, and, under certain conditions, that of the full posterior. To state this result simply, define the $d\times d$-dimensional matrix $\Upsilon:=\Upsilon(\theta_{0})$, and denote the $(d_{2}\times d_{2})$-block of $\Upsilon$ by $\Upsilon_{22}$ (see Assumption 3); let $\mathcal{M}$ be a $d\times d$ matrix with $(d_{1}\times d_{1})$-block $W:=(\mathcal{I}_{p(11)}^{-1}-\mathcal{I}_{11.2}^{-1})$, $(d_{2}\times d_{2})$-block $V:=\mathcal{I}_{22}^{-1}\mathcal{I}_{21}W\mathcal{I}_{12}I_{22}^{-1}$, and $(d_{1}\times d_{2})$-block $-W\mathcal{I}_{12}\mathcal{I}_{22}^{-1}$, where $\mathcal{I}_{11.2}:=\mathcal{I}_{11}-\mathcal{I}_{12}\mathcal{I}_{22}^{-1}\mathcal{I}_{21}$. We remind the reader that $\mathcal{I}$, $\mathcal{I}_{jk}$ ($j,k\in\{1,2\}$) and $\mathcal{I}_{p(11)}$ are defined in Section 2.2.

Part one of Theorem 1 implies that if the cut posterior is inefficient relative to the full posterior, as measured by $\text{\rm tr}\Upsilon\mathcal{M}\geq 2\|\Upsilon\mathcal{M}\|$, then the S-SMP will be at least as accurate (in P-risk) as the cut posterior, and potentially more accurate than the full posterior.⁶⁶6Theorem 1 applies even if $\eta=0$: when there is no misspecification bias the cut and full posterior means will be similar and the weight $\widehat{\omega}_{+}$ will be close to unity so that the S-SMP will resemble the full posterior. The second part of Theorem 1 gives a sufficient, but not necessary, condition which guarantees that the P-risk of the full posterior dominates that of the S-SMP. This condition is likely to be satisfied when the difference in posterior locations is larger than the difference in posterior variances.

When $d>2$ it is also possible to obtain an analytic expression for the P-risk of the S-SMP if we set $q(\theta,\theta_{0})=\frac{1}{2}(\theta-\theta_{0})^{\top}\mathcal{M}^{-1}(\theta-\theta_{0})$, so that $\Upsilon=\mathcal{M}^{-1}$. The requirement that $d>2$ is commonly encountered in the risk analysis of James-Stein estimators and is a consequence of the fact that $\pi_{\widehat{\omega}_{+}}(\theta\mid z_{1:n})$ can be viewed as performing a type of posterior shrinkage.