Gibbs posterior concentration rates under sub-exponential type losses

Consider a measurable space $(\mathbb{U},\mathcal{U})$, with $\mathcal{U}$ a $\sigma$-algebra of subsets of $\mathbb{U}$, on which a probability measure $P$ is defined. A random element $U\sim P$ need not be a scalar, and many of the applications we have in mind involve $U=(X,Y)$ or $U=(X,Y,Z)$, where $Y$ denotes a “response” variable and $X$ or $(X,Z)$ denotes a “predictor” variable, and $P$ encodes the dependence between the entries in $U$. Then the real-world phenomenon under investigation is determined by $P$ and our goal is to make inference on a relevant feature of $P$, which we define as a given functional $\theta=\theta(P)$, taking values in $\Theta$. Note that $\theta$ could be finite-, high-, or even infinite-dimensional.

The specific way $\theta$ relates to $P$ guides our posterior construction. Suppose there is a loss function, $\ell_{\theta}(u)$, that measures how closely a generic value of $\theta$ agrees with a data point $u$. (As is customary, “$\theta$” will denote both the quantity of interest and a generic value of that quantity; when we need to distinguish the true from a generic value, we will write “$\theta^{\star}$.”) For example, if $u=(x,y)$ is a predictor–response pair, and $\theta$ is a function, then the loss might be

depending on whether $y$ is continuous or discrete/binary, where $1(A)$ denotes the indicator function at the event $A$. Another common situation is when one specifies a statistical model, say, $\{P_{\theta}:\theta\in\Theta\}$, indexed by a parameter $\theta$, and sets $\ell_{\theta}(u)=-\log p_{\theta}(u)$, where $p_{\theta}$ is the density of $P_{\theta}$ with respect to some fixed dominating measure. In all of these cases, the idea is that a loss is incurred when there is a certain discrepancy between $\theta$ and the data point $u$. Then our inferential target is the value of $\theta$ that minimizes the risk or average loss/discrepancy.

Given that estimation/inference is our goal, our focus will be on case where the risk minimizer, $\theta^{\star}$, is unique, so that the “$\in$” in (2) is an equality. But this is not absolutely necessary for our theory. Indeed, the main results in Section 3 remain valid even if the risk minimizer is not unique, and we make a few brief comments about this extension in the discussion following Theorem 3.2.

The risk function itself is unavailable—it depends on $P$—and, therefore, so is $\theta^{\star}$. However, suppose that we have an independent and identically distributed (iid) sample $U^{n}=(U_{1},\ldots,U_{n})$ of size $n$, with each $U_{i}$ having marginal distribution $P$ on $\mathbb{U}$. The iid assumption is not crucial, but it makes the notation and discussion easier; an extension to independent but not identically distributed (inid) cases is discussed in the context of an example in Section 5.4. In general, we have data $U^{n}$ taking values in the measurable space $(\mathbb{U}^{n},\mathcal{U}^{n})$, with joint distribution denoted by $P^{n}$. From here, we can naturally replace the unknown risk in Definition 2.1 with an empirical version and proceed accordingly.

Naturally, if the inferential target is the risk minimizer, then it makes sense to estimate that quantity based on data $U^{n}$ by minimizing the empirical risk, i.e.,

This is the M-estimator based on an objective function determined by the loss $\ell_{\theta}$; when $R_{n}$ is differentiable, the root of $\dot{R}_{n}$, the derivative of $R_{n}$, is a Z-estimator and “$\dot{R}_{n}(\theta)=0$” is often called an estimating equation (Godambe,, 1991; van der Vaart,, 1998). Since $\theta\mapsto\ell_{\theta}$ need not be smooth or convex, and $R_{n}$ is an average over a finite set of data, it is possible that its minimizer is not unique, even if $\theta^{\star}$ is. These computational challenges are, in fact, part of what motivates the Gibbs posterior, as we discuss below.

There is a rich literature on the asymptotic distribution properties of M-estimators, which can be used for developing hypothesis tests and confidence intervals (Maronna et al.,, 2006; Huber and Ronchetti,, 2009). As an alternative, one might consider a Bayesian approach to quantify uncertainty, but there is an immediate obstacle, namely, no statistical model/likelihood connecting the data to the quantity of interest. If we did have a statistical model, with a density function $p_{\theta}$, then the most natural loss is $\ell_{\theta}(u)=-\log p_{\theta}(u)$ and the likelihood is $\exp\{-nR_{n}(\theta)\}$. It is, therefore, tempting to follow that same strategy for general losses, resulting in a sort of generalized posterior distribution for $\theta$.

We will assume that the right-hand side of (5) is integrable in $\theta$, so that the proportionality constant is well-defined. Integrability holds whenever the loss function is bounded from below, like for those in (1), but this could fail in some cases where the loss is not bounded away from $-\infty$, e.g., when $\ell_{\theta}(u)$ is a negative log-density. In such cases, extra conditions on the prior distribution would be required to ensure the Gibbs posterior is well-defined.

An immediate advantage of this approach, compared to the M-estimation strategy described above, is that the user is able to incorporate available prior information about $\theta$ directly into the analysis. This is especially important in cases where the quantity of interest has a real-world meaning, as opposed to being just a model parameter, where having genuine prior information is the norm rather than the exception. Additionally, even though there is no likelihood, the same computational methods, such as Markov chain Monte Carlo (Chernozhukov and Hong,, 2003) and variational approximations (Alquier et al.,, 2016), common in Bayesian analysis, can be employed to numerically approximate the Gibbs posterior.

We have opted here to define the Gibbs posterior directly as an object to be used and studied, but there is a more formal, more principled way in which Gibbs posteriors emerge. In the PAC-Bayes literature, the goal is to construct a randomized estimator that concentrates in regions of $\Theta$ where the risk, $R(\theta)$, or its empirical version, $R_{n}(\theta)$, is small (Valiant,, 1984; McAllester,, 1999; Alquier,, 2008; Guedj,, 2019). That is, the Gibbs posterior can be viewed as a solution to an optimization problem rather than a solution to the updating-prior-beliefs problem. More formally, for a given prior $\Pi$ on $\Theta$, suppose the goal is to find

where the infimum is over all probability measures $\mu$ that are absolutely continuous with respect to $\Pi$, and $K$ denotes the Kullback–Leibler divergence. Then it can be shown (e.g., Zhang,, 2006; Bissiri et al.,, 2016) that the unique solution is $\Pi_{n}$, the Gibbs posterior defined in (5). Therefore, the Gibbs posterior distribution is the measure minimizing a penalized risk, averaged with respect to a given prior, $\Pi$.

Readers familiar with M-estimation may not recognize the learning rate, $\omega$. This does not appear in the M-estimation context because all that influences the optimization problem—and the corresponding asymptotic distribution theory—is the shape of the loss/risk function, not its magnitude or scale. On the other hand, the learning rate is an essential component of the Gibbs posterior distribution in (5) since the distribution depends on both the shape and scale of the loss function. Data-driven strategies for tuning the learning rate are available (Holmes and Walker,, 2017; Syring and Martin,, 2019; Lyddon et al.,, 2019; Wu and Martin,, 2022, 2021).

Here we focus on how the learning rate affects posterior concentration. In typical examples, our results require the learning rate to be a sufficiently small constant. That constant depends on features of $P$, which are generally unknown, so, in practice, the learning rate can be taken to be a slowly vanishing sequence, which has a negligible effect on the concentration rate. In more challenging examples, we require the learning rate to vanish sufficiently fast in $n$; this is also the case in Grünwald and Mehta, (2020).

A generalized posterior is any data-dependent distribution other than a well-specified Bayesian posterior. Examples include Gibbs and misspecified Bayesian posteriors, which we compare first.

A key characteristic of the misspecified Bayesian posterior is that it is accidentally misspecified. That is, the data analyst does his/her best to posit a sound model, $Q_{\gamma}$, for the data-generating process and obtains the corresponding posterior for the model parameter $\gamma$. That model will typically be misspecified, i.e., $P\not\in\{Q_{\gamma}:\gamma\in\Gamma\}$, so the aforementioned posterior will, under certain conditions, concentrate around the point $\gamma^{\dagger}$ that minimizes the Kullback–Leibler divergence of $Q_{\gamma}$ from $P$ (Kleijn and van der Vaart,, 2006; De Blasi and Walker,, 2013; Ramamoorthi et al.,, 2015). For a feature $\theta(\cdot)$ of interest, typically $\theta(Q_{\gamma^{\dagger}})\neq\theta(P)$, so there is generally a bias that the data analyst can do nothing about. A Gibbs posterior, on the other hand, is purposely misspecified—no attempt is made to model $P$. Rather, it directly targets the feature of interest via the loss function that defines it. This strategy avoids model misspecification bias, but its point of view also sheds light on the importance of the choice of learning rate. Since the data analyst knows the Gibbs posterior is not a correctly specified Bayesian posterior, they know the learning rate must be handled with care.

A number of authors have studied generalized posterior distributions formed using a likelihood raised to a power $\eta\in(0,1]$; see, e.g., Martin et al., (2017) and Grünwald and van Ommen, (2017) among others. These $\eta$-generalized posteriors tend to be robust to misspecification of the probability model, and data-driven methods to tune $\eta$ are developed in, e.g., Grünwald and van Ommen, (2017). These posteriors coincide with Gibbs posteriors based on a log-loss and with the learning rate $\omega$ corresponding to $\eta$.

A common non-Bayesian approach to statistics bases inferences on moment conditions of the form $Pf_{j}=0$ for functions $f_{1},\ldots f_{J}$ , rather than on a fully-specified likelihood. A number of authors have extended this framework to posterior inference. Kim, (2002) uses the moment conditions to construct a so-called limited-information likelihood to use in place of a fully-specified likelihood in a Bayesian formulation. Similarly, Chernozhukov and Hong, (2003) construct a posterior by taking a (pseudo) log-likelihood equal to a quadratic form determined by the set of moment conditions. Chib et al., (2018) studies Bayesian exponentially-tilted empirical likelihood posterior distributions, also defined by moment conditions. In some cases the Gibbs posterior distribution coincides with the above moment-based methods. For instance, in the special case that the risk $R$ is differentiable at $\theta^{\star}$ with derivative $\dot{R}(\theta^{\star})$, risk-minimization corresponds to the single moment condition $\dot{R}(\theta^{\star})=0$.

A large part of the Bayesian literature concerns the asymptotic concentration properties of their posterior distributions. Roughly, if data are generated from a distribution for which the quantity of interest takes value $\theta^{\star}$, then, as $n\to\infty$, the posterior distribution ought to concentrate its mass around that same $\theta^{\star}$. As we will show, optimal concentration rates are possible with Gibbs posteriors, so the robustness achieved by not specifying a statistical model has no cost in terms of (asymptotic) efficiency.

Towards this, for a fixed $\theta^{\star}\in\Theta$, let $d(\theta;\theta^{\star})$ denote a divergence measure on $\Theta$ in the sense that $d(\theta;\theta^{\star})\geq 0$ for all $\theta$, with equality if and only if $\theta=\theta^{\star}$. The divergence measure could depend on the sample size $n$ or other deterministic features of the problem at hand, especially in the independent but not iid setting; see Section 5.4. Our objective is to provide conditions under which the Gibbs posterior will concentrate asymptotically, at a certain rate, around $\theta^{\star}$ relative to the divergence measure $d$. Throughout this paper, $(\varepsilon_{n})$ denotes a deterministic sequence of positive numbers with $\varepsilon_{n}\to 0$, which will be referred to as the Gibbs posterior concentration rate.

In the PAC-Bayes literature, the Gibbs posterior distribution is interpreted as a “randomized estimator,” a generator of random $\theta$ values that tend to make the risk difference small. For iid data and with risk divergence $d(\theta;\theta^{\star})=\{R(\theta)-R(\theta^{\star})\}^{1/2}$, a concentration result like that in Definition 3.1 makes this strategy clear, since the $\Pi_{n}$-probability of the event $\{R(\theta)-R(\theta^{\star})\leq\varepsilon_{n}^{2}\}$ would be $\to 1$.

If the Gibbs posterior concentrates around $\theta^{\star}$ in the sense of Definition 3.1, then any reasonable estimator derived from that distribution, such as the mean, should inherit the $\varepsilon_{n}$ rate at $\theta^{\star}$ relative to the divergence measure $d$. This can be made formal under certain conditions on $d$; see, e.g., Corollary 1 in Barron et al., (1999) and the discussion following the proof of Theorem 2.5 in Ghosal et al., (2000).

Besides concentration rates, in certain cases it is possible to establish distributional approximations to Gibbs posteriors, i.e., Bernstein–von Mises theorems. Results for finite-dimensional problems and with sufficiently smooth loss functions can be found in, e.g., Bhattacharya and Martin, (2022) and Chernozhukov and Hong, (2003).

Here we discuss a general strategy for proving Gibbs posterior concentration and the kinds of sufficient conditions needed for the strategy to be successful. To start, set $A_{n}=\{\theta:d(\theta,\theta^{\star})>M_{n}\varepsilon_{n}\}\subset\Theta$. Then our first step towards proving concentration is to express $\Pi_{n}(A_{n})$ as the ratio

The goal is to suitably upper and lower bound $N_{n}(A_{n})$ and $D_{n}$, respectively, in such a way that the ratios of these bounds vanish. Two sufficient conditions are discussed below, the first primarily dealing with the loss function and aiding in bounding $N_{n}(A_{n})$ and the second primarily concerning the prior distribution and aiding in bounding $D_{n}$. Both conditions concern the excess loss $\ell_{\theta}(U)-\ell_{\theta^{\star}}(U)$ and its mean and variance:

In this section we present general results on Gibbs posterior concentration. Proofs can be found in Section 1 of the supplementary material. Our first result establishes Gibbs posterior concentration, under Condition 1 and a local prior condition, for sufficiently small constant learning rates.

For a brief sketch of the proof bound the posterior probability of $A_{n}$ by

where $b_{n}$ is as above. Taking expectation of both sides, and applying (11) and the consequence (9) of Condition 1, we get

Then the right-hand side is generally of order $(n\varepsilon_{n}^{r})^{-1}\to 0$. To compare with the results in Grünwald and Mehta, (2020, Example 2), their upper bound on $P^{n}\Pi_{n}(A_{n})$ is $O(M_{n}^{-1})$, which vanishes arbitrarily slowly.

For the case where the risk minimizer in (2) is not unique, certain modifications of the above argument can be made, similar to those in Kleijn and van der Vaart, (2006, Theorem 2.4). Roughly, to our Theorem 3.2 above, we would add the requirements that (12) and Condition 1 hold uniformly in $\theta^{\star}\in\Theta^{\star}$, where $\Theta^{\star}$ is the set of risk minimizers. Then virtually the same proof shows that $P^{n}\Pi_{n}(\{\theta:d(\theta,\Theta^{\star})\gtrsim\varepsilon_{n}\})\to 0$, where $d(\theta,\Theta^{\star})=\inf_{\theta^{\star}\in\Theta^{\star}}d(\theta,\theta^{\star})$.

Theorem 3.2 is quite flexible and can be applied in a range of settings; see Section 5. However, one case in which it cannot be directly applied is when $n\varepsilon_{n}^{r}$ is bounded. For example, in sufficiently smooth finite-dimensional problems, we have $r=2$ and the target rate is $\varepsilon_{n}=n^{-1/2}$. The difficulty is caused by the prior bound in (12), since it is impossible—at least with a fixed prior—to assign mass bounded away from 0 to a shrinking neighborhood of $\theta^{\star}$. One option is to add a logarithmic factor to the rate, i.e., take $\varepsilon_{n}=(\log n)^{k}n^{-1/2}$, so that $e^{-Cn\varepsilon_{n}^{2}}$ is a power of $n^{-1/2}$. Alternatively, a refinement of the proof of Theorem 3.2 lets us avoid slowing down the rate.

The learning rate is critical to the Gibbs posterior’s performance, but in applications it may be challenging to determine the upper bound $\bar{\omega}$ for which Condition 1 holds. For a simple illustration, suppose the excess loss $\ell_{\theta}-\ell_{\theta}^{\star}$ is normally distributed with variance $\sigma^{2}(\theta)$ satisfying $\sigma^{2}(\theta)\leq c\{R(\theta)-R(\theta^{\star})\}$, a kind of Bernstein condition. In this case, Condition 1 holds for $d(\theta,\theta^{\star})=\{R(\theta)-R(\theta^{\star})\}^{1/2}$, with $r=2$, if $\omega<2c^{-1}$. Bernstein conditions can be verified in many practical examples, but the factor $c$ would rarely be known. Consequently, we need $\omega$ to be sufficiently small, but the meaning of “sufficiently small” depends on unknowns involving $P$. However, any positive, vanishing learning rate sequence $\omega=\omega_{n}$ satisfies $\omega_{n}\in(0,\bar{\omega})$ for all sufficiently large $n$. And if $\omega_{n}$ vanishes arbitrarily slowly, then it has no effect on the Gibbs posterior concentration rate; see Section 5.6. All we require to accommodate a vanishing learning rate is a slightly stronger $\omega_{n}$-dependent version of the prior concentration bound in (12).

The proof of Theorem 3.4 is almost identical to that of Theorem 3.2, hence omitted. But to see the basic idea, we mention two key observations. First, since Condition 1 holds for $\omega_{n}$ for all sufficiently large $n$, and since $\omega_{n}$ is deterministic, by the same argument producing the bound in (9), we get

Second, the same argument producing the bound in (11) shows that

Then the only difference between this situation and that in Theorem 3.2 is that, here, the numerator bound only holds for “sufficiently large $n$,” instead of for all $n$. This makes Theorem 3.4 is slightly weaker than Theorem 3.2 since there are no finite-sample bounds.

When $\omega_{n}\equiv\omega$ the constant learning rate is absorbed by $C$ and there is no difference between the prior bounds in (12) and (14). But, the prior probability assigned to the $(m\vee v)$-neighborhood of $\theta^{\star}$ does not depend on $\omega_{n}$, so if it satisfies (12), then the only way it could also satisfy (14) is if $\varepsilon_{n}$ is bigger than it would have been without a vanishing learning rate. In other words, Theorem 3.2 requires $n\varepsilon_{n}^{r}\rightarrow\infty$ whereas Theorem 3.4 requires $n\omega_{n}\varepsilon_{n}^{r}\rightarrow\infty$, which implies that for a given vanishing $\omega_{n}$ Theorem 3.2 potentially can accommodate a faster rate $\varepsilon_{n}$. Therefore, we see that a vanishing learning rate can slow down the Gibbs posterior concentration rate if it does not vanish arbitrarily slowly. There are applications that require the learning rate to vanish at a particular $n$-dependent rate, and these tend to be those where adjustments like in Section 4 are needed; see Sections 5.3.2 and 5.5.2.

If we can use the data to estimate $\bar{\omega}$ consistently, then it makes sense to choose a learning rate sequence depending on this estimator. Suppose our data-dependent learning rate $\hat{\omega}_{n}$ satisfies $\hat{\omega}_{n}<\bar{\omega}$ with probability converging to $1$ as $n\rightarrow\infty$. For $\omega\equiv\hat{\omega}_{n}$ the conclusion of Theorem 3.2 holds for all sufficiently large $n$; and see Section 4.2 in the Supplementary Material. One advantage of this strategy is that it avoids using a vanishing learning rate, which may slow concentration.

Of course, Condition 1 is only useful if it can be checked in practically relevant examples. As mentioned in Section 3.2, Grünwald and Mehta’s strong central and witness conditions are sufficient for Condition 1. A pair of slightly stronger, but practically verifiable conditions require the excess loss is sub-exponential with first and second moments that obey a Bernstein condition. For the examples we consider in Section 5 where Condition 1 can be used, these two conditions are convenient.

In some cases the moment generating function bound in Condition 1 can be verified in a neighborhood of $\theta^{\star}$ but not for all $\theta\in\Theta$. For example, suppose $\theta\mapsto\ell_{\theta}(u)$ is Lipschitz with respect to a metric $\|\cdot\|$ with uniformly bounded Lipschitz constant $L=L(u)$, and that, for some $\delta>0$,

That is, the Bernstein condition (17) holds but for different values of $\alpha$ depending on $\theta$. This is the case for quantile regression; see Section 5.6. This class of problems, where the Bernstein exponent varies across $\Theta$, apparently has not been considered previously. For example, Grünwald and Mehta, (2020) only consider cases where the Bernstein exponent $\alpha$ is constant across the entire parameter space; consequently, their results assume $\Theta$ is bounded (e.g., Grünwald and Mehta,, 2020, Example 10), so (17) holds trivially with exponent $\alpha=0$.

To address this problem, our idea is a simple one. We propose to introduce a sieve which is large enough that we can safely assume it contains $\theta^{\star}$ but small enough that, on which, the $m$ and $v$ functions can be appropriately controlled. Towards this, let $\Theta_{n}$ be an increasing sequence of subsets of $\Theta$, indexed by the sample size $n$. The “size” of $\Theta_{n}$ will play an important role in the result below. While more general sieves are possible, to keep the notion of size concrete, let $\Theta_{n}=\{\theta\in\Theta:\|\theta\|\leq\Delta_{n}\}$, so that size is controlled by the non-decreasing sequence $\Delta_{n}>0$, which would typically satisfy $\Delta_{n}\to\infty$ as $n\to\infty$. The metric $\|\cdot\|$ in the definition of $\Theta_{n}$ is at the user’s discretion, it would be chosen so that $\theta\in\Theta_{n}$ provides information that can be used to control the excess loss $\ell_{\theta}-\ell_{\theta^{\star}}$. This leads to the following straightforward modification of Condition 1.

Aside from the restriction $\Theta_{n}$, the key difference between the bounds here and in Condition 1 is in the exponent. Instead of there being a constant $K$, there is a sequence $K_{n}$ which is determined by the sieve’s size, controlled by $\Delta_{n}$. If the sequence $K_{n}$ is increasing, as we expect it will be, then we can anticipate that the overall concentration rate would be adversely affected—unless the learning rate is vanishing suitably fast. The following theorem explains this more precisely.

Condition 2 can be used exactly as Condition 1 to upper bound $P^{n}N_{n}(A_{n}\cap\Theta_{n})$. However, the Gibbs posterior probability assigned to $A_{n}\cap\Theta_{n}^{\text{\sc c}}$ must be handled separately.

There are two aspects of Theorem 4.1 that deserve further explanation. We start with the point about separate handling of $\Theta_{n}^{\text{\sc c}}$. Condition (22) is easy to check for well-specified Bayesian posteriors, but these results do not carry over even to a misspecified Bayesian model. Of course, (22) can always be arranged by restricting the support of the prior distribution to $\Theta_{n}$, which is the suggestion in Kleijn and van der Vaart, (2006, Theorem 2.3) for the Bayesian case and how we handle (22) for our infinite-dimensional Gibbs example in Section 5.6. However, as (22) suggests, this is not entirely necessary. Indeed, Kleijn and van der Vaart, (2006) describe a trade-off between model complexity and prior support, and offer a more complicated form of their sufficient condition, which they do not explore in that paper. The fact is, without a well-specified likelihood, checking a condition like our (22) or Equation (2.13) in Kleijn and van der Vaart, (2006) is a challenge, at least in infinite-dimensional problems. For finite-dimensional problems, it may be possible to verify (22) directly using properties of the loss functions. For example, we use convexity of the check loss function for quantile estimation to verify (22) in Section 5.1.

Next, how might one proceed to check Condition 2? Go back to the Lipschitz loss case at the start of this subsection. For a sieve $\Theta_{n}$ as described above, if $\theta$ and $\theta^{\star}$ are in $\Theta_{n}$, then $\|\theta-\theta^{\star}\|$ is bounded by a multiple of $\Delta_{n}$. This, together with the Lipschitz property and (16), implies

where $C_{n}=O(1+\Delta_{n}\omega_{n})$. Suppose the user chooses $\omega_{n}$ and $\Delta_{n}$ such that $\omega_{n}\Delta_{n}=O(1)$; then we can replace $C_{n}$ by a constant $C$. If there exist functions $g$ and $G$ such that

then the above upper bound simplifies to

Then Condition 2 holds with $K_{n}=g(\Delta_{n})-C\omega_{n}G(\Delta_{n})$, and it remains to balance the choices of $\omega_{n}$ and $\Delta_{n}$ to achieve the best possible concentration rate $\varepsilon_{n}$; see Section 5.6.

When the excess loss is heavy-tailed, i.e., not sub-exponential, like in Section 5.3, its moment-generating function does not exist and, therefore, Condition 1 cannot be satisfied. In this section, we assume the loss function $\ell_{\theta}(u)$ is non-negative or lower-bounded by a negative constant. In the latter case we can work with the shifted loss—$\ell_{\theta}$ minus its lower bound. Many practically useful loss functions are non-negative, including squared-error loss, which we cover in Section 5.3.

In such cases, it may be reasonable to replace the heavy-tailed loss with a clipped version

where $t_{n}>0$ is a diverging clipping sequence. Since $\ell_{\theta}^{n}(u)$ is bounded in $(u,\theta)$ for each fixed $n$, the strategy for checking Condition 1 described in Section 3.4.1 suggests that, for certain choices of $(t_{n},\varepsilon_{n},\omega_{n})$, $\Pi_{n}$ places vanishing mass on the sequence of sets $\{\theta:P\ell^{n}_{\theta}-P\ell_{\theta_{n}^{\star}}^{n}>\varepsilon_{n}\}$, where $\theta_{n}^{\star}=\arg\min P\ell_{\theta}^{n}$. This makes the $\theta_{n}^{\star}$ the (moving) target of the Gibbs posterior, instead of the fixed $\theta^{\star}$. On the other hand, if the loss function admits more than one finite moment, then for a corresponding, increasing clipping sequence $t_{n}$ the clipped risk neighborhoods of $\theta^{\star}_{n}$ contain the risk neighborhoods of $\theta^{\star}$ for all large $n$, that is,