Optimal tests following sequential experiments

Despite the vast amount of work on the development of sequential learning algorithms, the literature on inference following the use of such algorithms is relatively sparse. One approach gaining some popularity in computer-science is called ‘any-time inference’. Here, one seeks to construct tests and confidence intervals that are correctly sized no matter how, or when, the experiment is stopped. We refer to Ramdas et al. (2022) for a survey and to Grünwald et al. (2020), Howard et al. (2021), Johari et al. (2022) for some recent contributions. The uniform-in-time size constraint is a stringent requirement, and this comes at the expense of lower power than could be achieved otherwise. By contrast, our focus in this paper is on classical notions of testing, where size control is only achieved when the experimental protocol, i.e., the specific sampling rule and stopping time, is followed exactly. In essence, this requires the decision maker to pre-register the experiment and fully commit to the protocol. We believe this is valid assumption in most applications; adaptive experiments are usually constructed with the explicit goal of welfare maximization, so there is little incentive to deviate from the protocol as long as the preferences of the experimenter and the end-user of the experiment are aligned (e.g., in the case of online marketplaces they would be the same entity). In other situations, pre-registration of the experimental design is usually mandatory, see, e.g., the FDA guidance on sequential designs (CBER, 2016).

There are other recent papers which propose inferential methods under the ‘classical’ hypothesis-testing framework. Zhang et al. (2020) and Hadad et al. (2021) suggest asymptotically normal tests for some specific classes of sequential experiments. These tests are based on re-weighing the observations. There are also a number of methods for group sequential and linear boundary designs commonly used in clinical trials, see Hall (2013) for a review. However, it is not clear if any of them are optimal even within their specific use cases.

Finally, in prior and closely related work to our own, Hirano and Porter (2023) obtain an Asymptotic Representation Theorem (ART) for batched sequential experiments that is different from ours and apply this to testing. The ART of Hirano and Porter (2023) is a lot more general than our own, e.g., it can be used to determine optimal conditional tests given outcomes from previous stages. However, this generality comes at a price as the state variables increase linearly with the number of batches. Here, we build on and extend these results to show that only a fixed number of sufficient statistics are needed to match the unconditional asymptotic power of any test, irrespective of the number of batches (our results also apply to asymptotic power conditional on stopping times). We also derive a number of additional results that are new to this literature: First, our ART for stopping-time experiments applies to fully adaptive experiments (this result is not based on Hirano and Porter, 2023; rather, it makes use of a representation theorem for stopping times due to Le Cam, 1979). Second, our analysis covers non-parametric models, which is important for applications. Third, we characterize the properties of optimal tests in a number of different scenarios, e.g., for testing linear combinations of parameters, or under unbiased and $\alpha$-spending requirements. This is useful as UMP tests do not generally exist otherwise.

As noted earlier, this paper employs the local asymptotic power criterion to rank tests. This criterion naturally leads to ‘diffusion asymptotics’, where the limit experiment consists of Gaussian diffusions. Diffusion asymptotics were first introduced by Wager and Xu (2021) and Fan and Glynn (2021) to study the properties of a class of sequential algorithms. In previous work (Adusumilli, 2021), this author demonstrated some asymptotic equivalence results for comparing the Bayes and minimax risk of bandit experiments. Here, we apply the techniques devised in those papers to study inference.

Before describing our procedures, it can be instructive to consider some examples of sequential experiments.

Consider a decision-maker (DM) who wishes to conduct an experiment involving some outcome variable $Y$. Before starting the experiment, the DM registers a stopping time, $\hat{\tau}$, that describes the eventual sample size in multiples of $n$ observations (see below for the interpretation of $n$). The choice of $\hat{\tau}$ may involve a balancing a number of considerations such as costs, ethics, welfare etc. Here, we abstract away from these issues and take $\hat{\tau}$ as given. In the course of the experiment, the DM observes a sequence of outcomes $Y_{1},Y_{2},\dots$ . The experiment ends in accordance with $\hat{\tau}$, which we assume to be adapted to the filtration generated by the outcome observations. Let $P_{\theta}$ denote a parametric model for the outcomes. Our interest in this section is in testing $H_{0}:\theta=\Theta_{0}$ vs $H_{1}:\theta\in\Theta_{1}$ where $\Theta_{0}\cap\Theta_{1}=\emptyset$. Let $\theta_{0}\in\Theta_{0}$ denote some reference parameter in the null set.

There are two notions of asymptotics one could employ in this setting, and consequently, two different interpretations of $n$. In many settings, e.g., group sequential trials, there is a limit on the maximum number of observations that can be collected; this limit is pre-specified and we take it to be $n$. Consequently, in these experiments, $\hat{\tau}\in[0,1]$. Alternatively, we may have open-ended experiments where the stopping time is determined by balancing the benefit of experimentation with the cost for sampling each additional unit of observation. In this case, we employ small-cost asymptotics and $n$ then indexes the rate at which the sampling costs go to $0$ (alternatively, we can relate $n$ to the population size in the implementation phase following the experiment, see Adusumilli, 2022). The results in this section apply to both asymptotic regimes.

Let $\varphi_{n}\in[0,1]$ denote a candidate test. It is required to be measurable with respect to $\sigma\{Y_{1},\dots,Y_{\left\lfloor n\tau\right\rfloor}\}$. Now, it is fairly straightforward to construct tests that have power 1 against any fixed alternative as $n\to\infty$. Consequently, to obtain a more fine-grained characterization of tests, we consider their performance against local perturbations of the form $\{\theta_{0}+h/\sqrt{n};h\in\mathbb{R}^{d}\}$. Denote $P_{h}:=P_{\theta_{0}+h/\sqrt{n}}$ and let $\mathbb{E}_{h}^{(a)}[\cdot]$ denote its corresponding expectation. Also, let $\nu$ denote a dominating measure for $\{P_{\theta}:\theta\in\mathbb{R}\}$, and set $p_{\theta}:=dP_{\theta}/d\nu$. We impose the following regularity conditions on the family $P_{\theta}$, and the stopping time $\hat{\tau}$:

Both assumptions are fairly innocuous. As noted previously, in many examples we already have $\tau\leq 1$.

Let $P_{nt,h}$ denote the joint probability measure over the iid sequence of outcomes $Y_{1},\dots,Y_{nt}$ and take $\mathbb{E}_{nt,\bm{h}}[\cdot]$ to be its corresponding expectation. Define the (standardized) score process $x_{n}(t)$ as

where $I:=\mathbb{E}_{0}[\psi(Y_{i})\psi(Y_{i})^{\intercal}]$ is the information matrix. It is well known, see e.g., Van der Vaart (2000, Chapter 7), that quadratic mean differentiability implies $\mathbb{E}_{nT,0}[\psi(Y_{i})]=0$ and that $I$ exists. Then, by a functional central limit theorem,

Here, and in what follows, $W(\cdot)$ denotes the standard $d$-dimensional Brownian motion. Assumption 1 also implies the important property of Sequential Local Asymptotic Normality (SLAN; Adusumilli, 2021): for any given $h\in\mathbb{R}^{d}$,

The above states that the likelihood ratio admits a quadratic approximation uniformly over all $t$.

In what follows, take $U$ to be a $\textrm{Uniform}[0,1]$ random variable that is independent of the process $x(\cdot)$, and define $\mathcal{F}_{t}:=\sigma\{x(s),U;s\leq t\}$ to be the filtration generated by $U$ and the stochastic process $x(\cdot)$ until time $t$.

Consider a limit experiment where one observes $U$ and a Gaussian diffusion $x(t):=I^{1/2}ht+W(t)$ with some unknown $h$, and constructs a test statistic $\varphi$ based on knowledge only of (i) an $\mathcal{F}_{t}$-adapted stopping time $\tau$ that is the limiting version of $\hat{\tau}$ (in a sense made precise below); and (ii) the stopped process $x(\tau)$. Let $\mathbb{P}_{h}$ denote the induced probability over the sample paths of $x(\cdot)$ given $h$, and $\mathbb{E}_{h}[\cdot]$ its corresponding expectation. The following theorem relates the original testing problem to the one in such a limit experiment:

The first part of Theorem 1 is essentially due to Le Cam (1979).

To the best of our knowledge, the second part of Theorem 1 is new. Previously, Le Cam (1979) showed that for $\{P_{\theta}\}$ in the exponential family of distributions,

Here, we extend the above to general families of distributions satisfying Assumption 1. We then derive an asymptotic representation theorem for $\varphi_{n}$ as a consequence of this result.

Note that in the second part of Theorem 1, $\tau$ is taken as given (this mirrors how $\hat{\tau}$ is taken as given in the context of the original experiment). It is chosen so that the first part of the theorem is satisfied. In order to derive optimal tests, one would need to know the joint distribution of $\tau,x(\tau)$. Unfortunately, the first part of Theorem 1 does not provide a characterization of $\tau$; it only asserts that such a stopping time must exist. Fortunately, in practice, most stopping times are functions, $\hat{\tau}=\tau(x_{n}(\cdot))$, of the score process, e.g., the optimal stopping time under costly sampling is given by $\hat{\tau}=\inf\{t:|x_{n}(t)|\geq\gamma\}$. Indeed, previous work by this author (Adusumilli, 2022) and others has shown that if the stopping time is to be chosen according some notion of Bayes or minimax risk, then it is sufficient to restrict attention to stopping times that depend only on $x_{n}(\cdot)$. In such cases, the continuous mapping theorem allows us to determine $\tau$ as $\tau=\tau(x(\cdot))$.

In this section, we study inference under a stronger version of the size constraint, inspired by the $\bm{\alpha}$-spending approach in group sequential trials (Gordon Lan and DeMets, 1983). Suppose that the stopping time is discrete, taking only the values $t=1,2,\dots,T$. Then, instead of an overall size constraint of the form $\mathbb{E}_{nT,\bm{0}}[\varphi_{n}]\leq\alpha$, we may specify a ‘spending-vector’ $\bm{\alpha}:=(\alpha_{1},\dots,\alpha_{T})$ satisfying $\sum_{t=1}^{T}\alpha_{t}=\alpha$, and require

In what follows, we call a test, $\varphi_{n}$, satisfying (2.4) a level-$\bm{\alpha}$ test (with a boldface $\bm{\alpha}$). Intuitively, if each $t$ corresponds to a different stage of the experiment, the $\bm{\alpha}$-spending constraint prescribes the maximum amount of Type-I error that may be expended at stage $t$. As a practical matter, it enables us to characterize a UMP or UMP unbiased test in settings where such tests do not otherwise exist. We also envision the criterion as a useful conceptual device: even if we are ultimately interested in a standard level-$\alpha$ test, we can obtain this by optimizing a chosen power criterion (average power, etc.) over the spending vectors $\bm{\alpha}:=(\alpha_{1},\dots,\alpha_{K})$ satisfying $\sum_{k}\alpha_{k}\leq\alpha$.

A particularly interesting example of an $\bm{\alpha}$-spending vector is $(\alpha P_{nT,0}(\hat{\tau}=1),\dots,\alpha P_{nT,0}(\hat{\tau}=k))$; this corresponds to the requirement that $\mathbb{E}_{nT,\bm{0}}\left[\left.\varphi_{n}\right|\hat{\tau}=t\right]\leq\alpha$ for all $t$, i.e., the test be conditionally level-$\alpha$ given any realization of the stopping time. This may have some intuitive appeal, though it does disregard any information provided by the stopping time for discriminating between the hypotheses.

Under the $\bm{\alpha}$-spending constraint, a test that maximizes expected power also maximizes expected power conditional on each realization of stopping time. This is a simple consequence of the law of iterated expectations. Consequently, we focus on conditional power in this section. Our main result here is a generalization of Theorem 1 to $\bm{\alpha}$-spending restrictions. The limit experiment is the same as in Section 2.2.

It may be possible to extend the above result to continuous stopping times using Le Cam’s discretization device, though we do not take this up here.

Earlier in this section, we took $\theta_{0}\in\Theta_{0}$ to be some reference parameter in the null set. However, such a choice may result in the limiting stopping time, $\tau$, collapsing to $0$. Consider, for example, the case of costly sampling (Example 1 in Section 1.2). In this experiment, the stopping time, $\hat{\tau}$, is itself chosen around a reference parameter $\theta_{0}$ (typically chosen so that the effect of interest is $0$ at $\theta_{0}$). But suppose we are interested in testing $H_{0}:\theta=\bar{\theta}_{0}$, for some $\bar{\theta}_{0}\neq\theta_{0}$. Under this null, $\hat{\tau}$ converges to $0$ in probability as $\bar{\theta}_{0}$ is a fixed distance away from $\theta_{0}$. This issue with the stopping time arises because the null hypothesis and the stopping time are not centered around the same reference parameter.

One way to still provide inference in such settings is to set the reference parameter to $\theta_{0}$, but employ a drifting null $H_{0}:h=h_{0}/\sqrt{n}$, where $h_{0}$ is taken to be fixed over $n$, and is calibrated as $\sqrt{n}(\bar{\theta}-\theta_{0})$. The null, $H_{0}$, thus changes with $n$, but for the observed sample size we are still testing $\theta=\bar{\theta}_{0}$. It is then straightforward to show that Theorems 1 and 2 continue to apply in this setting; asymptotically, the inference problem is equivalent to testing that the drift of $x(\cdot)$ is $I^{1/2}h_{0}$ in the limit experiment. The asymptotic approximation is expected to be more accurate the closer $\bar{\theta}_{0}$ is to $\theta_{0}$; but for distant values of $\bar{\theta}_{0}$, we caution that local asymptotics may not provide a good approximation.

So far we have described upper bounds on the asymptotic power functions of tests. Now, given a UMP test, $\varphi^{*}(\tau,x(\tau))$, in the limit experiment, we can construct a finite sample version of this, $\varphi_{n}^{*}:=\varphi^{*}(\hat{\tau},x_{n}(\hat{\tau}))$, by replacing $\tau,x(\tau)$ with $\hat{\tau},x_{n}(\hat{\tau})$. Since $x_{n}(\hat{\tau})$ depends on the information matrix, $I$, one would need to either calibrate it to $I(\theta_{0})$ (if $\theta_{0}$ is known), or replace it with a consistent estimate. We discuss variance estimators in Appendix B.1.

The test, $\varphi_{n}^{*}$, would then be asymptotically optimal, in the sense of attaining the power envelope, under mild assumptions. In particular, we only require that $\varphi^{*}(\cdot,\cdot)$ satisfy the conditions for an extended continuous mapping theorem. Together with (2.3) and the first part of Theorem 1, this implies

for any $h\in\mathbb{R}^{d}$. Then, a similar argument as in the proof of Theorem 1 shows that the local power of $\varphi_{n}^{*}$ converges to that of $\varphi^{*}$ in the limit experiment.