Inference in Nonparametric Series Estimation with Specification Searches for the Number of Series Terms

We consider the following nonparametric regression model

where $\{y_{i},x_{i}\}_{i=1}^{n}$ is i.i.d., $y_{i}$ is a scalar response variable, $x_{i}\in\mathcal{X}\subset\mathbb{R}^{d_{x}}$ is a vector of covariates, and $g_{0}(x)=E(y_{i}|x_{i}=x)$ is the conditional mean function. The theory of estimation and inference is well developed for nonparametric series (sieve) methods in a large body of econometrics and statistics literature. Series estimators have also received attention in applied economics because they have many appealing features, e.g., they can easily impose shape restrictions such as additive separability and monotonicity. Once the basis function is chosen (e.g., polynomial or regression spline series of fixed order), implementation requires a choice of the number of series terms $K=K_{n}$ where $K$ denotes the order of the polynomials or the number of knots in the splines. However, this often involves some ad-hoc specification searches over $K\in\mathcal{K}_{n}$. For example, when $x_{i}\in\mathbb{R}^{d_{x}}$ is vector valued, researchers often evaluate the different numbers of terms in each dimension separately and construct a set of bases with different powers and cross-products of covariates. Although specification search seems necessary in some cases, it may lead to misleading inference without considering the first-step specification search or series term selection.¹¹1As a referee noted, the bias and MSE of the series estimator depend on not only $K$ but also the specific bases or sieve spaces, e.g., the order of the splines. In this paper, we fix the basis function, and we do not allow searching over the specific bases or sieve spaces.

Existing theory for the asymptotic normality of t-statistics and valid inference imposes a so-called undersmoothing (i.e., overfitting) condition that is a faster rate of $K$ than the mean-squared error (MSE) optimal convergence rates, and many papers in the literature typically suggest rule-of-thumb rules that give the desired level of undersmoothing. Among many others, Newey (2013) suggested increasing $K$ until the standard errors are large relative to small changes in objects of interest. Newey, Powell, and Vella (1999) suggested using more terms than that chosen by cross-validation. Horowitz and Lee (2012) suggested increasing $K$ until the integrated variance suddenly increases and then adding additional terms.

In this paper, we formally justify these rule-of-thumb methods or “plug-in” methods with undersmoothed $\widehat{K}$ for valid inference in nonparametric series regression. Specifically, we provide pointwise inference for $g_{0}(x)$ with possibly data-dependent (undersmoothed) $\widehat{K}\in\mathcal{K}_{n}$, i.e., constructing $100(1-\alpha)\%$ confidence interval (CI),

with an estimator $\widehat{g}_{n}(K,x)$, variance $\widehat{V}_{n}(K,x)$ using $K$ series terms, and critical values $\widehat{c}_{1-\alpha}(x)$ from the supremum of the t-statistics. For this result, we first develop a uniform distributional approximation theory of the absolute value of the supremum of the t-statistics over different series terms to construct asymptotically valid confidence intervals, which are uniform in $K\in\mathcal{K}_{n}$,

The critical values $\widehat{c}_{1-\alpha}(x)$ can be easily implemented using simple simulation or weighted bootstrap methods.

Furthermore, this paper develops the construction of confidence bands for $g_{0}(x)$ with asymptotically uniform (in $K\in\mathcal{K}_{n}$) coverage with critical values $\widehat{c}_{1-\alpha}$ chosen to satisfy

Analogous to the pointwise inference in (1.2), we can show the validity of confidence bands with the data-dependent $\widehat{K}$. Even in pointwise inference, deriving a uniform asymptotic distribution theory for all sequences of t-statistics over $K\in\mathcal{K}$ may not be possible unless $p=|\mathcal{K}_{n}|$ is finite. Allowing $p\rightarrow\infty$ as $n\rightarrow\infty$, results in this paper build on coupling inequalities for the supremum of the empirical process developed by Chernozhukov, Chetverikov, and Kato (2014a, 2016) combined with anti-concentration inequality in Chernozhukov, Chetverikov, and Kato (2014b).

We also provide inference methods in a partially linear model setup focusing on the common parametric part. Unlike the nonparametric object of interest that has a slower convergence rate than $n^{1/2}$ (e.g., regression function or regression derivative), the t-statistics for the parametric object of interest are asymptotically equivalent for all sequences of $K$ under standard rate conditions $K/n\rightarrow 0$ as $n\rightarrow\infty$. To account for the dependency of the t-statistics with the different sequences of $K$ in this setup, we consider a faster rate of $K$ that grows as fast as the sample size $n$, as in Cattaneo, Jansson, and Newey (2018a, 2018b), and develop an asymptotic distribution of the t-statistics over $K\in\mathcal{K}_{n}$. Then, we discuss methods to construct confidence intervals that are similar to the nonparametric regression setup and provide uniform (in $K\in\mathcal{K}_{n}$) coverage properties.

We investigate finite sample coverage and length properties of the proposed CIs and uniform confidence bands in various simulation setups. As an illustrative example, we revisit nonparametric estimation of labor supply function using the entire individual piecewise-linear budget set as in Blomquist and Newey (2002). Imposing additive separability, which is derived by economic theory, Blomquist and Newey (2002) estimate the conditional mean of labor supply function using series estimation and report wage elasticity of the expected labor supply as well as other welfare measures with various specifications of the different number of series terms.

Several important papers have investigated the asymptotic properties of series (and sieve) estimators, including papers by Andrews (1991a); Eastwood and Gallant (1991); Newey (1997); Chen and Shen (1998); Huang (2003); Chen (2007); Chen and Liao (2014); Chen, Liao, and Sun (2014); Belloni, Chernozhukov, Chetverikov, and Kato (2015); and Chen and Christensen (2015), among many others. This paper extends inference based on the t-statistic under a single sequence of $K$ to the sequences of $K$ over a set $\mathcal{K}_{n}$ and focuses both on the pointwise and uniform inferences on $g_{0}(x)$, which is irregular (i.e., slower than a rate of $n^{1/2}$) and a linear functional, under an i.i.d. setup.

The supremum t-statistics have been used as a correction for multiple-testing problems and to construct simultaneous confidence bands, and the importance of multiple-testing problems (data mining or data snooping) has long been noted in various other contexts (see Leamer (1983), White (2000), Romano and Wolf (2005), Hansen (2005)).

There is also a growing literature on data-dependent series term selection and its impact on estimation and inference in econometrics and statistics. Asymptotic optimality results of cross-validation have been developed, e.g., in papers by Li (1987), Andrews (1991b), and Hansen (2015). Horowitz (2014) develops data-driven methods for choosing the sieve dimension in the nonparametric instrumental variables (NPIV) estimation such that resulting NPIV estimators attain the optimal sup-norm or $L^{2}$ norm rates adaptive to the unknown smoothness of $g_{0}(x)$. Although we do not pursue adaptive inference in this paper, there is also a large statistical literature on adaptive inference. For example, Giné and Nickl (2010), Chernozhukov, Chetverikov, and Kato (2014b) construct adaptive confidence bands in the density estimation problem (see Giné and Nickl (2015, Section 8) for comprehensive lists of references). However, once data-driven choice is obtained for adaptive estimation (e.g., Lepski (1990)-type procedures), one still requires an undersmoothing condition for inference to eliminate asymptotic bias terms (see Theorem 1 of Giné and Nickl (2010)), and this may result in similar specification search issues when choosing sufficiently “large” $K$ in practice.

We can, in principle, consider kernel-based estimation where several data-dependent bandwidth selections or explicit bias corrections have been proposed.²²2See Härdle and Linton (1994), Li and Racine (2007) for references. See also Hall and Horowitz (2013), Calonico, Cattaneo, and Farrell (2018), Schennach (2015) and references therein for various recent works on related bias issues and inference for kernel estimators. However, there exist many examples estimating $g_{0}(x)$ using (global) series estimation and imposing shape constraints easily (such as additive separability to reduce dimensionality) that are also interested in both pointwise and uniform inference. Given the issues of specification search, our paper is closely related to a recent paper by Armstrong and Kolesár (2018) which considers a bandwidth snooping adjustment for kernel-based inference.

Unlike kernel-based methods, little is known about the statistical properties of data-dependent selection rules and explicit bias formulas for general series estimation. Zhou, Shen, and Wolfe (1998) and Huang (2003) are two of the few exceptions. A recent paper, Cattaneo, Farrell, and Feng (2019), develops novel explicit asymptotic bias/integrated mean squared error (IMSE) formulas and asymptotic theory of the bias-correction methods for general partitioning-based series estimators. The results in Cattaneo, Farrel, and Feng (2019) can be used as an alternative to the undersmoothing approach to avoid specification search issues.

The remainder of the paper is organized as follows. Section 2 introduces the basic nonparametric series regression setup and the candidate set $\mathcal{K}_{n}$. Section 3 provides the pointwise inference, and Section 4 provides uniform inference in $x\in\mathcal{X}$. Section 5 extends our inference methods to the partially linear model setup. Section 6 summarizes Monte Carlo experiments in various setups, and Section 7 illustrates an empirical example as in Blomquist and Newey (2002). Then, Section 8 concludes the paper. Appendix A includes the main proofs, and Appendix B includes figures and tables. Additional supporting lemmas and simulation results are provided in the Online Supplementary Material available at Cambridge Journals Online (journals.cambridge.org/ect).

We introduce the nonparametric series regression setup in the model (1.1). Given a random sample $\{y_{i},x_{i}\}_{i=1}^{n}$, we are interested in inference on the conditional mean $g_{0}(x)=E(y_{i}|x_{i}=x)$ at a particular point $x\in\mathcal{X}\subset\mathbb{R}^{d_{x}}$ or uniform in $x\in\mathcal{X}$.

Let $\widehat{g}_{n}(K,x)$ be an estimator of $g_{0}(x)$ using $K=K_{n}\geq 1$ series terms $P(K,x)=(p_{1}(x),\cdots,p_{K}(x))^{\prime}$, which is a vector of basis functions that can change with $n$. Standard examples for the basis functions are power series, Fourier series, orthogonal polynomials, splines and wavelets. The series estimator is then obtained by the least square (LS) estimation of $y_{i}$ on regressors $P(K,x_{i})$

where $P^{K}=[P_{K1},\cdots,P_{Kn}]^{\prime},P_{Ki}\equiv P(K,x_{i})=(p_{1}(x_{i}),p_{2}(x_{i}),\cdots,p_{K}(x_{i}))^{\prime},Y=(y_{1},\cdots y_{n})^{\prime}$. Define the least square residuals as $\widehat{\varepsilon}_{Ki}=y_{i}-P_{Ki}^{\prime}\widehat{\beta}_{K}$,

and consider the t-statistic

Under standard regularity conditions (discussed in the next section), the t-statistic can be decomposed as follows:

where $Q_{K}=E(P_{Ki}P_{Ki}^{\prime})$, $r_{n}(K,x)=g_{0}(x)-P(K,x)^{\prime}\beta_{K}$, and $\beta_{K}\equiv E[P_{Ki}P_{Ki}^{\prime}])^{-1}E[P_{Ki}y_{i}]$ is the best linear $L^{2}$ projection coefficient. The first term in the decomposition (2.4) converges to a standard normal distribution for the deterministic sequence $K\rightarrow\infty$ as $n\rightarrow\infty$, and the second term does not necessarily converge to 0 due to approximation errors $r_{n}(K,x)$. The second term can be ignored with an undersmoothing assumption, and the asymptotic distribution of the t-statistic, $\widehat{T}_{n}(K,x)\overset{d}{\longrightarrow}N(0,1)$, is well known in the literature (see, for examples, Andrews (1991a), Newey (1997), Belloni et al. (2015), and Chen and Christensen (2015), among many others). Then, the $100(1-\alpha)\%$ confidence interval for $g_{0}(x)$ can be easily constructed using the normal critical value $z_{1-\alpha/2}$

However, it is not clear whether the conventional CI using normal critical values (2.5) has a correct coverage probability with a possibly data-dependent $\widehat{K}$ such as cross-validation or IMSE-optimal selection. First, $T_{n}({\widehat{K}},\theta_{0})\overset{d}{\rightarrow}N(0,1)$ may not hold with a random sequence of $\widehat{K}$, even if we assume the asymptotic bias is negligible. Second, it is well known that some data-dependent rules $\widehat{K}$ do not satisfy the undersmoothing rate conditions, which can lead to a large asymptotic bias and coverage distortion of the standard CI. For example, suppose that the researcher uses $\widehat{K}=\widehat{K}_{\texttt{cv}}$ selected by cross-validation; then, $\widehat{K}_{\texttt{cv}}$ is typically too “small” and violates the undersmoothing assumption needed to ensure the asymptotic normality without bias terms and the valid inference.

As discussed in the introduction, the undersmoothing assumption involves possibly ad-hoc methods to choose series terms $K$ over a candidate set $\mathcal{K}_{n}$ for a valid inference, and cross-validation methods naturally involve specification search over a set of the different number of series terms.

The following set assumption on $\mathcal{K}_{n}$ is constructed to allow a broad range of $K$ such that $\mathcal{K}_{n}$ can allow (unknown) an optimal MSE rate of $K$ as well as an undersmoothing rate that increases faster than the optimal MSE rate.

Here, we consider a possibly growing set of the number of series terms, and a similar assumption is used in the literature, for example, in Newey (1994a, 1994b). Suppose $g_{0}(x)$ belongs to the Hölder space of smoothness $s>0$, $\Sigma(s,\mathcal{X})$; then, we obtain optimal $L^{2}$ convergence rates $O_{p}(n^{-s/(2s+d_{x})})$ with $K\asymp n^{d_{x}/(d_{x}+2s)}$. Assumption 2.1 allows having optimal $L^{2}$ rates of $K$ in a large set of classes of functions. By setting $\mathcal{K}_{n}=[\underline{K},\overline{K}]\cap\mathbb{N}$, $\overline{K}\asymp n^{\overline{\phi}}$ and $\underline{K}\asymp n^{\underline{\phi}}$ with $\overline{\phi}=d_{x}/(d_{x}+2\underline{s}),\underline{\phi}=d_{x}/(d_{x}+2\overline{s})$, Assumption 2.1 contains the number of series terms that obtain an optimal $L^{2}$ rate of convergence for $g_{0}(x)\in\bigcup_{s\in S}\Sigma(s,\mathcal{X})$, $S=[\underline{s},\overline{s}]$. A similar assumption is used in the literature on adaptive inference, although we do not pursue this direction in the current paper.

Assumption 2.1 gives flexible choices of $K$, as we only assume the rates of $K$, for example, $\overline{K}=Cn^{\overline{\phi}},\underline{K}=cn^{\underline{\phi}}$, where $c$ and $C$ can be set arbitrarily small or large. We only require rate restrictions uniformly over $K\in\mathcal{K}$ to guarantee the linearization of the t-statistic in (2.4) and the rates of the cardinality $p=|\mathcal{K}_{n}|$. Since $K\in\mathcal{K}_{n}$ is a positive integer and $p\leq\overline{K}$, $p$ is growing at a rate much slower than $n$ under the rate restrictions in Section 3.

In this section, we focus on pointwise inference for $g_{0}(x)$. The goal of this section is to provide a uniform distributional approximation theory of $\widehat{T}_{n}(K,x)$ over a set $\mathcal{K}_{n}$ and provide uniform (in $K\in\mathcal{K}_{n}$) coverage properties of confidence intervals for $g_{0}(x)$ in (1.2), (1.3) with the construction of critical values.

From the decomposition of the t-statistic in (2.4), we first consider the (infeasible) test statistic

where $t_{n}(K,x)=n^{-1/2}\sum_{i=1}^{n}P(K,x)^{\prime}Q_{K}^{-1}P_{Ki}\varepsilon_{i}/V_{n}(K,x)^{1/2}$ with the series variance $V_{n}(K,x)=P(K,x)^{\prime}Q_{K}^{-1}\Omega_{K}Q_{K}^{-1}P(K,x)$, $\Omega_{K}=E(P_{Ki}P_{Ki}^{\prime}\varepsilon_{i}^{2})$. In general, $t_{n}(K,x),K\in\mathcal{K}_{n}$ does not have a limiting distribution because it is not asymptotically tight under Assumption 2.1 unless $|\mathcal{K}_{n}|$ is finite or under the restrictive assumption on $\mathcal{K}_{n}$.⁴⁴4In an earlier version of the paper, we provide the weak convergence of a series process under the same rates of $K\in\mathcal{K}_{n}$ and high-level assumptions. This can be viewed as an analogous result in the kernel estimation literature (see Section 2 of Armstrong and Kolesár (2018) and other references therein). However, we show below that there exists a sequence of random variables $\max_{1\leq j\leq p}\sum_{i=1}^{n}|Z_{ij}|$ such that $\big{|}\max_{K\in\mathcal{K}_{n}}|t_{n}(K,x)|-\max_{1\leq j\leq p}\sum_{i=1}^{n}|Z_{ij}|\big{|}=O_{p}(a_{n})$ for a sequence of constants $a_{n}\rightarrow 0$, where $Z_{i}=(Z_{i1},...,Z_{ip})^{\prime}$ is a Gaussian random vector in $\mathbb{R}^{p}$ such that $Z_{i}\sim N(0,\frac{1}{n}\Sigma_{n})$ with $(j,l)$ elements of the variance-covariance matrix

$\Omega_{K_{j},K_{l}}=E(P_{K_{j}i}P_{K_{l}i}^{\prime}\varepsilon_{i}^{2})$.

By replacing unknown $\Sigma_{n},V_{n}(K,x)$ with consistent estimators $\widehat{\Sigma}_{n},\widehat{V}_{n}(K,x)$, we show below that we can approximate $\max_{K\in\mathcal{K}_{n}}|\widehat{T}_{n}(K,x)|$ by $\max_{1\leq j\leq p}\sum_{i=1}^{n}|Z_{ij}|$ and then obtain critical values by using a simulation-based method to provide valid coverage properties in (1.2) and (1.3). We define $\widehat{c}_{1-\alpha}(x)$ as follows:

where $\widehat{\Sigma}_{n}$ is a consistent estimator of the variance-covariance matrix $\Sigma_{n}$ defined in (3.2), $\widehat{V}_{n}(K,x)$ is the simple plug-in estimator for $V_{n}(K,x)$ as in (2.2), and $\widehat{\varepsilon}_{Ki}=y_{i}-P_{Ki}^{\prime}\widehat{\beta}_{K},\forall K\in\mathcal{K}_{n}$. One can compute $\widehat{c}_{1-\alpha}(x)$ by simulating $B$ (typically $B=1000$ or $5000$) i.i.d. random vectors $\widehat{Z}_{i}^{b}\sim N(0,\frac{1}{n}\widehat{\Sigma}_{n})$ and by taking a $(1-\alpha)$ sample quantile of $\{\max\limits_{1\leq j\leq p}\sum_{i=1}^{n}|\widehat{Z}_{ij}^{b}|:b=1,\cdots,B\}$. Alternatively, we can use weighted bootstrap methods. See Section 4 for the implementation and the validity of bootstrap procedures in the construction of confidence bands.

To establish our main results, we impose mild regularity conditions uniform in $K\in\mathcal{K}_{n}$. For each $K\in\mathcal{K}_{n}$, define $\zeta_{K}\equiv\sup_{x\in\mathcal{X}}||P(K,x)||$ as the largest normalized length of the regressor vector and $\lambda_{K}\equiv(\lambda_{min}(Q_{K}))^{-1/2}$ for $K\times K$ design matrix $Q_{K}=E(P_{Ki}P_{Ki}^{\prime})$.

Assumptions 3.1(ii) and 3.2(i) are similar to those imposed in Belloni et al. (2015) and Chen and Christensen (2015), and all the discussions made there also apply here except that we impose rate conditions of $K$ uniformly over $\mathcal{K}_{n}$. The rate conditions can be replaced by the specific bounds of $\zeta_{K},c_{K},\ell_{K}$ with various sieve bases. For example, when $\mathcal{X}=[0,1]^{d_{x}}$, the probability density of $x_{i}$ is uniformly bounded above and bounded away from zero, and $g_{0}(x)\in\Sigma(s,\mathcal{X})$, i.e., the Hölder space of smoothness $s>0$, then $\lambda_{K}\lesssim 1$, $\zeta_{K}\lesssim\sqrt{K}$, $\ell_{K}c_{K}\lesssim K^{-(s\wedge s_{0})/d_{x}}$ for regression spline series of order $s_{0}$, and Assumption 3.2(i) is satisfied when $\sqrt{\overline{K}(\log^{3}\overline{K})/n}(1+\overline{K}^{1/2}\underline{K}^{-(s\wedge s_{0})/d_{x}})+\underline{K}^{-(s\wedge s_{0})/d_{x}}\log\overline{K}\rightarrow 0$. Other standard regularity conditions in the literature (e.g., Newey (1997) and Chen (2007)) can also be used here, and the rate condition can be improved with different pointwise linearization and approximation bounds in Huang (2003) for splines and Cattaneo et al. (2019) for partitioning-based estimators.

Assumption 3.2(ii) imposes either the bounded polynomial moment conditions or sub-exponential moments of the regression errors. Assumption 3.2(iii) imposes the consistency of variance estimator $\widehat{V}_{n}(K,x)$ uniformly in $K\in\mathcal{K}_{n}$, and this holds under mild regularity conditions (see Lemma 5.1 of Belloni et al. (2015) and Lemma 3.1-3.2 of Chen and Christensen (2015)).

Theorem 3.1 provides a uniform coverage property of the confidence interval over $K\in\mathcal{K}_{n}$ for the regression function $g_{0}(x)$. Equation (3.6) guarantees the asymptotic coverage of CI for data-dependent $\widehat{K}\in\mathcal{K}_{n}$ with undersmoothing. Note that standard inference methods in the nonparametric regression setup typically consider a singleton set $\mathcal{K}_{n}=\{K\}$ with $K\rightarrow\infty$ as $n\rightarrow\infty$. The rate restriction is mild because it only requires $\overline{K}/n^{1-2/q}\rightarrow 0$, up to $\log n$ terms, in case $(a)$ and $\overline{K}/n\rightarrow 0$, up to $\log n$ terms, in case $(b)$ when $\zeta_{K}\lesssim\sqrt{K}$ for splines and wavelet series. Theorem 3.1 builds upon a coupling inequality for maxima of sums of random vectors in Chernozhukov, Chetverikov, and Kato (2014a) combined with the anti-concentration inequality in Chernozukhov, Chetverikov, and Kato (2014b).

This section provides construction of uniform confidence bands for $g_{0}(x)$ (uniform in $K\in\mathcal{K}_{n}$) given in (1.4). We define the following empirical process

over $\mathcal{K}_{n}\times\mathcal{X}$, and we show below that the supremum of the empirical process $\sup_{(K,x)\in\mathcal{K}_{n}\times\mathcal{X}}|\widehat{T}_{n}(K,x)|$ can be approximated by a sequence of random variables $\sup_{(K,x)\in\mathcal{K}_{n}\times\mathcal{X}}|Z_{n}(K,x)|$, where $Z_{n}(K,x)$ is a tight Gaussian random process in $\ell^{\infty}(\mathcal{K}_{n}\times\mathcal{X})$ with zero mean and covariance function

Although the Gaussian approximation is an important first step, the covariance function (4.2) is generally difficult to construct for the purpose of uniform inference. Thus, we employ weighted bootstrap methods similar to Belloni et al. (2015) and show the validity of the bootstrap procedure for uniform confidence bands.

Let $e_{1},...,e_{n}$ be a sequence of i.i.d. standard exponential random variables that are independent of $X^{n}=\{x_{1},...,x_{n}\}$. For $(K,x)\in\mathcal{K}_{n}\times\mathcal{X}$, we define a (centered) weighted bootstrap process

where $\widehat{g}_{n}^{e}(K,x)=P(K,x)^{\prime}\widehat{\beta}_{K}^{e}$, and $\widehat{\beta}_{K}^{e}$ is obtained by the following weighted least squares regression

Define the critical value

and we consider confidence bands of the form

To provide the validity of the bootstrap critical values and confidence bands in (4.6), we show below that the conditional distribution of $\sup_{(K,x)\in\mathcal{K}_{n}\times\mathcal{X}}|\widehat{T}_{n}^{e}(K,x)|$ is “close” to the distribution of $\sup_{(K,x)\in\mathcal{K}_{n}\times\mathcal{X}}|Z_{n}(K,x)|$ and that of $\sup_{(K,x)\in\mathcal{K}_{n}\times\mathcal{X}}|\widehat{T}_{n}(K,x)|$ using coupling inequalities for the supremum of the empirical process and the bootstrap process as in Chernozhukov et al. (2016). Then, similar to Theorem 3.1, this gives bounds on the Kolmogorov distance for the distribution functions of $P(\sup_{K\in\mathcal{K}_{n},x\in\mathcal{X}}|\widehat{T}_{n}(K,x)|\leq u)$ and $P(\sup_{K\in\mathcal{K}_{n},x\in\mathcal{X}}|\widehat{T}_{n}^{e}(K,x)|\leq u|X^{n})$.

The following assumptions are used to establish the coverage probability of confidence bands uniformly over $K\in\mathcal{K}_{n}$. Define $\alpha(K,x)\equiv Q_{K}^{-1/2}P(K,x)/V_{n}(K,x)^{1/2}$, and