Self-convolved Bootstrap for M-regression under Complex Temporal Dynamics

There has been an increasing necessity to account for temporal non-stationarity in finance and economics. For instance, financial returns are frequently non-stationary (Francq and Sucarrat, 2023). In climate economics, climate change has become progressively unpredictable with extreme weather happening globally with greater intensity (Ebi et al., 2021). See also Dette et al. (2011); Vogt (2012); Nason (2013); Kreiss and Paparoditis (2014); Dahlhaus et al. (2019) among others for the recent literature on non-stationary time series analysis. As deviations from temporal homogeneity become more pronounced, static models may fail to capture the dynamic features of the data. In the context of regression analysis, one successful generalization is to use the varying-coefficient model (Hastie and Tibshirani, 1993), which is capable of describing the non-constant relationship between the predictors and the response (Hoover et al., 1998; Fan and Zhang, 2000; Karmakar et al., 2022). In the present work, we shall further consider the following time-varying M-regression (Huber, 1964, 1973) model for non-stationary time series:

where $\{{\bf x}_{i}=(x_{i1},\cdots,x_{ip})^{\top}\}_{i=1}^{n}$ is a $p$-dimensional predictor process, $\{e_{i}\}_{i=1}^{n}$ is the error process satisfying $\mathbb{E}(\psi(e_{i})|{\bf x}_{i},{\bf x}_{i-1},\cdots,{\bf x}_{1})=0$ almost surely with $\psi(\cdot)$ being the left derivative of a convex function $\rho(\cdot)$ , and $\mbox{\boldmath$\beta$}_{i}=\mbox{\boldmath$\beta$}(t_{i})=$ $(\beta_{0}(t_{i}),\cdots,\beta_{p-1}(t_{i}))^{\top}$ is the coefficient $\mbox{\boldmath$\beta$}(t)$ evaluated at time $t_{i}=i/n$. We study model (1) in a general framework where $\{{\bf x}_{i}\}$ and $\{e_{i}\}$ belong to a general class of non-stationary time series models with both smoothly and abruptly time-varying dynamics. The detailed assumptions of the model are deferred to Section 7. Observe that (1) includes a wide range of regression models with flexible choices of $\rho(\cdot)$. Prominent examples include the least square regression with $\rho(x)=x^{2}$, the quantile regression with $\rho(x)=\tau x^{+}+(1-\tau)(-x)^{+}$, $\tau\in(0,1)$, $x^{+}=\max(x,0)$, the $L^{q}$ regression with $\rho(x)=|x|^{q}$, $q>0$, the expectile regression with $\rho(x)=|I(x\leq 0)-\alpha|x^{2}$, $\alpha\in(0,1)$, and Huber’s robust regression with $\rho(x)=x^{2}I(|x|\leq\varsigma)/2+(\varsigma|x|-\varsigma^{2}/2)I(|x|>\varsigma)$, $\varsigma>0$.

This article aims to provide a unified framework for nonparametric inference of model (1). Due to the complex temporal dynamics and the broad choice of loss functions, it is difficult to directly estimate the limiting distributions of the M-estimators, in which case resampling methods such as the bootstrap is typically adopted to circumvent the difficulty. In the context of nonparametric M-regression, however, so far there exist no valid bootstrap methods for its simultaneous inference when the regressors and errors are non-stationary time series with possible jumps in the underlying data generating mechanism. The most related work can be found in Wu and Zhou (2017), who explored time-varying coefficient quantile regression with smoothly time-varying dynamics. We also refer to Friedrich and Lin (2022) who proposed an autoregressive sieve bootstrap framework, valid under $L_{2}$ regression with strictly stationary $\alpha$-mixing predictors. Extending to our current setup, the primary challenge in designing bootstrap methods originates from the inconsistency in estimating various key quantities around the jump points. Additionally, bootstrap inference for M-regression involves estimating a density-like function, which requires the delicate choice of a tuning parameter (Koenker, 2005). For instance, in quantile regression where $\rho(x)=\tau x^{+}+(1-\tau)(-x)^{+}$, the asymptotic behavior of local estimates of $\mbox{\boldmath$\beta$}(\cdot)$ is tied to a density-type quantity $\Sigma(\cdot)$ (Wu and Zhou, 2017). In moderate samples, the quality of the bootstrap inference depends on the estimation accuracy of $\Sigma(\cdot)$, which substantially entails the nontrivial task of selecting the tuning parameter. Lastly, the existing literature on bootstrap inference of nonparametric M-regression, such as Friedrich and Lin (2022) and Wu and Zhou (2017), often constructs the bootstrap based on residuals. In moderate samples, the performance of such residual-based bootstrap can be significantly hampered by the estimation error of the residuals as estimators of the true regression errors.

The methodological innovation of this paper lies in proposing a novel self-convolved bootstrap for the simultaneous nonparametric inference of Model (1), which shares a simple and unified construction among a broad class of M-estimators and hypothesis tests, and is consistent under very general forms of non-stationary temporal dependence and predictor-error dependence. As its name implies, the self-convolved bootstrap only necessitates the convolution of the M-estimators with i.i.d. auxiliary standard normal variables, eliminating the need for additional computation. Particularly, it avoids estimating the aforementioned density-like quantities or residuals. The construction principle applies uniformly to a wide class of M-estimators and therefore saves lots of effort when conducting inference based on various estimators simultaneously. The validity of the proposed bootstrap arises from two main observations. Firstly, by taking the second-order difference of the local M-estimators as shown in Section 3, their progressive sum can be well approximated by a weighted block sum of the data through a uniform Bahadur representation. Secondly, the weighted block sum convolved by i.i.d. auxiliary standard normal variables takes the form of a robust multiplier bootstrap (Zhou, 2013), which can consistently mimic the probabilistic behavior of their partial sum processes. We will show that the easy-to-implement bootstrap simulates the probabilistic behavior of the M-estimators consistently under complex temporal dynamics. Compared to the bootstrap methods developed in Karmakar et al. (2022), Wu and Zhou (2017) and Friedrich and Lin (2022), the self-convolved bootstrap enjoys broader applicability yet maintains a simple implementation.

Facilitating the self-convolved bootstrap, our framework utilizes the cumulative regression function (CRF)

instead of $\mbox{\boldmath$\beta$}(t)$ for inference, driven by two main reasons. First, as is suggested in Theorem 1, the CRF can be estimated at a parametric rate of $\sqrt{n}$. Such merits of getting $\sqrt{n}$ convergence via integration have been demonstrated by Hall and Marron (1987), Bickel and Ritov (1988), Huang and Fan (1999) and many others. Recently, Mies (2023) employed the method to estimate parameters of locally stationary time series, leading to a $\sqrt{n}$ functional central limit theorem. Cai and Juhl (2023) also obtained a similar result regarding rolling regression estimators, where the error process is assumed to be a martingale difference. Although the results in the existing literature have gradually adapted from the i.i.d. case to a broader non-stationary time series setup, they generally rely on a closed-form statistic, e.g., the least square estimator, and therefore do not apply to the M-regression setting. To our knowledge, no comparable research has been conducted on uniform nonparametric inference of the CRF under the time-varying M-regression framework. The second reason, possibly more salient, pertains to the complexity of the temporal dynamics considered. Specifically, it is difficult to consistently estimate the (asymptotic) distributional behavior of the M-estimators around the time points where ${\bf x}_{i}$ and/or $e_{i}$ experience abrupt changes. As a result, directly inferring the M-estimators of $\mbox{\boldmath$\beta$}(\cdot)$ simultaneously over time becomes challenging. Nevertheless, we discover that aggregated M-estimators of the CRF converge to a Gaussian process uniformly over time under mild conditions, which can be effectively approximated by the self-convolved bootstrap mentioned earlier. Consequently, it is simpler and more accurate to make simultaneous inferences of $\Lambda(\cdot)$ than $\mbox{\boldmath$\beta$}(\cdot)$ under complex temporal dynamics. As far as we know, no previous literature has used the method of integration or aggregation to alleviate the effect of complex dependence and heteroscedasticity in regression analysis.

The self-convolved bootstrap facilitated by the CRF addresses different types of tests using a unified asymptotic result, which ventures into a new avenue compared to the conventional methods. To elaborate, we consider three types of hypothesis tests: Exact Function Tests, Lack-of-fit Tests, and Qualitative Tests. The first type, Exact Function Test (EFT), takes the form of $H_{0}:\mbox{\boldmath$\beta$}(t)=\mbox{\boldmath$\beta$}_{0}(t)$ for some specific $\mbox{\boldmath$\beta$}_{0}(t)$, which, for example, enables one to check the significance of the variables by letting $\mbox{\boldmath$\beta$}_{0}(t)=\mathbf{0}$. Secondly, the Lack-of-fit Test (LOFT), also referred to as diagnostic test (Stute, 1997), tackles the circumstance where the null hypothesis is a parametric family while the alternative remains nonparametric (He and Zhu, 2003). This includes, but is not limited to, the tests of constancy and linearity of the function versus a general smooth alternative. An example in the realm of econometrics is (Chen and Hong, 2012), where the LOFT is used to tackle the long-standing problem of detecting structural changes in economic relationships. Apart from the aforementioned two types of tests, one may also want to check qualitative beliefs on functions where the null hypothesis is a relatively general nonparametric class of functions (Komarova and Hidalgo, 2020). For example, many economic models use certain shape restrictions (e.g., monotonicity, convexity) as plausible restrictions (Chetverikov, 2019; Fang and Seo, 2021). While those prespecified qualitative shape assumptions facilitate the inference and enhance performance (Friedman and Tibshirani, 1984; Matzkin, 1994), wrong conclusions can be drawn when these assumptions are not satisfied. The necessity to conduct Qualitative Tests (QT), therefore, becomes axiomatic.

Historically in the context of regression, much literature has focused on the Lack-of-fit Test on the regression mean function, serving as a key component in model checking. One popular approach is to estimate the parametric and nonparametric curves separately and compute their discrepancies as a natural test statistic, as seen in Hardle and Mammen (1993) and Hong and White (1995). Similarly, Chen and Hong (2012) proposed a generalized Hausman test for checking parameter stability in time series models, applicable to stationary data under a $L_{2}$ framework. Alternatively, another distinguished direction is to study the empirical process based on residuals; see Stute et al. (1998); Koul and Stute (1999). Recently, Mies (2023) utilized the integration technique for change-point detection of local parameters. In comparison, our current work is based on a new self-convolved bootstrap technique and applies to a broader range of tests and a more general M-regression setting. Different from the LOFT where the null hypothesis forms a parametric family, qualitative hypotheses often include shape constraints that are more complicated to deal with. To test the monotonicity of a regression curve, Bowman et al. (1998) exploited Silverman (1981)’s idea of critical bandwidth. Gijbels et al. (2000) and Ghosal et al. (2000) formulated monotonicity of the regression curve as the concordance of $X$ and $Y$. As a distinctive approach, Durot (2003) transformed the test of monotonicity into testing whether the integral coincides with its least concave majorant. A localized version of the test was later elaborated by Akakpo et al. (2014). Other contributions in nonparametric qualitative tests include Hall and Heckman (2000); I. Gijbels and Verhasselt (2017); Komarova and Hidalgo (2020); Fang and Seo (2021), among others.

The rest of the paper is structured as follows. Section 2 introduces the piece-wise locally stationary time series, dependence measures, and local linear M-estimators. The estimator of the CRF is proposed, with its asymptotic results stated in Theorem 1. Section 3 presents the self-convolved bootstrap. In Section 4, three types of hypothesis tests are discussed, where detailed algorithms and their asymptotic behaviors are provided. The finite-sample performance of our method is demonstrated in Section 5, followed by empirical illustrations using global climate data and Microsoft stock return in Section 6. Regularity conditions of the model and auxiliary theoretical results about Bahadur representation are given in Section 7. Detailed proofs are provided in the supplementary material.

We start by introducing some notations. For an $m$-dimensional (random) vector $\boldsymbol{v}=(v_{1},\cdots,v_{m}),m\geq 1$, let $|v|$ be the Euclidean norm and $|v|_{\infty}:=\max_{i=1}^{m}|v_{i}|$. Let $I\{\cdot\}$ be the indicator function and we denote the weak convergence by $\Rightarrow$.

The Gaussian process $\mathbf{U}(t)$ expressed in Theorem 1 provides a theoretical foundation for the simultaneous inference of $\Lambda(t)$, but the implementation is still unaccomplished due to its complex covariance structure. To circumvent the problem, we propose a self-convolved bootstrap method to mimic the behavior of the Gaussian process $\mathbf{U}(t)$.

Let $\mu=\int_{-2}^{2}[K^{*}(t-1)+K^{*}(t+1)-2K^{*}(t)]^{2}\;\mathrm{d}t$, $\{R_{i}\}_{i=1}^{n}$ be i.i.d. standard normal random variables independent of $\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{n}$. For a given bandwidth $c_{n}$, define the bootstrap process $\{\tilde{\boldsymbol{\Phi}}_{n}^{o}(t),t\in(0,1)\}$ as the linear interpolation of $\{\boldsymbol{\Phi}_{n,j}^{o}\}$, where

The self-convolved bootstrap $\tilde{\boldsymbol{\Phi}}_{n}^{o}(t)$ only requires the convolution of local M-regression estimates with i.i.d. standard normal random variables $\{R_{i}\}_{i=1}^{n}$. The heuristics stem from two fundamental observations. Firstly, the Bahadur representation of $\{\check{\mbox{\boldmath$\beta$}}(t_{i})\}$ in Section 7 suggests that $\check{\mbox{\boldmath$\beta$}}_{c_{n}}(t_{i}+c_{n})+\check{\mbox{\boldmath$\beta$}}_{c_{n}}(t_{i}-c_{n})-2\check{\mbox{\boldmath$\beta$}}_{c_{n}}(t_{i})$ can be expressed as a weighted block sum of $\{\mathbf{x}_{i}\psi(e_{i})\}_{i=1}^{n}$, where we take the second-order difference as a bias-correction technique. Secondly, Zhou (2013) demonstrated that for a broad range of non-stationary time series, progressive convolutions of their block sums with i.i.d. standard normal random variables can consistently mimic the joint probabilistic behavior of their partial sum processes. Observe that $\mathbf{U}(t)$ represents the limiting behavior of a weighted partial sum process of $\{\mathbf{x}_{i}\psi(e_{i})\}_{i=1}^{n}$. By leveraging these observations, the self-convolved bootstrap achieves an accurate simulation of the limiting Gaussian process of the CRF and retains a concise form for a broad range of M-estimators, utilizing just a single tuning parameter.

In contrast, it is difficult to make inference of $\mbox{\boldmath$\beta$}(t)$ directly based on $\check{\mbox{\boldmath$\beta$}}_{b_{n}}(t)$, as the density-like quantity involved in the distributional behavior of $\check{\mbox{\boldmath$\beta$}}_{b_{n}}(t)$ is hard to estimate in practice due to possible discontinuities and the no-trivial task of choosing smoothing parameters. As a result, the CRF considered in this paper, combined with the proposed self-convolved bootstrap, enables us to conduct inference more efficiently and accurately.

To quantify the consistency of $\{\tilde{\boldsymbol{\Phi}}_{n}^{o}(t),t\in(0,1)\}$, we derive a direct comparison of the distribution between $\{\boldsymbol{\Phi}_{n,j}^{o}\}_{j=\lceil 2nc_{n}\rceil}^{n-\lceil 2nc_{n}\rceil}$ and $\{\mathbf{U}(t_{j})\}{{}_{j=\lceil 2nc_{n}\rceil}^{n-\lceil 2nc_{n}\rceil}}$ in the following theorem. First, we define the $ns$-dimensional vector $\mathbf{\Theta}^{B}=[(\mathbf{\Theta}_{1}^{B})^{\top},(\mathbf{\Theta}_{2}^{B})^{\top},\cdots,(\mathbf{\Theta}_{n}^{B})^{\top}]$ with

and similarly define the $ns$-dimensional vector $\mathbf{U}=[(\mathbf{U}_{1})^{\top},(\mathbf{U}_{2})^{\top},\cdots,(\mathbf{U}_{n})^{\top}]$ with

Recall $\mathbf{U}(t)$ is defined in Theorem 1. Furthermore, we define

as a measure of the difference in covariance structure between the bootstrapped process and the limiting Gaussian process.

In principle, hypotheses regarding the regression function $\mbox{\boldmath$\beta$}(t)$ can be expressed equivalently in terms of the CRF $\Lambda(t)$. In this Section, we shall explore the application of the CRF and the self-convolved bootstrap to three general classes of hypothesis tests of the M-regression: Exact Function Test (EFT), Lack-of-fit Test (LOFT), and Qualitative Test (QT).