This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

A short note on model selection by LASSO methods in a change-point model

Fuqi Chen University of Windsor, 401 Sunset Avenue, Windsor, Ontario, N9B 3P4. Email: chen111n@uwindsor.ca    and   Sévérien Nkurunziza University of Windsor, 401 Sunset Avenue, Windsor, Ontario, N9B 3P4. Email: severien@uwindsor.ca
Abstract

In Ciuperca (2012) (Ciuperca. Model selection by LASSO methods in a change-point model, Stat. Papers, 2012;(in press)), the author considered a linear regression model with multiple change-points occurring at unknown times. In particular, the author studied the asymptotic properties of the LASSO-type and of the adaptive LASSO estimators. While the established results seem interesting, we point out some major errors in proof of the most important result of the quoted paper. Further, we present a corrected result and proof.

Keywords: Asymptotic properties; Change-points; Model selection; LASSO; Regression.

1 Introduction

In Ciuperca (2012), the author considered a linear regression model with multiple change-points occurring at unknown times. In particular, the author studied the asymptotic properties of the LASSO-type and that of the adaptive LASSO estimators. While the established results seem interesting, we point out a major error in proof of one of the important result. In particular, the proof of Part (ii) of Lemma 3 in Ciuperca (2011) is based on the inequality |a2−b2|⩽(a−b)2|a^{2}-b^{2}|\leqslant(a-b)^{2}, which is wrong. Indeed, take a=2a=2 and b=1b=1, we get |a2−b2|=3>(2−1)2=1|a^{2}-b^{2}|=3>(2-1)^{2}=1 which contradicts the inequality used in the quoted paper.

For the sake of clarity, we use the same notation and we suppose that the main assumptions in Ciuperca (2012) hold. Below, we recall these assumptions for the convenience of the reader. Namely, we consider the following model: Yi=fθ​(Xi)+εiY_{i}=f_{\theta}(X_{i})+\varepsilon_{i}, where

fθ​(Xi)=Xi′​ϕ1​𝕀{i<l1}+Xi′​ϕ2​𝕀{l1≤i<l2}+…+Xi′​ϕK+1​𝕀{i>lK},i=1,…​n,f_{\theta}(X_{i})=X^{\prime}_{i}\phi_{1}\mathbb{\mathbb{I}}_{\left\{i<l_{1}\right\}}+X^{\prime}_{i}\phi_{2}\mathbf{\mathbb{I}}_{\left\{l_{1}\leq i<l_{2}\right\}}+...+X^{\prime}_{i}\phi_{K+1}\mathbf{\mathbb{I}}_{\left\{i>l_{K}\right\}},\quad i=1,...n,

𝕀A\mathbb{\mathbb{I}}_{A} denotes the indicator function of the event AA, YiY_{i} denotes the response variable, XiX_{i} is a pp-vector of regressors, (εi)1⩽i⩽n(\varepsilon_{i})_{1\leqslant i\leqslant n} are the errors which are assumed to be independent and identically distributed (i.i.d.) random variables, ϕi∈Γ⊂ℝp\phi_{i}\in\Gamma\subset\mathbb{R}^{p}, Γ\Gamma is compact, i=1,2,…,Ki=1,2,\dots,K. The model parameters are given by θ=(θ1,θ2)\theta=(\theta_{1},\theta_{2}), with the regression parameters θ1=(ϕ1,…​ϕk+1)\theta_{1}=(\phi_{1},...\phi_{k+1}) and the change-points θ2=(l1,…,lk)\theta_{2}=(l_{1},...,l_{k}). In addition, we set θ10=(ϕ10,…​ϕk+10)\theta_{1}^{0}=(\phi_{1}^{0},...\phi_{k+1}^{0}) and θ20=(l10,…,lk0)\theta_{2}^{0}=(l_{1}^{0},...,l_{k}^{0}) to be the true values of θ1\theta_{1} and θ2\theta_{2}, respectively. As in Ciuperca (2012), we impose the following conditions.

Main Assumptions

(𝑯𝟏)\bm{(H_{1})}

There exists two positive constants u,c0(>0)u,c_{0}(>0) such that lr+1−lr⩾c0​[nu]l_{r+1}-l_{r}\geqslant c_{0}[n^{u}], for every r∈(1,…,K)r\in(1,...,K), with l0=1l_{0}=1 and lK+1=nl_{K+1}=n. Without loss of generality, we consider 3/4⩽u⩽13/4\leqslant u\leqslant 1, and c0=1c_{0}=1.

(𝑯𝟐)(\bm{H_{2}})

n−1​max1⩽i⩽n⁡(Xi′​Xi)→n→∞𝟎n^{-1}\,\displaystyle{\max_{1\leqslant i\leqslant n}}(X^{\prime}_{i}X_{i})\xrightarrow[n\rightarrow\infty]{}\bm{0} and for any r=1,…,K+1r=1,...,K+1, the matrix
Cn,r≡(lr−lr−1)​∑i=lr−1+1lrXi​Xi′→n→∞CrC_{n,r}\equiv(l_{r}-l_{r-1})\displaystyle{\sum_{i=l_{r-1}+1}^{l_{r}}}X_{i}X^{\prime}_{i}\xrightarrow[n\rightarrow\infty]{}C_{r}, where CrC_{r} is a non-negative definite matrix.

(𝑯𝟑)(\bm{H_{3}})

ε\varepsilon is a random variable absolutely continuous with E​(εi)=0\textrm{E}(\varepsilon_{i})=0, E​(εi2)=σ2\textrm{E}(\varepsilon_{i}^{2})=\sigma^{2}, i=1,2,…,ni=1,2,\dots,n.

We assume that ϕr≠ϕr+1\phi_{r}\neq\phi_{r+1}, r=1,…,kr=1,...,k, and consider the following penalized sum:

S​(l1,…,lk)=∑r=1k+1[infϕr∑i=lr−1+1lr(((Yi−Xi​ϕr)2)+λn,(lr−1,lr)lr−lr−1​∑u=1P|ϕr,u|γ)],S(l_{1},...,l_{k})=\sum_{r=1}^{k+1}\Big{[}\inf_{\phi_{r}}\sum_{i=l_{r-1}+1}^{l_{r}}\Big{(}((Y_{i}-X_{i}\phi_{r})^{2})+\frac{\lambda_{n,(l_{r-1},l_{r})}}{l_{r}-l_{r-1}}\sum_{u=1}^{P}|\phi_{r,u}|^{\gamma}\Big{)}\Big{]},

where λn,(lr−1,lr)=O​(lr−lr−1)1/2\lambda_{n,(l_{r-1},l_{r})}=O(l_{r}-l_{r-1})^{1/2} is the tuning parameter and γ>0\gamma>0. We define the LASSO-type estimator of (θ10,θ20)(\theta_{1}^{0},\theta_{2}^{0}), say (θ^1s,θ^2s)(\hat{\theta}_{1}^{s},\hat{\theta}_{2}^{s}), where θ^1s=(l^1s,…,l^ks)\hat{\theta}_{1}^{s}=(\hat{l}_{1}^{s},...,\hat{l}_{k}^{s}) and θ^2s=(ϕ^1s,…,ϕ^k+1s)\hat{\theta}_{2}^{s}=(\hat{\phi}_{1}^{s},...,\hat{\phi}_{k+1}^{s}), by

ϕ^rs=argminϕr∑i=lr−1+1lr((Yi−Xiϕr)2)+λn,(lr−1,lr)lr−lr−1∑u=1P|ϕr,u|γ),∀r=1,…,k+1,\hat{\phi}_{r}^{s}=\displaystyle{\arg\min_{\phi_{r}}}\sum_{i=l_{r-1}+1}^{l_{r}}\Big{(}(Y_{i}-X_{i}\phi_{r})^{2})+\frac{\lambda_{n,(l_{r-1},l_{r})}}{l_{r}-l_{r-1}}\sum_{u=1}^{P}|\phi_{r,u}|^{\gamma}\Big{)},\quad\forall r=1,...,k+1,

and

θ^1s=arg⁡minθ1⁡S​(l1,…,lk).\hat{\theta}_{1}^{s}=\displaystyle{\arg\min_{\theta_{1}}}S(l_{1},...,l_{k}).

Note that, for γ=1\gamma=1 and γ=2\gamma=2, we obtain the LASSO estimator and ridge estimator respectively.

The rest of this paper is organized as follows. Section 2 gives the main result of this paper, and in Section 3. The proof of the main result is given in the Appendix.

2 Main result

Lemma 2.1.

Under Assumptions (𝐇𝟐)(\bm{H_{2}}), (𝐇𝟑)(\bm{H_{3}}), for all n1n_{1}, n2∈Nn_{2}\in N, such that n1⩾nun_{1}\geqslant n^{u}, with 3/4≤u≤13/4\leq u\leq 1, n2≤nvn_{2}\leq n^{v}, v<1/4v<1/4, let be the model:

Yi\displaystyle Y_{i} =\displaystyle= Xi′​ϕ10+ϵi,i=1,…,n1\displaystyle X_{i}^{\prime}\phi_{1}^{0}+\epsilon_{i},\;i=1,...,n_{1}
Yi\displaystyle Y_{i} =\displaystyle= Xi′​ϕ20+ϵi,i=n1+1,…,n2,\displaystyle X_{i}^{\prime}\phi_{2}^{0}+\epsilon_{i},\;i=n_{1}+1,...,n_{2},

with ϕ10≠ϕ20\phi_{1}^{0}\neq\phi_{2}^{0}. We set An1+n2s​(ϕ)=∑i=1n1ηi;(0,n1)s​(ϕ,ϕ10)+∑i=n1+1n1+n2ηi;(n1,n1+n2)s​(ϕ,ϕ20)A_{n_{1}+n_{2}}^{s}(\phi)=\displaystyle{\sum_{i=1}^{n_{1}}}\eta_{i;(0,n_{1})}^{s}(\phi,\phi_{1}^{0})+\displaystyle{\sum_{i=n_{1}+1}^{n_{1}+n_{2}}}\eta_{i;(n_{1},n_{1}+n_{2})}^{s}(\phi,\phi_{2}^{0})   and
ϕ^n1+n2s=arg⁡minϕ⁡An1+n2s​(ϕ)\hat{\phi}_{n_{1}+n_{2}}^{s}=\displaystyle{\arg\min_{\phi}}A_{n_{1}+n_{2}}^{s}(\phi). Let δ∈(0,u−3​v)\delta\in(0,u-3v). Then,

  1. (i)

    ‖ϕ^n1+n2s−ϕ10‖≤n−(u−v−δ)/2||\hat{\phi}_{n_{1}+n_{2}}^{s}-\phi_{1}^{0}||\leq n^{-(u-v-\delta)/2}.

  2. (ii)

    If ϕ20=ϕ10+ϕ30​n−1/4\phi_{2}^{0}=\phi_{1}^{0}+\phi_{3}^{0}\,n^{-1/4} for some ϕ30\phi_{3}^{0}, then

    ∑i=1n1ηi;(0,n1)s​(ϕn1+n2s,ϕ10)=Op​(1).\displaystyle{\sum_{i=1}^{n_{1}}}\eta_{i;(0,n_{1})}^{s}(\phi_{n_{1}+n_{2}}^{s},\phi_{1}^{0})=O_{p}(1).
Remark 2.1.

It should be noted that, although Part (i) of the above lemma is the same as that of lemma 3 of Ciuperca (2012), Part (ii) is slightly different. The established result holds if ϕ20=ϕ10+ϕ30​n−1/4\phi_{2}^{0}=\phi_{1}^{0}+\phi_{3}^{0}\,n^{-1/4}, while the result stated in Ciuperca (2012) is supposed to hold for all ϕ20≠ϕ10\phi_{2}^{0}\neq\phi_{1}^{0}, but with incorrect proof. So far, we are neither able to correct the proof for all ϕ20≠ϕ10\phi_{2}^{0}\neq\phi_{1}^{0} nor to prove that the statement itself is wrong. Similarly, Part (ii) of Lemmas 4 and 8 hold under the condition that ϕ20=ϕ10+ϕ30​n−1/4\phi_{2}^{0}=\phi_{1}^{0}+\phi_{3}^{0}\,n^{-1/4}.

3 Concluding Remark

In this paper, we proposed a modification of Part (ii) of Lemma 3 given in Ciuperca (2012) for which the proof is wrong. Further, we provided the correct proof. It should be noted that there are several important results in the quoted paper which were established by using Lemma 3. In particular, the quoted author used this lemma in establishing Lemmas 4 and 8, as well as Theorems 1, 2 and 4.

Appendix A Appendix

Proof of Lemma 2.1.
  1. (i)

    The proof of Part (i) is similar to that in Ciuperca (2012).

  2. (ii)

    Let Zn​(ϕ)=∑i=1n1ηi​(ϕ,ϕ10)Z_{n}(\phi)=\displaystyle{\sum_{i=1}^{n_{1}}}\eta_{i}(\phi,\phi_{1}^{0}), tn​(ϕ)=∑i=n1+1n1+n2[(ϵi−Xi′​(ϕ−ϕ20))2−(ϵi−Xi′​(ϕ1−ϕ20))2]t_{n}(\phi)=\displaystyle{\sum_{i=n_{1}+1}^{n_{1}+n_{2}}}[(\epsilon_{i}-X_{i}^{\prime}(\phi-\phi_{2}^{0}))^{2}-(\epsilon_{i}-X_{i}^{\prime}(\phi_{1}-\phi_{2}^{0}))^{2}]. Then,

    |tn​(ϕ^n1+n2)|\displaystyle|t_{n}(\hat{\phi}_{n_{1}+n_{2}})| =\displaystyle= |−2∑i=n1+1n1+n2εiXi′(ϕ^n1+n2−ϕ10)+(ϕ^n1+n2−ϕ20)′∑i=n1+1n1+n2XiXi′(ϕ^n1+n2−ϕ20)\displaystyle|-2\sum_{i=n_{1}+1}^{n_{1}+n_{2}}\varepsilon_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0})+(\hat{\phi}_{n_{1}+n_{2}}-\phi_{2}^{0})^{\prime}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{2}^{0})
    −(ϕ10−ϕ20)′∑i=n1+1n1+n2XiXi′(ϕ10−ϕ20)|.\displaystyle-(\phi_{1}^{0}-\phi_{2}^{0})^{\prime}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}(\phi_{1}^{0}-\phi_{2}^{0})|.

    Since ‖ϕ^n1+n2−ϕ10‖≤n−(u−v−δ)/2||\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0}||\leq n^{-(u-v-\delta)/2}, by Cauchy-Schwarz inequality, we have

    |∑i=n1+1n1+n2εi​Xi′​(ϕ^n1+n2−ϕ10)|⩽O​(nv/2​n−(u−v−δ)/2)=o​(1).|\sum_{i=n_{1}+1}^{n_{1}+n_{2}}\varepsilon_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0})|\leqslant O(n^{v/2}n^{-(u-v-\delta)/2})=o(1).

    Further, let λmax\lambda_{\max} be the largest eigenvalue of 1n2​∑i=n1+1n1+n2Xi​Xi′\frac{1}{n_{2}}\displaystyle{\sum_{i=n_{1}+1}^{n_{1}+n_{2}}}X_{i}X_{i}^{\prime}. Then, using the fact that ϕ20=ϕ10+ϕ30​n−1/4\phi_{2}^{0}=\phi_{1}^{0}+\phi_{3}^{0}n^{-1/4} and Cauchy-Schwarz inequality, we have

    (ϕ^n1+n2−ϕ20)′​∑i=n1+1n1+n2Xi​Xi′​(ϕ^n1+n2−ϕ20)\displaystyle(\hat{\phi}_{n_{1}+n_{2}}-\phi_{2}^{0})^{\prime}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{2}^{0})
    =\displaystyle= (ϕ^n1+n2−ϕ10)′​n2​1n2​∑i=n1+1n1+n2Xi​Xi′​(ϕ^n1+n2−ϕ10)−2​ϕ30′​n−1/4​n2​1n2​∑i=n1+1n1+n2Xi​Xi′​(ϕ^n1+n2−ϕ10)\displaystyle(\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0})^{\prime}n_{2}\frac{1}{n_{2}}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0})-2\phi_{3}^{0^{\prime}}n^{-1/4}n_{2}\frac{1}{n_{2}}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0})
    +n−1/2​ϕ30′​n2​1n2​∑i=n1+1n1+n2Xi​Xi′​ϕ30\displaystyle+n^{-1/2}\phi_{3}^{0^{\prime}}n_{2}\frac{1}{n_{2}}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}\phi_{3}^{0}
    ⩽\displaystyle\leqslant n2​λmax​‖ϕ^n1+n2−ϕ10‖2+2​‖ϕ30‖​n−1/4​n2​λmax​‖ϕ^n1+n2−ϕ10‖+n−1/2​n2​λmax​‖ϕ30‖2,\displaystyle n_{2}\lambda_{\max}||\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0}||^{2}+2||\phi_{3}^{0}||n^{-1/4}n_{2}\lambda_{\max}||\hat{\phi}_{n_{1}+n_{2}}-\phi_{1}^{0}||+n^{-1/2}n_{2}\lambda_{\max}||\phi_{3}^{0}||^{2},

    and then,

    (ϕ^n1+n2−ϕ20)′​∑i=n1+1n1+n2Xi​Xi′​(ϕ^n1+n2−ϕ20)=O​(n(u−v−δ)​nv)+o​(1)+O​(nv−1/2)=o​(1).\displaystyle(\hat{\phi}_{n_{1}+n_{2}}-\phi_{2}^{0})^{\prime}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}(\hat{\phi}_{n_{1}+n_{2}}-\phi_{2}^{0})=O(n^{(u-v-\delta)}n^{v})+o(1)+O(n^{v-1/2})=o(1).

    Also, we have

    (ϕ10−ϕ20)′​∑i=n1+1n1+n2Xi​Xi′​(ϕ10−ϕ20)=n−1/2​ϕ30′​∑i=n1+1n1+n2Xi​Xi′​ϕ30=O​(nv−1/2)=o​(1).\displaystyle({\phi}_{1}^{0}-\phi_{2}^{0})^{\prime}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X^{\prime}_{i}(\phi_{1}^{0}-\phi_{2}^{0})=n^{-1/2}\phi_{3}^{0^{\prime}}\sum_{i=n_{1}+1}^{n_{1}+n_{2}}X_{i}X_{i}^{\prime}\phi_{3}^{0}=O(n^{v-1/2})=o(1).

    Therefore, |tn​(ϕ^n1+n2)|=op​(1)|t_{n}(\hat{\phi}_{n_{1}+n_{2}})|=o_{p}(1). Further, since
    Zn​(ϕ10)=tn​(ϕ10)=0Z_{n}(\phi_{1}^{0})=t_{n}(\phi_{1}^{0})=0, Zn​(ϕ^n1+n2)+tn​(ϕ^n1+n2)⩽Zn​(ϕ10)+tn​(ϕ10)Z_{n}(\hat{\phi}_{n_{1}+n_{2}})+t_{n}(\hat{\phi}_{n_{1}+n_{2}})\leqslant Z_{n}(\phi_{1}^{0})+t_{n}(\phi_{1}^{0}), we have

    0⩾zn​(ϕ^n1+n2)+tn​(ϕ^n1+n2)⩾infϕZn​(ϕ)−|tn​(ϕ^n1+n2)|=infϕZn​(ϕ)−|op​(1)|.0\geqslant z_{n}(\hat{\phi}_{n_{1}+n_{2}})+t_{n}(\hat{\phi}_{n_{1}+n_{2}})\geqslant\inf_{\phi}Z_{n}(\phi)-|t_{n}(\hat{\phi}_{n_{1}+n_{2}})|=\inf_{\phi}Z_{n}(\phi)-|o_{p}(1)|.

    Hence

    |zn​(ϕ^n1+n2)|−|tn​(ϕ^n1+n2)|⩽|zn​(ϕ^n1+n2)+tn​(ϕ^n1+n2)|⩽|infϕZn​(ϕ)|+op​(1),|z_{n}(\hat{\phi}_{n_{1}+n_{2}})|-|t_{n}(\hat{\phi}_{n_{1}+n_{2}})|\leqslant|z_{n}(\hat{\phi}_{n_{1}+n_{2}})+t_{n}(\hat{\phi}_{n_{1}+n_{2}})|\leqslant|\inf_{\phi}Z_{n}(\phi)|+o_{p}(1),

    which implies that

    |zn​(ϕ^n1+n2)|⩽|infϕZn​(ϕ)|+op​(1)+|tn​(ϕ^n1+n2)|=|infϕZn​(ϕ)|+op​(1).|z_{n}(\hat{\phi}_{n_{1}+n_{2}})|\leqslant|\inf_{\phi}Z_{n}(\phi)|+o_{p}(1)+|t_{n}(\hat{\phi}_{n_{1}+n_{2}})|=|\inf_{\phi}Z_{n}(\phi)|+o_{p}(1).

    Let ϕ^n1=arg⁡minϕ⁡Zn​(ϕ)\hat{\phi}_{n_{1}}=\arg\min_{\phi}Z_{n}(\phi) and λmax\lambda_{\max} be the largest eigenvalue of n1−1​∑i=1n1Xi​Xi′n_{1}^{-1}\sum_{i=1}^{n_{1}}X_{i}X^{\prime}_{i}. Then, by Cauchy-Schwarz inequality,

    infϕZn​(ϕ)⩽(n1​‖ϕ^n1−ϕ10‖)2​λmax+2​n1​|(ϕ^n1−ϕ10)′​n1−1/2​∑i=1n1εi​Xi|,\displaystyle\inf_{\phi}Z_{n}(\phi)\leqslant\left(\sqrt{n_{1}}\left\|\hat{\phi}_{n_{1}}-\phi_{1}^{0}\right\|\right)^{2}\lambda_{\max}+2\sqrt{n_{1}}\left|(\hat{\phi}_{n_{1}}-\phi_{1}^{0})^{\prime}n_{1}^{-1/2}\sum_{i=1}^{n_{1}}\varepsilon_{i}X_{i}\right|,

    and then,

    infϕZn​(ϕ)=Op​(1)+Op​(1)​Op​(1)=Op​(1), and |zn​(ϕ^n1+n2)|=Op​(1).\displaystyle\inf_{\phi}Z_{n}(\phi)=O_{p}(1)+O_{p}(1)O_{p}(1)=O_{p}(1),\quad{}\mbox{ and }\quad{}|z_{n}(\hat{\phi}_{n_{1}+n_{2}})|=O_{p}(1).

    Now, let

    Zns​(ϕ)\displaystyle Z_{n}^{s}(\phi) =\displaystyle= ∑i=1n1ηi​(ϕ,ϕ10)+λn;(0,n1)​[∑k=1p(|ϕ,k|γ−|ϕ1,k0|γ)]\displaystyle\sum_{i=1}^{n_{1}}\eta_{i}(\phi,\phi_{1}^{0})+\lambda_{n;(0,n_{1})}[\sum_{k=1}^{p}(|\phi_{,k}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})]
    tns​(ϕ)\displaystyle t_{n}^{s}(\phi) =\displaystyle= ∑i=n1+1n1+n2[(ϵi−Xi′​(ϕ−ϕ20))2−(ϵi−Xi′​(ϕ1−ϕ20))2]\displaystyle\sum_{i=n_{1}+1}^{n_{1}+n_{2}}[(\epsilon_{i}-X_{i}^{\prime}(\phi-\phi_{2}^{0}))^{2}-(\epsilon_{i}-X_{i}^{\prime}(\phi_{1}-\phi_{2}^{0}))^{2}]
    +λn;(n1,n1+n2)​[∑k=1p(|ϕ,k|γ−|ϕ1,k0|γ)].\displaystyle\quad{}+\lambda_{n;(n_{1},n_{1}+n_{2})}[\sum_{k=1}^{p}(|\phi_{,k}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})].

    Then,

    An1+n2s​(ϕ)=Zns​(ϕ)+tns​(ϕ)−(ϵi−Xi′​(ϕ1−ϕ20))2+λn;(n1,n1+n2)​[∑k=1p(|ϕ1,k0|γ−|ϕ2,k0|γ)].A_{n_{1}+n_{2}}^{s}(\phi)=Z_{n}^{s}(\phi)+t_{n}^{s}(\phi)-(\epsilon_{i}-X_{i}^{\prime}(\phi_{1}-\phi_{2}^{0}))^{2}+\lambda_{n;(n_{1},n_{1}+n_{2})}[\sum_{k=1}^{p}(|\phi_{1,k}^{0}|^{\gamma}-|\phi_{2,k}^{0}|^{\gamma})].

    Then ϕ^n1+n2s=arg⁡minϕ⁡(Zns​(ϕ)+tns​(ϕ))=arg⁡minϕ⁡An1+n2s​(ϕ)\hat{\phi}_{n_{1}+n_{2}}^{s}=\arg\min_{\phi}(Z_{n}^{s}(\phi)+t_{n}^{s}(\phi))=\arg\min_{\phi}A_{n_{1}+n_{2}}^{s}(\phi). In addition, using the similar approach as previous, with the fact that ‖ϕ^n1+n2s−ϕ10‖⩽n−(u−v−δ)/2||\hat{\phi}_{n_{1}+n_{2}}^{s}-\phi_{1}^{0}||\leqslant n^{-(u-v-\delta)/2} and ϕ20=ϕ10+ϕ30​n−1/4\phi_{2}^{0}=\phi_{1}^{0}+\phi_{3}^{0}n^{-1/4}, we have

    |tns​(ϕ^n1+n2s)|\displaystyle|t_{n}^{s}(\hat{\phi}_{n_{1}+n_{2}}^{s})| ⩽\displaystyle\leqslant o​(1)+λn;(n1,n1+n2)​[∑k=1p(|ϕ^n1+n2,ks|γ−|ϕ1,k0|γ)]\displaystyle o(1)+\lambda_{n;(n_{1},n_{1}+n_{2})}[\sum_{k=1}^{p}(|\hat{\phi}_{n_{1}+n_{2},k}^{s}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})]
    =\displaystyle= op​(1)+O​(nv/2)​Op​(‖ϕ^n1+n2s−ϕ10‖)=Op​(n−(u−2​v−δ)/2)=op​(1).\displaystyle o_{p}(1)+O(n^{v/2})O_{p}(||\hat{\phi}_{n_{1}+n_{2}}^{s}-\phi_{1}^{0}||)=O_{p}(n^{-(u-2v-\delta)/2})=o_{p}(1).

    Besides, Zns​(ϕ10)=tns​(ϕ10)=0Z_{n}^{s}(\phi_{1}^{0})=t_{n}^{s}(\phi_{1}^{0})=0, thus

    0⩾infϕ(Zns​(ϕ10)+tns​(ϕ10))=Zns​(ϕ^n1+n2s)+tns​(ϕ^n1+n2s)=Zns​(ϕ^n1+n2s)−|op​(1)|\displaystyle 0\geqslant\inf_{\phi}(Z_{n}^{s}(\phi_{1}^{0})+t_{n}^{s}(\phi_{1}^{0}))=Z_{n}^{s}(\hat{\phi}_{n_{1}+n_{2}}^{s})+t_{n}^{s}(\hat{\phi}_{n_{1}+n_{2}}^{s})=Z_{n}^{s}(\hat{\phi}_{n_{1}+n_{2}}^{s})-|o_{p}(1)|
    ⩾infϕZns​(ϕ)−|op​(1)|.\displaystyle\geqslant\inf_{\phi}Z_{n}^{s}(\phi)-|o_{p}(1)|.

    Hence

    |Zns​(ϕ^n1+n2s)|⩽|infϕZns​(ϕ)|+op​(1).|Z_{n}^{s}(\hat{\phi}_{n_{1}+n_{2}}^{s})|\leqslant|\inf_{\phi}Z_{n}^{s}(\phi)|+o_{p}(1).

    On the other hand, since

    0⩾infϕZns​(ϕ)⩾infϕZn​(ϕ)+λn;(0,n1)​infϕ[∑k=1p(|ϕ,k|γ−|ϕ1,k0|γ)],0\geqslant\inf_{\phi}Z_{n}^{s}(\phi)\geqslant\inf_{\phi}Z_{n}(\phi)+\lambda_{n;(0,n_{1})}\inf_{\phi}[\sum_{k=1}^{p}(|\phi_{,k}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})],
    |infϕZns​(ϕ)|⩽|infϕZn​(ϕ)|+|λn;(0,n1)​infϕ[∑k=1p(|ϕ,k|γ−|ϕ1,k0|γ)]|.|\inf_{\phi}Z_{n}^{s}(\phi)|\leqslant|\inf_{\phi}Z_{n}(\phi)|+|\lambda_{n;(0,n_{1})}\inf_{\phi}[\sum_{k=1}^{p}(|\phi_{,k}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})]|.

    Further,

    infϕ[∑k=1p(|ϕ,k|γ−|ϕ1,k0|γ)]⩽∑k=1p(|ϕ^n1,k|γ−|ϕ1,k0|γ)=Op​(‖ϕ^n1−ϕ10‖)=Op​(n1−1/2),\inf_{\phi}[\sum_{k=1}^{p}(|\phi_{,k}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})]\leqslant\sum_{k=1}^{p}(|\hat{\phi}_{n_{1},k}|^{\gamma}-|\phi_{1,k}^{0}|^{\gamma})=O_{p}(||\hat{\phi}_{n_{1}}-\phi_{1}^{0}||)=O_{p}(n_{1}^{-1/2}),

    and infϕZn​(ϕ)=Op​(1)\inf_{\phi}Z_{n}(\phi)=O_{p}(1). It follows that |infϕZns​(ϕ)|⩽Op​(1)|\inf_{\phi}Z_{n}^{s}(\phi)|\leqslant O_{p}(1). Hence,

    |Zns​(ϕ^n1+n2s)|⩽|infϕZns​(ϕ)|+op​(1)=Op​(1).|Z_{n}^{s}(\hat{\phi}_{n_{1}+n_{2}}^{s})|\leqslant|\inf_{\phi}Z_{n}^{s}(\phi)|+o_{p}(1)=O_{p}(1).

∎

References

  • [1] Ciuperca, G. (2012). Model selection by LASSO methods in a change-point model. Stat Papers, DOI: 10.1007/s00362-012-0482-x (in press).